HtmlUnit vs HttpUnit
October 4, 2007 at 11:48 am (HtmlUnit, HttpUnit, Java)
There’s a lot of misinformation out there regarding web application test tools, so I’ve decided to post a series of short articles comparing some of the open source options available here in Java-land, circa 2007. The first of these articles will focus on HtmlUnit and HttpUnit. Please take my criticism and praise with a grain of salt, as I’m a committer to the HtmlUnit project and thus probably biased. Nevertheless, I will do my best to be objective. I may even overcompensate in the other direction!
Confusion
The HtmlUnit and HttpUnit projects are often confused due to the similarity of their names. And the similarity doesn’t end there: they are both open source projects; they are both 100% Java frameworks, rather than drivers for native browsers like IE or Firefox; and they are both fairly mature projects.
This confusion is compounded by the fact that many test frameworks which once used HttpUnit under the covers have since switched to using HtmlUnit, mainly in order to benefit from its excellent JavaScript support. Examples include JWebUnit, whose FAQ briefly explains the switch, and Canoo WebTest, which switched in 2004 due to JavaScript support issues and an unresponsive development team [1].
HttpUnit
HttpUnit is the granddaddy web app testing framework. Started in the summer of 2000 by Russ Gold [2], it was the first project to focus on this niche area. The project has since stagnated somewhat, with nearly 40% of bugs remaining open, some of them nearly three years old. Its latest maintenance release is about a year and a half old.
The API is fairly low-level, modeling web interactions at something approaching the HTTP request and response level. The following is a slightly modified example from the HttpUnit Cookbook:
WebConversation wc = new WebConversation();
WebResponse resp = wc.getResponse("http://www.google.com/");
WebLink link = resp.getLinkWith("About Google");
link.click();
WebResponse resp2 = wc.getCurrentPage();
As you can see, things center around WebConversations, WebRequests and WebResponses. Unfortunately, any page with a decent amount of JavaScript is likely to break HttpUnit, and you can absolutely forget testing any pages which use third party JavaScript libraries.
Nevertheless, HttpUnit continues to generate 3,000 to 4,000 downloads per month. A good analogy, if I may be allowed a brief subjective comment, is that HttpUnit is to the web app testing world what Struts is to the web app framework world: there are many “better” options out there, but it just won’t go away! 
HtmlUnit
HtmlUnit is itself a fairly old project, having been started by Mike Bowler in early 2002. Mike has since ceased active development, but the project currently boasts 3 or 4 active developers and a total of seven committers (whereas HttpUnit remains a one-man show). It averages about three releases per year, and has seen increased developer activity in the past six months or so, especially in the area of JavaScript support.
HtmlUnit’s API is a bit more high-level than HttpUnit’s, modeling web interaction in terms of the documents and interface elements which the user interacts with:
WebClient wc = new WebClient();
HtmlPage page = (HtmlPage) wc.getPage("http://www.google.com");
HtmlForm form = page.getFormByName("f");
HtmlSubmitInput button = (HtmlSubmitInput) form.getInputByName("btnG");
HtmlPage page2 = (HtmlPage) button.click();
As you can see, the code centers around WebClients, as well as pages, links, forms, buttons, etc. Pages with a modicum of custom JavaScript will probably work when tested with HtmlUnit. Unfortunately, pages which use third party libraries might or might not work when tested via HtmlUnit. As of the current version, Prototype, Script.aculo.us, DWR and jQuery are known to be supported fairly well, Dojo is a bit of an unknown, YUI is known to be unsupported, and GWT is known to work with fairly simple applications. Most of this compatibility has been achieved in the past two or three releases, so obviously things are fairly fluid.
Conclusion
If you’re using HttpUnit for legacy reasons, it’s a fairly solid package, but don’t expect to get much support when you need to report a bug or submit a patch for a new feature. If you’re starting a new project and are trying to decide between these two frameworks, HtmlUnit wins hands down. It has the features, the community and the momentum.
Of course, if you’re considering web application testing tools, you’re probably looking at more than just these two options. Canoo WebTest, TestMaker, JWebUnit, Selenium, WebDriver and JMeter are all likely to be on your list. Depending on your project budget and requirements, Squish and Mercury QTP may also be under consideration. If that’s the case, stay tuned, because I intend to post a series of web app testing framework comparisons in the coming months — all of them involving HtmlUnit, of course!
[1] It’s interesting to note that both Marc Guillemot and I (two HtmlUnit committers) began by using HttpUnit, submitting patches for missing features — but settled on HtmlUnit when the patches were not applied in a timely manner.
[2] The HttpUnit website states that Russ currently works for Oracle, developing the OC4J application server. Coincidentally, this is the production application server we’re using at my day job. Thanks, Russ! 
HtmlUnit vs HttpUnit « Marc Guillemot’s blog said,
October 11, 2007 at 4:32 am
[...] Daniel Gredler blog post: HtmlUnit vs HttpUnit. [...]
Simon R said,
October 12, 2007 at 10:57 am
We’re moving our tests to the new HtmlUnit, having had the same frustrations with HttpUnit - however I was frustrated that HtmlUnit didn’t provide an equivalent of HttpUnit’s in process testing capability (i.e. ServletUnit). I’d really rather not test my apps out-of-process if I can at all help it. So I’ve wired up HtmlUnit as the browser emulator and ServletUnit as the in process servlet runner - if anyone wants details, please let me know - nospamsimonrohrer@hotmail.com.
Assertions in HtmlUnit « A Public Scratchpad said,
October 15, 2007 at 7:30 pm
[...] back through the pros and cons of JWebUnit as part of my research for the HtmlUnit vs Foo series of articles I’m writing. One of JWebUnit’s big draws is the set of easy-to-use assertion methods [...]
Why crawling doesn’t matter | tssci security said,
December 2, 2007 at 3:42 pm
[...] the history and other detailed information can be read about on this wonderful blog entry entitled HtmlUnit vs. HttpUnit (a must read!). That article is also very good to hear about support of external Javascript [...]
Vinny Carpenter’s blog » Daily del.icio.us for January 13th through January 14th said,
January 14, 2008 at 7:30 pm
[...] HtmlUnit vs HttpUnit « A Public Scratchpad - If you?re using HttpUnit for legacy reasons, it?s a fairly solid package, but don?t expect to get much support. If you?re starting a new project and are trying to decide between these two frameworks, HtmlUnit wins hands down. It has the features, Tags: , , ADA, advocacy, ajax, algorithms, apache, api, article, bigdecimal, browser, calendar, cms, code, collaboration, color, ComputerScience, criptaculous, curriculum, data, database, date, design, development, dhtml, ecm, education, effects, framework, gallery, geek, graphics, howto, html, htmlunit, httpunit, Humor, ie, infoq, interesting, java, javascript, JMeter, JWebUnit, layout, learning, library, lightbox, Linux, mashup, mathematics, microsoft, money, opensource, performance, php, presentation, programming, prototype, purifier, puzzles, scraping, scripting, scripts, security, Selenium, sxhtml, testing, tools, tweaks, ubuntu, web, Web2.0, webdesign, WebDriver, webservices, workflow, xml, xslt, xss [...]
Senthil said,
January 27, 2008 at 11:27 am
Hi Daniel,
I am a Tester, and we have decided to write automated functionals test using HtmlUnit. We are using eclipse and we have added HtmlUnit jar files to the libraries. We are testing a secure site. however, we are unable to invoke the application using :
final WebClient webClient = new WebClient();
final HtmlPage page = (HtmlPage) webClient.getPage(”https:/….com/”);
assertEquals(”HtmlUnit - Welcome to HtmlUnit”, page.getTitleText());
anyone who can help us.. ksenthil_77@yahoo.com
Daniel Gredler said,
January 27, 2008 at 2:27 pm
Hi Senthil,
Please post a description of your problem, including a stack trace (if you have one) to HtmlUnit’s user mailing list:
http://sourceforge.net/mail/?group_id=47038
or
http://www.nabble.com/HtmlUnit—General-f2599.html
Christian Hall said,
February 6, 2008 at 3:00 pm
Is there any plan to include what Simon suggests within HtmlUnit?
Daniel Gredler said,
February 6, 2008 at 11:16 pm
Christian: Not at the moment. You may want to chime in on the dev list though, and see what others think: http://www.nabble.com/HtmlUnit—Dev-f2598.html
Juuro said,
March 18, 2008 at 9:16 am
Hi!
Thanks your article helped me very very!
Best regards, Juuro
WolfgangFahl said,
April 1, 2008 at 12:50 pm
Some of the statements here are out of date. Since mid 2007 I’m a committer on the httpunit project. So it’s not a one man show at this time.
First I should say that I’ve been sharing your frustration about how httpunit was handled since mid 2004. I’ve been sticking to my own patched version until the middle of 2007 when I finally decided to ask Russell for committer
rights which he kindly assigned to me.
I’ve worked hard to get the patches of some 3 years into the project and I’m done with that. I’ve done some 200 tracker items in the last three days alone and version 1.7 and 2.0 are due soon. The javascript support of httpunit is being improved with a new DOM scripting engine. There are over 750 running test cases at this time.
From my point of view it’s a pitty that there is a split of the developer resources around the unit testing issue of html pages. I don’t know what Russell’s opinion on this is.
Feel free to contact me via the contact form on my webpage or the e-mail shown there.
Marc Guillemot said,
April 2, 2008 at 5:58 am
Wolfgang,
interesting to see that HttpUnit is not fully dead. On the other hand I don’t expect so much progress particularly in JS support, no matter how many energy you deploy: HttpUnit current JS support capabilities are approximatively comparable to what HtmlUnit could do in 2004. Looking back at its state in 2004 I can say that it represents only a small part of the current HtmlUnit JS support capabilities. 750 running tests is not really a high number: HtmlUnit has currently over 3300 running tests (some of them are large test suites of AJAX libraries) and this is surely not enough.
I agree with you that a collaboration could allow to progress faster nevertheless I’ve already contacted you for a few days… but without response.
WolfgangFahl said,
May 13, 2008 at 5:40 am
Thank you Daniel and Marc for getting in touch.
I wonder what current httpunit users are saying. I did not try out htmlunit myself yet so I can’t state anything yet about how the two projects compare.
Let’s see what happens during the upcoming months.
Sargon Benjamin said,
May 22, 2008 at 8:26 pm
Thanks for the post and great comments! In response to Wolfgang, I’m currently a BIG fan of httpunit. Its easy to add to and to build a framework around.
My web app under test doesn’t use javascript at all so I am lucky in the regard. The one nicety is that httpunit lets me know the content type of various response pages (text/html or text/xml) and I can then do further processing there. However, httpunit can’t provide any verification features (eg getElementNames() or verifyTextPresent() ) on pages whose content type is ‘text/xml’. I wouldn’t expect it to either so I’m just using the Response.getText() and then doing some custom verification (XMLUnit and simple java regex).
I don’t have much reason to move to a new framework; however, I have been reading about JWebUnit and looking at its 2.0 features (Selenium integration seems nice). It would be nice to have Selenium scripts using the same framework I’m using for my non GUI tests and JWebUnit has the potential to do that. JWebUnit is built on top of HtmlUnit, atleast thats what I’m reading, and so I may just make the switch over to JWebUnit.
Is there anything that HTTPUnit can do that HtmlUnit / JWebUnit can’t do (I don’t use the ServletUnit functionality that Simon mentioned so thats a non issue for us. If Simon could elaborate on what he’s built, that would be nice too!)
Sargon Benjamin said,
May 23, 2008 at 1:30 am
I just noticed that HttpUnit 1.7 was released on May 20, 2008. I also noted that Wolfgang is a key committer and revived the HttpUnit project - kudos to you!
Oudated knowledge of “experts”: the example of HttpUnit « Marc Guillemot’s blog said,
May 23, 2008 at 7:02 am
[...] his blog post “HtmlUnit vs HttpUnit“, Daniel Gredler wrote “HttpUnit is to the web app testing world what Struts is to the [...]