I’ve been examining HtmlUnit’s performance from a couple of different angles lately. As a pure-Java headless browser intended for integration testing, one of HtmlUnit’s big draws is improved performance vis-a-vis native browsers and libraries which drive native browsers (Selenium, WebDriver, etc).
One the one hand, it’s easy to see that HtmlUnit reduces overhead by forgoing a GUI. No layouting, no drawing, no problem. If you poke around a little bit, you’ll also find that HtmlUnit does not download most images (there are some exceptions), nor does it download external CSS files if CSS has been disabled — all advantages in terms of network usage.
However, as you get closer to the RIA end of the web application spectrum, these performance advantages become increasingly overshadowed by JavaScript performance. HtmlUnit relies on Rhino to do the JavaScript heavy lifting behind the scenes, so as web applications become more functional, we’re going to be relying more and more on Rhino’s muscle.
Google just released version 3 of their V8 JavaScript Benchmark Suite, which tests pure JavaScript and pretty much ignores the DOM manipulation side of things — making it a perfect worst-case scenario benchmark with which to compare HtmlUnit to native browsers. In other words, if you use HtmlUnit such that all of its traditional performance advantages are negated (unlikely though that may be), how does it stack up against the native browsers?
Not too bad, as it turns out (bigger numbers are better):
The good news is that Rhino is more performant than IE 6 or IE 7, so HtmlUnit still beats these browsers in this unrealistic worst-case scenario.
The bad news is that IE is by far the slowest native browser out there in terms of JavaScript execution speed; we can’t assume that it will remain slow forever… can we?
Christophe Furmaniak said,
February 5, 2009 at 4:40 am
What about I.E8RC1 and Google Chrome? (sorry, I’m too lazy to benchmark them by myself :])
Simon Stewart said,
February 5, 2009 at 9:27 am
Just a quick note: WebDriver has always supported an HtmlUnitDriver mode, which offers the WebDriver interfaces backed by HtmlUnit.
Daniel Gredler said,
February 5, 2009 at 9:32 pm
@Christophe: I don’t have them installed, so I didn’t test them. Based on the numbers I’ve seen out there, I would assume IE8 to be incrementally better than IE7 (i.e. about even with Rhino) and Chrome to be an order of magnitude faster than IE.
@Simon: Yep, WebDriver straddles the fence in this regard, though my impression is that the HtmlUnit driver is somewhat of a second class citizen (JavaScript disabled by default, etc).
Andris said,
February 6, 2009 at 5:10 am
Whay Rhino is slow?