January 20, 2010 at 11:17 am (Java, Optimization, Spring)
The application I’m currently working on is built around Spring. A couple of weeks ago, as I waited for it to start up for the 27th time that day, I began wondering if there was any way to get it to start faster. Coincidentally, our whole development team had just received upgraded laptops, so the fact that numbers are stagnating in the “Megahertz” department (but not in the “CPU cores” department) had been recently emphasized.
Further, I had just read a series of articles by Cédric Beust regarding recent improvements to the TestNG threading engine. In short, TestNG allows you to run tests on multiple threads in order to shorten the test suite run time. However, when you start creating tests with dependencies on other tests, TestNG punts and runs all of these groups of dependent tests serially, on a single thread. Cédric wanted to remove this limitation, and approached the enhancement as a topological sort problem, viewing the tests as nodes in a directed acyclic graph.
Most of the startup time in Spring-centric applications involves the initialization of singleton beans. It wasn’t too much of a mental leap to wonder if the same graph/sort approach could be applied to the application startup optimization question — with the Spring beans acting as the nodes in the graph, instead of the tests.
It turns out to be (sort of) possible, and (sort of) worth the time. I’ve posted the details, including the source, here. Feel free to check it out and make suggestions. The Spring framework does contain a very coarse lock that required a pretty horrible hack in order to get any performance gains out of the experiment, but once this is worked out I don’t see why an implementation of this approach couldn’t make it into real applications. Again, see the project page for more details.
On a side note, this is the first time that I’ve used GitHub to host any code, and I have to say that I really like the fact that the README file serves the dual purposes of local project documentation and homepage markup!
Leave a Comment
September 3, 2009 at 12:03 pm (Uncategorized)
HtmlUnit 2.6 has just been released! A ton of work has gone into this release, so I hope people find it useful.
Some highlights:
- Marc Guillemot tweaked the caching infrastructure a bit, which should result in improved performance, especially for those of you hitting remote websites for scraping and such.
- Ahmed Ashour has fixed quite a few bugs in a focused effort to make it easier for the GWT folks to incorporate HtmlUnit.
- Our WebClient serialization support was very broken in 2.5; this has now been fixed.
- Many, many more bugfixes!
Of course, HtmlUnit continues to support use with a wide variety of JavaScript libraries.
Give it a try and let us know what you think!
2 Comments
August 5, 2009 at 10:13 pm (Uncategorized)
As per Simon’s post on the WebDriver mailing list, WebDriver and Selenium are starting the merge that will define Selenium 2.0… wasn’t Selenium 1.0 just released a couple of months ago? And didn’t they take something like 6 years to get to 1.0? Things are certainly speeding up! Anyone care to place a bet on how far out 2.0 is?
Leave a Comment
June 3, 2009 at 11:25 am (Uncategorized)
Ahmed Ashour and I will be talking about HtmlUnit today at 2:50 PM (room 301). We’ll be discussing browser driving tools and browser simulation tools, their pros and cons, and where HtmlUnit fits into the landscape. Stop by if you’re interested in web application integration testing in general, and HtmlUnit in particular!
Leave a Comment
March 17, 2009 at 11:15 pm (Uncategorized)
Leave a Comment
March 5, 2009 at 1:56 am (HtmlUnit, JRuby)
I have to say, JRuby is cool stuff.
About a year ago the guys at FINN.no decided to wrap HtmlUnit in a Watir-ish API; Celerity was born, and HtmlUnit was introduced into the Ruby ecosystem.
Now Celerity is itself being wrapped by Culerity, which integrates Celerity and Cucumber.
How cool is that?
1 Comment
February 18, 2009 at 11:17 pm (Humor)
Step 0: Write just enough code to actually qualify as an open source project. Maybe a little less.
Step 1: Hack together a project website that’s more about SEO than it is about promoting your project. Don’t forget to mention the Killer Ajax Framework which you developed in 2007.
Step 2: Host your project on SourceForge, and ask the SourceForge administrators to artificially inflate your project’s ranking. Because you rock, and they know it!
Step 3: When the SourceForge admins politely tell you to get lost, take matters into your own hands and artificially inflate your project’s ranking yourself by spamming your own forums. Number 1, baby!
Step 4: Take advantage of TheServerSide’s new and improved RubberStampTM editorial process by submitting an eloquent and compelling introduction to your project.
Step 5: Profit, you crazy diamond!
4 Comments
February 18, 2009 at 10:26 pm (Note to Self, Oracle)
To find all sessions for a specific user connected via the Oracle JDBC driver:
select * from v$session where username='X' and program like '%JDBC%';
To kill all of these database sessions:
declare
statement varchar(100);
cursor c is
select 'alter system kill session '''
|| sid || ',' || serial# || ''' immediate'
from v$session
where username='X'
and program like '%JDBC%';
begin
dbms_output.enable;
open c;
loop
fetch c into statement;
exit when c%notfound;
dbms_output.put_line('Executing: ' || statement);
execute immediate statement;
end loop;
close c;
end;
This will result in output similar to the following:
Executing: alter system kill session '225,46' immediate
Executing: alter system kill session '226,110' immediate
Executing: alter system kill session '233,6617' immediate
1 Comment
February 5, 2009 at 2:48 am (HtmlUnit, JavaScript, Rhino)
I’ve been examining HtmlUnit‘s performance from a couple of different angles lately. As a pure-Java headless browser intended for integration testing, one of HtmlUnit’s big draws is improved performance vis-a-vis native browsers and libraries which drive native browsers (Selenium, WebDriver, etc).
One the one hand, it’s easy to see that HtmlUnit reduces overhead by forgoing a GUI. No layouting, no drawing, no problem. If you poke around a little bit, you’ll also find that HtmlUnit does not download most images (there are some exceptions), nor does it download external CSS files if CSS has been disabled — all advantages in terms of network usage.
However, as you get closer to the RIA end of the web application spectrum, these performance advantages become increasingly overshadowed by JavaScript performance. HtmlUnit relies on Rhino to do the JavaScript heavy lifting behind the scenes, so as web applications become more functional, we’re going to be relying more and more on Rhino’s muscle.
Google just released version 3 of their V8 JavaScript Benchmark Suite, which tests pure JavaScript and pretty much ignores the DOM manipulation side of things — making it a perfect worst-case scenario benchmark with which to compare HtmlUnit to native browsers. In other words, if you use HtmlUnit such that all of its traditional performance advantages are negated (unlikely though that may be), how does it stack up against the native browsers?
Not too bad, as it turns out (bigger numbers are better):
The good news is that Rhino is more performant than IE 6 or IE 7, so HtmlUnit still beats these browsers in this unrealistic worst-case scenario.
The bad news is that IE is by far the slowest native browser out there in terms of JavaScript execution speed; we can’t assume that it will remain slow forever… can we?
4 Comments
January 10, 2009 at 9:47 pm (Java, Spring)
If you tell Spring 2.5.6 to scan the classpath for application context XML files and you’re deploying to JBoss 5, you may run into this bug; something to do with JBoss’ new Virtual File System (VFS) and its non-standard URLs. It looks like this issue was caught in time to be fixed for 2.5.6, but I guess the SpringSource guys didn’t have time to test it before releasing — the fix doesn’t really work.
It appears there will be a 2.5.7 release within the next couple of months (see slide 14), but it’s probably going to be the first of the commercial-only Spring releases. However, Spring 3.0 is also due within the next couple of months (see slide 5); let’s hope for a fix then. Otherwise, they may have to take JBoss off of the deploy graph on slide 4 
3 Comments
