Custom Benchmark

I wrote a specific test case just to test Bind operations and added it to the slamd suite under ApacheDS Optimizaion Tests.

Unrealistic Benchmark This is not a real world test.

This benchmark simply let's us regression test changes to the Bind handling pathway to see the effects of our changes in a clean fashion. We can also use the test to profile the server. A description of the test and it's parameters can be found here.

This test as you'll see soon enough tells us some very interesting things about the server. This is because it isolates the bind code from the rest of the server. The same connection is reused to reissue bind requests over and over again. Even the same user is reused so the entry cache does not play into the picture.

BTW here's the output of a profiling session with Yourkit 5.5.x.  BTW you can get an HTML dump of the call tree with the most expensive pathway expanded here.

Baseline Runs

Here's a look at a baseline run. You might like to take a look at it within SLAMD herewith the guest/guest account. Below is the overlaid plot of the average bind requests per second processed with a successful bind response.

So this is cool! ApacheDS is on average processing 1785 BindRequests per second, authenticating the user binding, and returning a BindResult back to the client.

Fastest Authentication The fastest way to authenticate a user is to just try to Bind as that user to the DS. However you're client should reuse whatever connection it has to the server instead of creating a new connection every time. Also it might be a good idea to pool several connecions to issue bind requests across them. This way a multithreaded application can get lightning fast authentications using LDAP. Note that a low level LDAP API other than generalized JNDI is needed to do this properly.

Here are the baselines for older ApacheDS releases:

Release Revision	Test Number	ave. req/sec	ave. time per req
1.0-RC1	20060718102847-09325054	961	73 (ms)
1.0-RC2	20060718102028-12008453	814	99 (ms)
1.0-RC3	20060718095911-00514552	784	103 (ms)

As you can see we've (Emmanuel really has) more than doubled the performance for apacheds since the start of the RC series.

Always Returning Success

Here's an interesting scenario. What if ApacheDS did not tap into it's backend to pull lookup the user entry to authenticate it? What if ApacheDS just returned a SUCCESS BindResponse back to every client? I tried this and here are the results:

Yeah that's pretty amazing but it makes sense. 3917.307 BindRequests with a SUCCESS BindResponse sent every time. It's over twice the baseline. This shows us the upper limit with only the ASN.1 LDAP codec and MINA in the picture.

Baselines for OpenLDAP

Just as a reference let's take a look at OpenLDAP 2.3 baseline for this benchmark.

ApacheDS w/ Credential Cache

As a test I enabled a credential cache within the SimpleAuthenticator to see how much of a boost we'd get. It turns out that performance increases by almost 500 requests per second. Before the optimization we were experiencing a rate of 1785.438 requests per second. After the optimization we were experiencing a rate of 2260.703 requests per second. This certainly is a performance boost about 27% according to the standard deviation observed. Below is the new graph of rate verses the amount of time for each authentication:

Gotta thank Noel for this recommendation! Way to go!

How do we stack up against our past?