Security manager benchmarking

[Gary Benson <gbenson at redhat dot com>]

Hi all,

Andrew Haley asked me about the Tomcat benchmark I've been using to
identify bottlenecks caused by enabling a security manager.  I thought
I'd post the details here in case anyone else is interested in them.

The short story is I build gcj with the attached patch which allows
user code to disable parts of the security architecture.  I run Tomcat
with this, BC compiled, and with a security manager enabled.  I use
the attached webapp to select how much of the security manager I want
to use, and I run ApacheBench (the Apache HTTP server benchmarking
tool) on it.

The details (somewhat Fedora specific):
 Building gcj:
   Apply gcj-benchmark-hacks.patch and build :) FWIW I installed it in
   /opt/gcc and did rpm -e libgcj-devel to make sure I wasn't picking
   up anything from that.  After you install you have to copy
   /usr/lib/security/*.security into /opt/gcc/lib/security.

 Building java-1.4.2-gcj-compat:
   I built a custom java-1.4.2-gcj-compat rpm using jgc-hacks.patch.

 Installing Tomcat:
   yum install tomcat5-webapps tomcat5-admin-webapps
   Apply tomcat5-init.patch to /etc/init.d/tomcat5.

   I'm not sure if it's necessary but I rebuilt all the BC libraries
   with my custom gcj.  Note that gcj's make install will trash your
   classmap.db, so you need to re-run rebuild-gcj-db every time.

 Installing the webapp:
   You can't deploy using the manager on Fedora unfortunatly, so this
   is a manual thing.

     mkdir /var/lib/tomcat5/webapps/test
     cd $_
     jar xf test.war

 Start Tomcat:
   service tomcat5 start

 Benchmark:
   Check the webapp is running with GET http://localhost:8080/test
   Change settings with GET http://localhost:8080/test?command=clear etc
     (or just use a browser)
   Benchmark with ab -c10 -n100000 http://localhost:8080/test
   The first run should be ignored as: it will be significantly slower
   because the server will not have warmed up.

The different settings are:
  Security manager is set / AccessController is enabled
    This is normal "with a security manager" operation

  Security manager is null / AccessController is enabled
    This is normal "without a security manager" operation

  Security manager is set / AccessController is disabled
    This disables AccessController.getContext().  The drop in
    performance between this and "set/enabled" is due to all the
    stuff that happens in all the if (securitymanager != null)
    blocks.

  Security manager is null / AccessController is disabled
    This is pretty meaningless.
    
Enjoy :)

Gary

Attachment: gcj-benchmark-hacks.patch
Description: Text document

Attachment: jgc-hacks.patch
Description: Text document

Attachment: tomcat5-init.patch
Description: Text document

Attachment: test.war
Description: Binary data