BRIDGING JAVA OBJECTS AND RELATIONAL DATABASES

Important note: This document covers very early releases of OJB (0.1.30 - 0.1.48). Some technical details described here do not apply to the current releases of OJB. The test results presented here should be carefully checked against current releases.

EJB performance analysis

© Thomas Mahler, December 2000

abstract

As a part of a J2EE architecture project my team performed a performance analysis for IBM WebSphere.

Our aim was to study different persistence mechanisms under a variety of application scenarios in order to find a solution that provides the best compromise of performance, scalability and architectural “beauty”.

This article presents a rough overview of our approach and our test results.

Test-scenarios

We decided to implement real-world test-scenarios against an existing database. We used a database with the german postalcodes (270.000 rows) as a base for the following scenarios:

1. Scenario "find all postalcodes of the city 'Baden-Baden' and fully read each entry"

2. Scenario "find all postalcodes of the city ‘Berlin’ with streets named Hauptstrasse and fully read each entry"

3. Scenario "update all postalcode entries for city 'Baden-Baden'"

4. Scenario "Iterate through 1000 entries and update 32 entries"

5. Scenario "Iterate through 500 entries and update randomly selected entries"

These scenarios represent activities (and their distributions) that are typical for applications that we build for our customers. We did not test INSERT operations. Insert operations modify the index structure and may lead to performance degradations for later tests. Thus we ommited inserts to keeps our tests reproducable.

The study of relationships between entities was beyond the scope of the present analysis.

It is important to note that the strucure of the scenarios has a deep impact on the test results. Our scenarios may be not suited to represent the call structure of other applications.

We used the JUnit framework to implement our scenarios as TestCases. We used the JUnitEE Servlet based html Testclient. We chose the servlet based client to eliminate network overhead between client and EJB server.

Observed persistence operations

We identified the following operations as the “atoms” of our scenarios:

1. Looking up an object from the db. I.e. perform a “SELECT FROM …” and materialize the object from the returned ResultSet.

2. Reading object attributes i.e. calling getters

3. Setting object attributes and Updating the underlying database row.

We used a simple “Profiler” that allows to count method calls and to accumulate the time spend in these method calls.

The tested persistence mechanisms

1. Container managed Entity Beans (CMP) with VAP (IBM VisualAge Persistence) generated JDBC code. This is the pure IBM way of implementing CMP with VisualAge for Java and WebSphere.

2. BeanManaged Entity beans (BMP) that use the ObJectBridge PersistenceBroker for performing DB read/write operations and object caching.

3. A pure JDBC reference implementation that does not run within an EJB container. It represent a “normal” JDBC application using prepared statements.

Our test environment

The Application server:

IBM Netfinity M10

-         4x Pentium XEON 400 MHz

-         2 GB RAM

-         Windows NT 4.0 Server, SP 6

-         WebSphere 3.5 Advanced Edition

-         CMP Code generated by IBM VisualAge 3.5 with IBM VAP

-         BMP Code uses the ObJectBridge 0.1.30 release

-         DB UDB Client 6.1 and respective JDBC App-Driver

Database server

IBM Intellistation Mpro

-         2x Pentium III 500 MHz

-         512 MB Ram

-         NT Workstation 4.0 SP 6

-         IBM DB2 UDB 6.1

Testclient

IBM PC

-         1x Pentium III 500 MHz

-         256 MB RAM

-         NT Workstation 4.0 SP 6

-         MS InternetExplorer 5.0 SP1

The test results

The following figure presents the results of the performance test. For each scenario we tested all tree persistence mechanisms and recorded values for the three observed persistence operations.

The following figure presents results for each persistence operation. It also shows totals and average values.

The following figure is a condensed view that compares performance per scenario and per operation. The times for the JDBC variant are taken as reference and the factors of CMP/VAP resp. BMP/OJB are given behind the totals.

Remarks

General observations

From an OJB point of view the results look quite promising. The overall performance of typical EJB applications can be improved by a factor of 8,41 (i.e. 841%).

The performance for the getters is improved by a factor of 24 and the performance for updates is even improved by a factor of 30. Only lookup of objects could not be improved. In a future release of OJB prepared statements will be used for primary key lookups. I assume this will enhance the lookup performance by a remarkable factor.

ObJectBridge and EJB containers

We anticipated the performance gain for getters, as we observed the behaviour of the WebSphere EJB container. After calling any method on an Entity it is stored and passivated immediately. For any subsequent method-call it has first to be found, loaded and activated (which implies three ejbLoad calls !) . This Activation/Passivation strategy results in an enormous performance degradation if an ejbLoad() call has to access the RDBMS.

If the ejbStore() uses and UPDATE for each call this results in completely unneccesary write operations if the entity has not been modified during the method call.

The CMP/VAP code works in this way,so we were not surprised about the bad results for the getters.

OJB uses an ObjectCache. Thus the ejbLoad() just reads an object once from the rdbms all subsequent calls retrieve the objects from the cache.

In addition, OJB uses smart updates, i.e. if the broker detects that an object has not been modified it will NOT perform an update against the RDBMS. If attributes have been modified it will only update modified columns. Thus the ejbStore() calls are not as expensive as for the CMP/VAP scheme.

Using a caching O/R layer in the context of EJB containers seems to be a good strategy to balance the containers benefits against its performance-costs.

If the Lookup speed can be further improved, OJB will help EJB developers to increase the performance of their applications by more than an order of magnitude.

I think the most important result is: The overhead for using an EJB container instead of writing a pure JDBC application where you have to build all the needed infrastructure and services on your own is only 2,78.

This is really not much compared to the benefits you gain from using an EJB container: O/R mapping, transaction handling, scalability, remote object infrastructure, security, etc.

Open questions

My biggest surprise are the very good results for the Updates. It’s hard to believe that OJB (with its dynamic SQL generation using the slow Java-Reflection) can outperform a native JDBC application using a fixed set of prepared statements (“SELECT * FROM … WHERE ID=?”, “UPDATE … WHERE ID=?”).

I have two possible explanation for our results:

1. There is a bug in the BMP/OJB Code and updates are not executed at all.

2. OJB is really faster because it uses smart updates (only modified columns are updated). The JDBC prepared statement updates all columns.
   To use a fixed set of prepared statements for smart updates is not an option due to a combinatorial explosion: for a table with n columns you would need a set of 2ⁿ-1 statements.

I will soon check wich explanation holds true.

Further evaluations

We plan to port our test-cases to other EJB containers (JBOSS and Bea Weblogic). We want to compare the overall performance of IBM WebSphere with that of different containers. Other containers might implement different activation/passivation startegies that reduce the container overhead. They might also use different persistence layers for CMP that do a better job than VAP.
I’m especially interested how ObJectBridge compares to ‘industrial strength’ persistence layers.

release: 0.7.235, date: 2001-11-24