Just when you think the database will never get smarter, it does. And the database that matters most to Enterprise Developers is Oracle.

What happens to coherency between your distributed cache and the database when another application changes the database? If it is a Java application, you can add Ehcache to it and configure it to connect to the distributed cache. But this might require work you do not want to do right now. Or there might be tens of apps involved. Or there might be scripts which are run by DBAs from time to time. Or it could be another language. We have the Cache Server, where you can expose a distributed cache via REST or SOAP and then invalidation becomes as simple as sending a HTTP Delete to the Element resource. See the Cache Server Documentation for more on this.

For a few years, MySQL users have had access to cache invalidation for Memcached using libmemcached. The database calls back to memcached. Brian Aker mentioned to me he was adding this into MySQL a few years ago. See http://www.mysqlconf.com/mysql2009/public/schedule/detail/6277 for a decent introduction.

For Oracle, back in 2005 I tried to use their message queue integration to achieve the same effect. Back then it didn’t even work. I have heard that it works now, but it is a very messy solution with lots of moving parts.

Fortunately, starting from 11g Release 1 (11.1), the Oracle JDBC driver provides support for the Database Change Notification feature of Oracle Database. Using this functionality of the JDBC drivers, multitier systems can take advantage of the Database Change Notification feature to maintain a data cache as up-to-date as possible, by receiving invalidation events from the JDBC drivers. See  the Database Change Notification chapter in the Oracle docs for details.

This lets you achieve a new and very simple way of doing a cache invalidation protocol to keep your cache and the database in sync.

You create a DatabaseChangeListener and register it with your connection:

// conn is a OracleConnection object.
// prop is a Properties object containing the registration options.
DatabaseChangeRegistration dcr = conn.registerDatabaseChangeNotifictaion(prop);
...
// Attach the listener to the registration.
// Note: DCNListener is a custom listener and not a predefined or standard
// lsiener
DCNListener list = new DCNListener();
dcr.addListener(list);

I like this so much I think we might add a standard DatabaseChangeListener to Ehcache for it.

For those of you looking for the older Hibernate Subversion repository it is here: http://anonsvn.jboss.org/repos/hibernate/core/trunk/

It is now on GitHub, but the older SVN is still useful for legacy.

Is it that time of year again. A time for reflection and of New Year’s resolutions. I therefore thought this was good time to reflect on what has been happening with Ehcache.

Ehcache and Terracotta got together in August 2009. We got our first combined release done with Ehcache backed by Terracotta three months later. That was really only the beginning. We have done 5 major releases to date with of course another one in the cooker right now.

The big news is that Ehcache and Terracotta together have been a great success. Most Terracotta users now use it as a distributed backing store for Ehcache. Most of Terracotta’s revenue comes from that use case. This means that Ehcache’s needs are driving the evolution of Terracotta.

At the same time, Ehcache as an open source project has seen a huge investment. Most of the features added are designed to work in Ehcache open source as well as with Terracotta as the backing distributed store.

So let’s look at what we added in 2010 and what is in the cooker so far for 2011.

What we did in 2010

New Hibernate Provider

When we merged Terracotta had just done a Hibernate provider and Ehcache had one too. Neither supported the new Hibernate 3.4 SPI which uses CacheRegionFactories instead of CachingProviders. So we combined the two into a new implementation and added support for the new SPI at the same time. This meant that for the first time Ehcache supported all of the cache strategies, including transactional, in Hibernate and importantly supported them across a caching cluster.  (http://ehcache.org/documentation/hibernate.html)

XA Transactions

Right now XA transactions have fallen from favour partly because of flaky support for them in XAResource implementations out there. But what if you could create a canonically correct implementation that could be absolutely relied on in the world’s most demanding transactional applications? We hired one of the Java world’s foremost experts in transactions (Ludovic Orban, the author of the Bitronix transaction manager) and came out with just that. We challenge anyone to prove that our implementation is not correct and does not fully deal with all failure scenarios. If you need to be absolutely sure that the cache is in sync with your other XAResources with a READ_COMMITTED isolation level you have come to the right place. (http://ehcache.org/documentation/jta.html)

Terabyte Scale Out

Ehcache backed by Terracotta initially held keys in memory in each application that had Ehcache in it. This effectively limited the size of the caching clusters that could be created. With a new storage strategy, we blew the lid of that and stopped storing the keys in memory. The result – horizontal scaling to the terabyte level.

Write-through, behind and every which way

What happens when you have off-loaded reads from your database but now your writes are killing you? The answer is write-behind. You write to the cache. It calls a CacheWriter which you implement and connect to the cache which is called periodically with batches of transactions. In your CacheWriter you open a transaction write the batch and then close the transaction. Much easier for the database. And all done in HA with the write-behind queue is safe because it is stored on the Terracotta cluster.

More caching in more places

We were really happy to extend our support for popular frameworks. During the year:

• Ehcache became the default caching provider for Grails
• We created an OpenJPA provider
• We created Ruby Gems for JRuby and Rails 2 and 3 caching providers
• We created a Google App Engine module

Acknowledgement of the CAP Theorem

Originally Ehcache with Terracotta was consistent above all else. During the year we flexed both ways to allow CAP tradeoffs to be made by our users. We added XA and manual locking modes on the even stricter side and we added an unlocked reads view of a cache and even coherent=false for scale out without coherence on the looser side. And you can choose this cache by cache. There is a tradeoff between latency and consistency, so you choose the highest speed you can afford for a particular cache.

And rather than just blocking on a network partition, we added NonStopCache, a cache decorator which allows you to choose whether to favour availability or consistency.

BigMemory

BigMemory was a big hit. It surprised a lot of people and frankly it surprised us. We were looking to solve our own GC issues in the Terracotta server and found something that was more generally useful than that one use case. So we added BigMemory to Ehcache standalone as well as the server. In the server we have lifted our field engineering recommendation from 20GB of storage per server partition to 100Gb. And we have tested BigMemory itself out to 350GB and it works great!

A new Disk Store

Let’s say you are using a 100GB in Ehcache standalone. When you restart your JVM you want the cache to be there otherwise it might take hours or days to repopulate such a large cache. So we created a new DiskStore that keeps up with BigMemory. It writes at the rate of 10MB/s. So when it is time to shutdown your JVM it just needs to do a final sync and then your are done. And it starts up straight away and gradually loads data into memory. A nice complement to BigMemory and very important.

Ehcache Monitor/Terracotta Dev Console Improvements

For those using Ehcache standalone we have only ever had a JMX API. That is fine but we found many people built their own web app to gather stats. So we did the same and the result was Ehcache Monitor. One of the highlights is the charts including a chart of estimated memory use per cache.

The Terracotta Developer Console got an Ehcache panel, and as we added features to Ehcache we added more to the panel. If you are using Ehcache with a backing Terracotta store then it is a full featured tool which gives you deep introspection.

What is Coming in 2011

Speed, speed and more speed

What does everybody want? More speed. We are splitting hairs in our concurrency model to enable as much speed as possible for each use case. We now have two and will be adding more modes to allow the best tuning for each use case.

Search

Ehcache is based on a Map API. Maps have keys and values. They have a peculiar property – you need to know the key to access the value. What if you want to search for a key, or you want to index values in numerous ways and search those. All of this is coming to Ehcache in February 2011 and is available right now in beta. Oh and one cool thing: search performance is O(log n/partitions). So as your data grows and spreads out onto more Terracotta server partitions, your search performance stays constant! (http://ehcache.org/documentation/search.html)

New Transaction Modes

We already did the hard one: XA. Now we are adding Local Transactions. If you just want transactionality within the caches in your CacheManager and there are no other XAResources, you can use a Local Transaction. It will be three times faster than an XA cache. (http://ehcache.org/documentation/jta.html)

.NET Client

Quite a few customers use Java but also some .NET. And they want to be able to share caches. We have lots of users happily using ehcache for cross-platform use cases, but are planning on extending our cross-platform support still further – for example with a native .NET client

Bigger BigMemory

We are looking at ongoing speedups and testing against larger and larger memory sizes for BigMemory. We are also looking to provide further speed in BigMemory by allowing pluggable Serialization strategies. This will allow our users to use their Serialization framework of choice – and there are now quite a few.

Online Petition

This year’s JavaOne was a dismal affair. Crammed into the Hilton hotel and Parc 55, the feeling was that Oracle had ruined the conference. And the dual conference idea also caused Java people problems: those that tried to attend the key note at Moscone with JavaOne passes were turned away – instead needing to go to the Hilton ballroom to see it televised.

Last year, Larry shocked many with a goof ball suggestion to port OpenOffice to JavaFX. This year attendees were shocked to hear of the cancellation of JavaFX. Those giving and attending sessions on the topic felt it was futile. I was never convinced that JavaFX would be able to get the sort of dominance required to make it work as a browser platform. But what of Java 7, something I am interested in? The beer talk I heard at the Thirsty Bear, was that the JCP has stalemated for the past year on Java 7 over Oracle wanting to add a “restricted field of use condition” to it restricting the OpenJDK to desktop and server, not mobile. One possibility is for Oracle to abandon the JCP and just release it. The other rumour floating around is that future free versions of the JDK will be reference implementations, with higher quality or more fully featured versions only available under commercial license.

All of this suggests to me that Java as we have known it is over. Should we wait for Java to lose momentum and popularity to other languages? Or should we as a community step up and go in a new direction. I prefer the latter. Following is a sketch of how this could be done.

What to call the fork

Java is famous for coffee but also for volcanoes. So let’s call the new fork Lava.

Lava Foundation

We don’t want one company to take over the fork. What would be best is if  a foundation, like the Linux Foundation, Mozilla Foundation, or Eclipse Foundation be formed. This group would be funded by corporations with deep enough pockets to make it work such as Google, IBM, HP and RedHat.

It would be a non-profit foundation.

It would  perform the following duties:

Write Once, Run Anywhere

So how would we maintain this guarantee? The Lava compiler and Virtual Machine would remain backwardly compatible with Java 6. That way the vast array of existing code would work. Then depending on IP restrictions, Lava versions could add support for new language features in later versions of Java. If IP restrictions would preclude that it stays with Java 6.

The question then becomes whether developers would write to the new Java versions or to the evolving Lava. The answer likely would depend on market traction. In favour of Oracle is that it is the real Java. However  if you need to pay license fees to Oracle for the higher quality implementations you would likely want to run in production, then that would come at a cost. If enough commercial companies supported Lava so that it was very high quality, then developers would follow. And developers would want the open source version to win.

I am interested in what the community thinks of this idea. Ping me at gluck AT gregluck.com.

Updates

October 7:

Stefan Asemota created a Lava Foundation facebook page here.

October 14:

Well, some interesting developments have occurred in the last week. IBM and Oracle jointly announced that IBM was switching from project Harmony to OpenJDK and would:

work with IBM and others to enhance and improve the JCP.

What does this last one mean? Trink Guarino clarified this for me:

This includes improving the collaboration with other standards bodies, increasing the participation in the JCP processes and expert groups as well as improving the efficiency of the specification process.

A close reading of the various corporate blogs and press releases shows that the approach between the two companies was made after the reaction to this blog. So, here is hoping that IBM, by negotiating in it’s own interest, will also open things up for the community.

Finally, what about the conference? Happily I was asked to give some feedback on JavaOne, which I did in great detail. Beyond that I am going to Devoxx 2010 and will be speaking there on “The essence of Caching”. With Google there, and hopefully the European non-attenders of JavaOne this year, it should be a bumper conference and may well be the largest Java conference this year.

Today we released the press release for Ehcache’s new Off-Heap Memory Store. Later this week there will be a public beta.

Tonight we released the documentation for it which you can get here.

<repository>
    <id>jboss</id>
    <url>http://repository.jboss.com/maven2</url>
</repository>

After hunting around I had to change it to:

<repository>
    <id>jboss.releases</id>
    <name>JBoss Releases</name>
    <url>https://repository.jboss.org/nexus/content/repositories/releases/</url>
</repository>

It would have been nice if the old name could have been mapped to the new location.

I thought I was done. But no, in the meantime JGroups decided to become more correct and go to a package name. Now, Ehcache did this about three years ago. So the old and new are as follows:

        <dependency>
            <groupId>jgroups</groupId>
            <artifactId>jgroups</artifactId>
            <version>2.6.8.GA</version>
        </dependency>

        <dependency>
            <groupId>org.jgroups</groupId>
            <artifactId>jgroups</artifactId>
            <version>2.10.0.GA</version>
        </dependency>

This whole thing was a hassle. Played out over the entire development community it turns into a lot of wasted productivity. To me each of these changes is no different to people changing interfaces in code. You have to change to rebuild. As it happens, later on I found out JGroups also did that with getLocalAddress, which is no longer castable to IPAddress.

A few weeks ago I blogged about the fantastic tc-maven plugin which works just as well as the Jetty plugin and makes life easy for Maven-based developers. My surveys from talks I do suggest that the mix of Maven and Ant based builds is 40-60% Maven. Three years ago it was about 10% Maven. Interestingly in Philadelphia the maven usage was 40% versus 60% in San Francisco. But many Ant people have tried Maven and had a less than stellar experience. My own experience was that Maven was as painful as EJB ever was. But it has been getting better over time.

The forthcoming release of Ehcache bundles the Terracotta server and I am very interested in making this as easy as possible for developers. The 2.1-beta kit has instructions for using the tc-maven plugin. The upcoming 2.1 final will also support Ant. Fortunately Ant and Maven interoperate and we will support Ant via the Maven Ant Tasks library.

Installation

Install Maven

Download and install Maven which is just expanding the download somewhere on your file system. Version 2.2.1 or higher is required.

Installing Maven Ant Tasks

There are a couple of choices documented at the Maven site. For simplicity, download the Maven Tasks for Ant 2.1.0 and copy the jar into your $ANT_HOME/lib directory.

Create a pom.xml

Maven requires a pom.xml which is placed in the same directory as build.xml. Use this sample which has all you need:

<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>test</groupId>
<artifactId>test</artifactId>
<version>1.0</version>
<dependencies>
<dependency>
<groupId>net.sf.ehcache</groupId>
<artifactId>ehcache-terracotta</artifactId>
<version>2.1.0-SNAPSHOT</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.terracotta.maven.plugins</groupId>
<artifactId>tc-maven-plugin</artifactId>
<version>1.5.1</version>
<executions>
<execution>
<id>run-integration</id>
<phase>pre-integration-test</phase>
<goals>
<goal>run-integration</goal>
</goals>
</execution>
<execution>
<id>terminate-integration</id>
<phase>post-integration-test</phase>
<goals>
<goal>terminate-integration</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<pluginRepositories>
<pluginRepository>
<id>terracotta-snapshots</id>
<url>http://www.terracotta.org/download/reflector/maven2</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
</project>

Additions to build.xml

Add these to build.xml:

    <macrodef name="maven">
        <attribute name="options" default=""/>
        <attribute name="goal"/>
        <attribute name="basedir"/>
        <attribute name="resultproperty" default="maven.result"/>
        <element name="args" implicit="true" optional="true"/>
        <sequential>
            <java classname="org.codehaus.classworlds.Launcher" fork="true"
                  dir="@{basedir}" resultproperty="@{resultproperty}">
                <jvmarg value="-Xmx512m"/>
                <classpath>
                    <fileset dir="${maven.home}/boot">
                        <include name="*.jar"/>
                    </fileset>
                    <fileset dir="${maven.home}/lib">
                        <include name="*.jar"/>
                    </fileset>
                </classpath>
                <sysproperty key="classworlds.conf" value="${maven.home}/bin/m2.conf"/>
                <sysproperty key="maven.home" value="${maven.home}"/>
                <arg line="--batch-mode @{options} @{goal}"/>
            </java>
        </sequential>
    </macrodef>

    <property name="maven.home" value="/Users/gluck/work/apache-maven-2.2.1"/>

    <target name="start_terracotta" description="Starts the Terracotta Server">
        <maven basedir="${basedir}"
               goal="tc:start"
               resultproperty="maven.build.result"
                />
    </target>

    <target name="stop_terracotta" description="Stops the Terracotta Server">
        <property name="maven.home" value="/Users/gluck/work/apache-maven-2.2.1"/>
        <maven basedir="${basedir}"
               goal="tc:stop"
               resultproperty="maven.build.result"
                />
    </target>

Ensure you change your maven.home property value to where you installed Maven.

Usage

Starting Terracotta Server

ant start_terracotta
Buildfile: /Users/gluck/work/ehcache/core/build.xml

start_terracotta:
     [java] [INFO] Scanning for projects...
     [java] [INFO] ------------------------------------------------------------------------
     [java] [INFO] Building Unnamed - test:test:jar:1.0
     [java] [INFO]    task-segment: [tc:start]
     [java] [INFO] ------------------------------------------------------------------------
     [java] [INFO] [tc:start {execution: default-cli}]
     [java] 2010-05-01 12:34:26,737 INFO - Terracotta 3.2.1, as of 20100304-100300 (Revision 14673 by cruise@su10mo5 from 3.2)
     [java] 2010-05-01 12:34:27,018 INFO - Configuration loaded from the Java resource at '/com/tc/config/schema/setup/default-config.xml', relative to class com.tc.config.schema.setup.StandardXMLFileConfigurationCreator.
     [java] 2010-05-01 12:34:27,152 INFO - Log file: '/Users/gluck/terracotta/server-logs/terracotta-server.log'.
     [java] [INFO] ------------------------------------------------------------------------
     [java] [INFO] Starting DSO Server
     [java] [INFO] [dso start] 2010-05-01 12:34:27,576 INFO - Terracotta 3.2.1, as of 20100304-100300 (Revision 14673 by cruise@su10mo5 from 3.2)
     [java] [INFO] OK
     [java] [INFO] [dso start] 2010-05-01 12:34:28,065 INFO - Configuration loaded from the Java resource at '/com/tc/config/schema/setup/default-config.xml', relative to class com.tc.config.schema.setup.StandardXMLFileConfigurationCreator.
     [java] [INFO] [dso start] 2010-05-01 12:34:28,255 INFO - Log file: '/Users/gluck/terracotta/server-logs/terracotta-server.log'.
     [java] [INFO] [dso start] 2010-05-01 12:34:30,759 INFO - Available Max Runtime Memory: 506MB
     [java] [INFO] [dso start] 2010-05-01 12:34:30.575::INFO:  Logging to STDERR via org.mortbay.log.StdErrLog
     [java] [INFO] [dso start] 2010-05-01 12:34:30.620::INFO:  jetty-6.1.8
     [java] [INFO] [dso start] 2010-05-01 12:34:30.663::INFO:  Started TerracottaConnector@0.0.0.0:0
     [java] [INFO] [dso start] 2010-05-01 12:34:33,097 INFO - JMX Server started. Available at URL[service:jmx:jmxmp://0.0.0.0:9520]
     [java] [INFO] DSO Server status: OK
     [java] [INFO] ------------------------------------------------------------------------
     [java] [INFO] BUILD SUCCESSFUL
     [java] [INFO] ------------------------------------------------------------------------
     [java] [INFO] Total time: 14 seconds
     [java] [INFO] Finished at: Sat May 01 12:34:33 EST 2010
     [java] [INFO] Final Memory: 22M/133M
     [java] [INFO] ------------------------------------------------------------------------
     [java] [INFO] [dso start] 2010-05-01 12:34:33,840 INFO - Terracotta Server instance has started up as ACTIVE node on 0.0.0.0:9510 successfully, and is now ready for work.

BUILD SUCCESSFUL

Total time: 17 seconds

The Terracotta server will be running on its default port of 9510.

Stopping Terracotta Server

ant stop_terracotta

Buildfile: /Users/gluck/work/ehcache/core/build.xml

stop_terracotta:
     [java] [INFO] Scanning for projects...
     [java] [INFO] ------------------------------------------------------------------------
     [java] [INFO] Building Unnamed - test:test:jar:1.0
     [java] [INFO]    task-segment: [tc:stop]
     [java] [INFO] ------------------------------------------------------------------------
     [java] [INFO] [tc:stop {execution: default-cli}]
     [java] [INFO] ------------------------------------------------------------------------
     [java] [INFO] Stopping DSO Server
     [java] [INFO] [dso stop] No host or port provided. Stopping the Terracotta server instance at 'localhost', port 9520 by default.
     [java] [INFO] OK
     [java] [INFO] ------------------------------------------------------------------------
     [java] [INFO] BUILD SUCCESSFUL
     [java] [INFO] ------------------------------------------------------------------------
     [java] [INFO] Total time: 7 seconds
     [java] [INFO] Finished at: Sat May 01 12:48:37 EST 2010
     [java] [INFO] Final Memory: 21M/123M
     [java] [INFO] ------------------------------------------------------------------------

BUILD SUCCESSFUL
Total time: 10 seconds

More Information

The Terracotta plugin is documented on the Forge.

I am interested in people’s experiences using this. Ping me at gluck AT gregluck.com or post questions to the Ehcache Forum.

Having servers at development time is pain. You need tooling to make it smooth. Fortunately, Terracotta has the tc-maven plugin for this purpose.

Integration Testing with Maven

To start and stop the server pre and post integration tests, add the following to your pom.xml:

<pluginRepositories>
        <pluginRepository>
            <id>terracotta-snapshots</id>
            <url>http://www.terracotta.org/download/reflector/maven2</url>
            <releases>
                <enabled>true</enabled>
            </releases>
            <snapshots>
                <enabled>true</enabled>
            </snapshots>
        </pluginRepository>
    </pluginRepositories>
 

    <plugin>
        <groupId>org.terracotta.maven.plugins</groupId>
        <artifactId>tc-maven-plugin</artifactId>
        <version>1.5.1</version>
        <executions>
            <execution>
                <id>run-integration</id>
                <phase>pre-integration-test</phase>
                <goals>
                    <goal>run-integration</goal>
                </goals>
            </execution>
            <execution>
                <id>terminate-integration</id>
                <phase>post-integration-test</phase>
                <goals>
                    <goal>terminate-integration</goal>
                </goals>
            </execution>
        </executions>
    </plugin>


 
Interactive Use

Interactively, with the pom configured, it is just

mvn tc:start to start

and mvn tc:stop to stop