April 18, 2008

Open Message Queue Session at Community One

Hi

I am presenting a session on JMS and OpenMQ at Community One 2008 as follows:

Session ID: S297892
Session Title: OpenMQ
Session Abstract: This session discusses why you should consider Java™ Message Service (JMS) and OpenMQ for scalable and reliable architectures. The presentation includes a use case from the No. 2 Australian e-business site.
Track: GlassFish™
Duration: 55
Speaker(s): Greg Luck
Where: Moscone South - Esplanade Room 305

(This talk was initially presented at Sun Technology Days in Sydney last month by Dave Whitla where it generated a great deal of interest.)

Posted by gluck at 02:33 PM | Comments (0)

Glassfish Unconference - Moscone Center San Francisco 4 May 2008

I am going to the Glassfish Unconference which is being held at the Moscone Center San Francisco 4 May 2008.

I am interested in or might lead discussions on:

SPNEGO - Kerberos and Glassfish Open Message Queue Maven Glassfish Plugin Ehcache and Glassfish

If you are interested in these please come along and have a chat. See http://wikis.sun.com/display/GFunconfSF08/GlassFish+unconference+planning.

Posted by gluck at 11:35 AM | Comments (0)

March 12, 2008

Glassfish Case Study from Sun Tech Days' Glassfish Day in Sydney March 2008

I gave a case study on our use of Glassfish V1UR1 at the Sun Tech Days' Glassfish Day in Sydney March 2008 for which the slides and podcast are available here.

With Glassfish V2UR1 now out and a great roadmap to V3, Glassfish is looking great.

Posted by gluck at 12:37 PM | Comments (0)

March 09, 2008

Ehcache is the SourceForge.net February project of the month

Ehcache has been honoured as the February 2008 SourceForge project of the month. See http://sourceforge.net/community/index.php/potm-200802/.

Posted by gluck at 01:44 PM | Comments (0)

November 29, 2007

Ehcache Guide and Reference now available as a book

Ehcache Guide and Reference for the forthcoming 1.4.0 release is available as a book. The price for the high quality bound edition is USD49.95 plus shipping. The price for the downloadable print-ready version is USD37.50. The book has 154 pages and covers the following:

- Preface
- Introduction
- Getting Started
- Dependencies
- Cache Concepts
- Cache Configuration
- Storage Options
- Cache Eviction Algorithms
- Code Samples
- Class loading and Class Loaders
- Performance Considerations
- Garbage Collection
- Cache Decorators
- Java EE Servlet Caching
- Distributed Caching
- Distributed Design
- Shutting Down Ehcache
- Logging & Debugging
- JMX Management & Monitoring
- CacheManager Event Listeners
- Cache Event Listeners
- Cache Exception Handlers
- Cache Extensions
- Cache Loaders
- Hibernate Caching
- Constructs Design
- Using ehcache with Hibernate
- Tomcat Issues and Best Practices
- JSR107 (JCACHE) Support
- Building From Source
- FAQ

Many people have asked for a format for the documentation where it can be printed out as a book and even where they can get it.

Of course, full documentation in html will always remain available online at http://ehcache.sf.net.

Buy it from the Ehcache Storefront at Lulu Books.

Posted by gluck at 02:07 PM | Comments (0)

August 27, 2007

Ehcache-1.4 beta Released

The beta version of ehcache-1.4 is available on sourceforge and the Maven central repository.

This version significantly enrichens the ehcache API and enhances extension of ehcache. The new features are:

You can now define CacheLoaders in the ehcache core API. They supplement the existing SelfPopulatingCache. They can be configured in ehcache.xml or programmatically. They can be used for cache warming, and simple pull-through caching. See the Cache Loaders chapter in the documentation for more details.

You can now define a CacheExceptionHandler and register it progammatically or in ehcache.xml. The enable an exception handler to be registered with a cache so that it is called on exception rather than the exception being thrown. See the Cache Exception Handler chapter in the documentation for more details.

You can now define CacheExtensions and register them progammatically or in ehcache.xml. A CacheExtension is a generic facility which holds a reference to a cache and is bound to the cache lifecycle, making possible a wide variety of extensions to behaviour. Usage examples are a cache refresh timer or a file monitoring mechanism. See the Cache Extensions chapter in the documentation for more details.

There are 5 minor bug fixes in the release. At the time of release there are no open bugs in ehcache.

See http://ehcache.sourceforge.net/ for more information.

Posted by gluck at 08:03 PM | Comments (0)

August 19, 2007

Dynamic Proxy Performance

Ehcache 1.4, which is nearing completion, adds some user requested features such as ExceptionHandler(s). These are implemented in ehcache as dynamic proxies. One todo I have had is to investigate the performance degradation from use of same. An article by Brian Goetz (fellow JSR107 member) on IBM developerworks a few years ago predicted a 2x cost.

I have done some testing using a memory only cache in ehcache and come with a line ball. For ehcache afficianados, the test is in CacheTest.testProportionMemoryAndDiskPerformance. This is a test that protects against regression in ehcache by asserting outside times for variations in memory and disk store sizes. I added a memory only test using the new CacheExceptionHandler dynamic proxy.

It does 5000 puts and gets, sleeps for 500ms to give the hotspot compiler a chance to optimise, and repeats 10 times. Then it does the same for a dynamic proxy.

INFO: Time for MemoryStore: 120
Aug 19, 2007 5:10:52 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for MemoryStore: 123
Aug 19, 2007 5:10:53 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for MemoryStore: 63
Aug 19, 2007 5:10:53 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for MemoryStore: 40
Aug 19, 2007 5:10:54 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for MemoryStore: 128
Aug 19, 2007 5:10:55 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for MemoryStore: 30
Aug 19, 2007 5:10:55 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for MemoryStore: 131
Aug 19, 2007 5:10:56 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for MemoryStore: 29
Aug 19, 2007 5:10:56 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for MemoryStore: 35
Aug 19, 2007 5:10:57 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for MemoryStore: 35
Aug 19, 2007 5:10:57 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for MemoryStore: 130

Mean: 86ms

Aug 19, 2007 5:10:58 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for exception handling MemoryStore: 69
Aug 19, 2007 5:10:59 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for exception handling MemoryStore: 46
Aug 19, 2007 5:10:59 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for exception handling MemoryStore: 91
Aug 19, 2007 5:11:00 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for exception handling MemoryStore: 158
Aug 19, 2007 5:11:00 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for exception handling MemoryStore: 69
Aug 19, 2007 5:11:01 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for exception handling MemoryStore: 51
Aug 19, 2007 5:11:02 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for exception handling MemoryStore: 137
Aug 19, 2007 5:11:02 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for exception handling MemoryStore: 39
Aug 19, 2007 5:11:03 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for exception handling MemoryStore: 45
Aug 19, 2007 5:11:03 PM net.sf.ehcache.CacheTest testProportionMemoryAndDiskPerformance
INFO: Time for exception handling MemoryStore: 38

Mean: 74ms

The result is that I cannot detect a performance differnece.

Maybe the reason is that ehcache just passes the requests on unless they throw an exception. We have

try {
invocationResult = method.invoke(ehcache, args);
} catch (Exception e) {

In Java 5 at least the cost seems to be negligle, even with an in-memory cache.

Posted by gluck at 06:22 PM | Comments (0)

July 27, 2007

Interim Governance Board for the GlassFish community.

I heard today that I am going to be on the Interim Governance Board for the GlassFish community. It might be worth mentioning some of the stuff I have been doing with Glassfish so that everyone can see the perspective I will be bringing to the board.

At work we went live with Glassfish V1 about 6 months ago. It has been a very smooth ride since then.

Since then I have worked on the SPNEGO extension for Glassfish V2. I am also working with some Sun guys on testing Hardware Security Module support in Glassfish V2. Because of these security iniatives we have reason to promptly upgrade to GFV2.

We are currently using the Catalina web container, which can be switched for Grizzly, due to a show stopper bug we found in Grizzly, which is fixed and available to those with a support contract from Sun for Glassfish. For us it is easier to change over when we upgrade to GFV2.

We have a couple of C Ruby applications deployed to Apache. We will be looking at moving those to Glassfish for easier development and deployment in our environment.

Our use of Glassfish uses most of JEE5 including lots of EJB3. We also use a ton of open source, notably Spring and webwork.

Finally, we are shortly going live on Open Message Queue, as our enterprise message queue. I have been working on porting from Active MQ to Open Message Queue. As with Glassfish, Open Message Queue is much stricter. It also rocks. We got higher performance on Open Message Queue over Active MQ, and very simple integration with our JEE apps in Glassfish.

For my own part, I tend to do open source projects that fill an infrastructure gap. The gaps I have hit and needed to write something for, have in chronological order been, ehcache.sf.net, jpam.sf.net and spnego.dev.java.net.

Other than the Glassfish Board, I am also active a co spec lead of JSR107 JCACHE, which has reawakened and is getting a lot of work done on it by Brian Goetz and Manish Surtani.

So, I think I bring three perspectives to the board: comprehensive enterprise user, open source maintainer and standards community contributor.

I encourage anyone to raise suggestions, issues or concerns with me concerning the Interim Governance Board at gluck A T gregluck.com.

Posted by gluck at 04:13 PM | Comments (0)

June 28, 2007

JMX Demo for ehcache-1.3.0

Posted by gluck at 07:02 PM | Comments (0)

June 14, 2007

JSR107 (JCache) Update

Mailing List Set up

At JavaOne the members of JSR107 present agreed to conduct technical deliberations in public view. Anyone can go to https://jsr107.dev.java.net and see what we are up to. In particular discussions are taking place on the jsr107 AT jsr107.dev.java.net mailing list. Go to https://jsr107.dev.java.net/servlets/ProjectMailingListList
to search the archive or set up an RSS feed.

JSR107 Development Philosophy

I believe it is a widely held community view that there has been a problem in the JCP that some JSRs have failed because they created technology and APIs with insufficient community involvement. Based on things I have seen and read I do not think the original JSR107 draft API is well considered in the community.

Given the length of time that has elapsed since the start of JSR107, we are now in a situation where the open source and commercial caches have evolved a feature set and APIs based on organic growth, driven by demand. There are many instances where ehcache, which started out simply, got requests for feature xxx like in OSCache, or "other caches do this so why don't you". These caches are now around 4 years old so this process has had that long to run its course.

The second point is that caching is now very widely used. There are therefore a large group of people with expectations about how caching works in Java.

So, I think the community will be expecting JSR107 to deliver something like they already get from the various open source caches and to a lesser extent the commercial caches. These caches provide a lot of utility. Simply standardising the API and the SPI so that the community can use an implementation with a low cost of change is, I think, the most valuable part of JSR107.

I see the involvement of representatives from Ehcache, JBossCache and Tangosol as being key to delivering on this expectation.

I also think it vital to allow simple, no-overhead, read-only access to the technical debates on features so that possibility of feedback from the community to EG members is possible.



So this is the background I have to the current JSR107 deliberations. It might be useful to get an exchange of these philosophical views now rather than playing them out feature by feature.

Posted by gluck at 10:54 AM | Comments (0)

June 04, 2007

Spnego 1.0 Released - Now everyone can do Kerberos

The spnego project has released version 1. The Spnego project is a Kerberos over SPNEGO plugin for JSR196 compliant application servers. Right now the Glassfish betas are compliant, with other appservers like JBoss expected to be compliant soon.

The plugin lets your Java web applications participate in a Kerberise Single Sign On environment. Log in to Kerberos in an enterprise, and you will not even need a login page in your web apps. IE, Firefox and Safari all work. And Windows, Mac OS X and Linux all work.

This project is something of a missing link for Java application servers. Apache has had mod_auth_kerb which does the same thing for Apache apps for some time. And Websphere and Weblogic have had proprietary implementations. But this release lets you use any application server.

See http://spnego.dev.java.net for full documentation and downloads.

Posted by gluck at 04:57 PM | Comments (1)

May 23, 2007

What's happening with JSR107?

At JavaOne 2007 the JSR107 Expert Group reconstituted and conducted three sessions.

We admitted some new members. Manik Surtani from JBoss is joining. Crazy Bob Lee too.
I was already a member but bot made co-spec lead to help make sure the JSR kept moving.

We decided to conduct technical discussions in the open. For those of you who are interested
you can track these at jsr107@jsr107.dev.java.net. You can go to the project at https://jsr107.dev.java.net/ and subscribe.

It was also decided to use DocBook format for creating the specification, so that multiple EG members could work concurrently. Manik Surtani has committed the current draft spec done by Cameron in DocBook format to the jar107.dev.java.net CVS repository. You can also check out the repository and see what is going on.

We hope to get moving with this and get a spec completed in the next few months. Stay tuned.

Posted by gluck at 07:42 PM | Comments (0)

May 21, 2007

Ruby in decline...

At JavaOne 2007 I spent some of my time trying to figure out where things are going. It has always seemed to me that you either knock yourself out keeping up with every last thing (shiny new toy syndrome) or you try to figure out what is going to be important and focus on that (pragmatist). I am in the latter camp, but fortunately I know many people in the former camp. For programming languages I read the TIOBE Programming Index each month.

The May 2007 edition confirms a trend that I mentioned to a few people at JavaOne. Ruby had leveled of at the end of 2006. It is now in decline. From Tiobe's commentary:

Although, Ruby is the rising star if you look back 1 year, it is not growing more the last couple of months. Even worse, since April it is slightly declining. If this appears to be the new trend, then also Ruby does not become the "next big programming language".

Java, C and C++ have been dominating the TIOBE index from the start and it seems as if they will keep this status for a long while. Possible new candidates are Lua (from 55 to 23 in 1 years time) and Groovy (from 103 to 52). These are, just like Ruby, lightweight scripting languages. I have the impression there is a maximum to such dynamically interpreted languages. Although very popular for web-based programming, statically compiled languages remain the core of all enterprise software systems.

The significance of the drop in May and the leveling of is that "Ruby does not become the 'next big programming language'". I learned both Ruby (and Python) in 2005 in preparation/mitigation of that happening.

Does this mean that Ruby is not important? No.

Firstly, Ruby (and Rails) are having enormous influence on the development and direction of the more entrenched languages and frameworks. In evidence of this I cite what is happening to Java. By Java I specifically include the JVM runtime. Within this broad church we see JRuby, JSR241 aka Groovy, JSR292 invokeDynamic (http://blogs.sun.com/jrose/entry/jsr_292_deep_dive_notes), and numerous influences to web frameworks such as SEAM. I have suggested to Paul Hammant that he create a graphical representation of web framework influences, which would make this last point clearer.

Secondly, if you look at the chart above, Python, JS, PHP and Ruby make up a pretty big chunk. They are not going away. Taken as a group they are slowly growing. So let the appservers and runtimes (see Glassfish V3 and JVM7) run them. If developers do not use Java they will be using Java infrastructure. And just as the dynamic languages run everywhere, so does Java Infrastructure.

So we see that the Sun, and the broader Java industry, is taking an each way bet. Suck up the good ideas from the Ruby community into your own stuff so that you keep your own adherents. Failing that be a great place to deploy apps written in dynamic languages. Either way you stay relevant.

Posted by gluck at 11:23 AM | Comments (4)

May 19, 2007

Comparing Memcached and Ehcache Performance

Comparing Memcached and Ehcache Performance

Performance Comparison

I did some research recently on memcached and how it compares to ehcache. The following graph shows the time taken for 10,000 puts, gets and removes, for 10,000 cache items. It uses the latest released versions of memcached and ehcache. In memcached's case libevent is installed. The computer was a Mac Book Core Duo with 1.25GB of memory. The test used was the standard benchmark used in memcached's Java client. The client used was the Java client.

The results are that put and gets are 500 to 1000 times faster in ehcache compared with memcached. The grey bar, barely visible, is ehcache with the cache all in memory. The blue bar is with 9,900 of the cache items in the disk store. Even the disk performance of ehcache is way faster than memcached. Of course memcached is entirely in memory.

Why?

The AMP people always make the claim that memcached is really fast. The memcached mailing list has a long thread arguing back and forth as to whether memcached is actually any faster than using the MySQL in memory cache tables. It seems in some circumstances they are pretty close. I think of memcached as a client-server cache.

In the diagram above you can see that each put, get or remove happens over the network between a memcached client running on the web server and a memcached server using the memcache wire protocol. There is network and marshalling overhead for each request.

By comparison, the ehcache architecture is shown below. The cache is local and in-process. The cost of gets is that of referencing memory or loading from disk, with appropriate thread safety.

Puts are really fast because they are always synchronous to the memory store, with transfer to disk or to other nodes in the cluster happening asynchronously by batch.

The performance differences are due to the different architectures.

Implications

In-process caching and asynchronous replication are a clear performance winner.

Ehcache and other in-process caches are very widely used in the Java world. One thing I see happening is new languages reusing Java infrastructure. An example is Grails (Groovy on Rails) which can re-use the Java stack. Grails is using JPA, which lets it use Hibernate and in turn second level caching. Now memcached is typically used with Rails apps running on Apache with the share nothing approach. But now JRuby on Rails is able to be deployed in Java web containers (See ThoughtWorks' Mingle product which is done this way for an example). There are notes on how to deploy JRuby on Rails apps to Glassfish in a WAR, and Glassfish V3 takes deployment simplicity it even further. The JRuby on Rails (JRails anyone?) folks are planning to move to JPA (from what I heard at JavaOne).

So it seems that regardless of coding in Java, GRails or JRails, you will be able to reuse the Java infrastructure and in particular the in-process Java caches that come with it. And that is a very good thing.

Posted by gluck at 09:43 AM | Comments (0)

April 17, 2007

JPam - a bridge between JAAS and Unix PAM security

I have just released JPam 1.0.

JPam is a Java-PAM bridge. PAM, or Pluggable Authentication Modules, is a standard security architecture used on Linux, Mac OS X, Solaris, HP-UX and other Unix systems. JPam is the missing link between the two.

JPAM permits the use of PAM authentication facilities by Java applications running on those platforms.

These facilities include:

1. account
   
2. auth
   
3. password
   
4. session
   

It features JAAS and direct APIs, support for most Unix OSs and architectures and much more...

JPam is available under an Apache open source license and is actively developed, maintained and supported.

You can use it in both JSDK and JEE environments. I use it in the Glassfish application server with a custom AuthModule to utilise Vasco hardware tokens for login.

See http://jpam.sf.net for more details.

Posted by gluck at 06:11 PM | Comments (0)

April 11, 2007

Spnego for Glassfish - First working implementation achieved today

I am feeling a little excited right now.

Today we got an implementation of SPNEGO going for Glassfish. It will be refined over coming days, but it works. You can use Glassfish in a Kerberos Single Sign On environment. It works with Firefox, IE and Safari.

Some features/limitations at present:

* We use the new JMAC JSR196: Java Authentication Service Provider Interface for Containers. This means you need to use Glassfish V2 build 41 or later, which have working JMAC implementations.

* We rely on JGSS features found in Java 6. We hope to make some changes and support Java 5 as well.

* The current implementation has only a few dependencies on Glassfish. We hope to remove those to allow the Spnego module to work with any and all JMAC implementations.

* JMAC has limited interaction with JAAS. At the moment it is not possible to fall through to a JAAS module for authentication. Not sure if this is a problem or not in a SSO environment.

Over the next days and weeks I will be refining Spnego. It is likely that it will be included as a standard part of Glassfish V2 when it is released in August 2007.

See http://spnego.dev.java.net for the project.

Now would be a good time for anyone looking to port this code to another JMAC compliant app server to get involved.

Posted by gluck at 06:58 PM | Comments (1)

April 08, 2007

Ehcache Talk at JavaOne 2007

Session ID: TS-6175
Session Title: Distributed Caching, Using the JCACHE API and ehcache, Including a Case Study on Wotif.com
Track: The Next Generation Web
Room: Esplanade 307-310
Date: 08-MAY-07
Start Time: 10:50

Posted by gluck at 10:45 AM | Comments (0)

March 02, 2007

Glassfish does not support SPNEGO and Kerberos - Yet

I had an interesting time this week working out whether Glassfish supports Kerberos and SPNEGO. For those who have not read my previous post, pretty much everything now has baked in support for Kerberos. For web apps, there is a security protocol over HTTP called SPNEGO which lets browsers use single sign on with web servers.

So, does Glassfish support SPNEGO and Kerberos? The short answer is no. The longer answer is that all of the bits that you need to implement it are in place such as:

1. Krb5LoginModule (Added in JDK1.4.2)
   
2. SPNEGO (Added in Java 6)
   
3. A richer security plugin facility in JMAC/JSR196 for which Glassfish is the reference implementation
   

I am thinking about writing, or better still, convincing someone else to write this missing piece.
It would work as follows:

1. JMAC module in servlet container rejects request with HTTP 401 (Unauthorized) "WWW-Authenticate: Negotiate" header
   
2. browser computes SPNEGO token and resends request with Authorization: Negotiate header containing base64 encoded SPNEGO token (as you pointed out, we can find browsers that already know how to do this).
   
3. JMAC module decodes token and uses it to call
   gss_accept_security_context.
   
4. JMAC module returns an appropriate HTTP status code and header based on the outcome of the gss accept.
   

Posted by gluck at 09:09 PM | Comments (0)

February 26, 2007

I will be presenting at JavaOne 2007 on ehcache

For anyone attending JavaOne 2007 at the Moscone Centre in San Francisco I will be presenting a session entitled "Distributed Caching, Using the JCACHE API and ehcache, Including a Case Study".

Abstract follows:

Title: Distributed Caching, Using the JCACHE API and ehcache, Including a Case Study on Wotif.com

Abstract: Java EE web applications are typically used in e-commerce systems. They generally involve multiple application servers communicating with a database. Caching can play an important role in enabling scaling out. Caching can be applied to servlet responses, collection caching, and Java Persistence API caching. JSR 107 (JCACHE) is an effort to standardize the cache API. It benefits users by providing a lowest common denominator for caches and by reducing the cost of change of caching implementations. This presentation looks at the features of JCACHE and the ehcache implementation of JCACHE (ehcache-1.3) in particular. It also includes a case study of wotif.com, the third-largest e-commerce site in Australia by revenue, which does a large amount of caching at each of the levels mentioned above. The case study shows how wotif.com achieved horizontal scaling by using the distributed caching features available in ehcache--all running on the GlassFish project application server. It demonstrates how to replicate servlet responses across a cluster, so that regardless of the view generator--whether it be JavaServer Pages (JSP) software, Velocity, or XML responses to Ajax calls--the servers can share a cache of responses. Next the presentation looks at how to cache computed results represented as collections. It examines pull-through-cache patterns and gives examples. Last, it discusses caching with the Java Persistence API. Wotif.com uses Hibernate, which enables pluggable second-level caching, and the presentation shows how the site uses Hibernate with a distributed cache to minimize database access. The session closes with a discussion of the timeline for release of the JCACHE specification and how to get started.

Posted by gluck at 05:59 PM | Comments (0)

February 13, 2007

Opening Up Glassfish to remote JMX monitoring

We are doing monitoring of our apps using ManageEngine Applications Manager. It supports JMX remoting. The latest patch release supports composite types. Very nice.

Right now Glassfish is the new kid on the block. Support for it is not yet baked in to Applications Manager (but will be, according to their product manager). In the meantime you can use generic JMX monitoring to monitor it.

The glassfish domain defaults do not work for remote monitoring. After a bit of playing around I got it to work.

You need to add the following to your domain.xml configuration in Glassfish:

Add these to the java-config element:
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false

Add this to the admin-service element.

<jmx-connector accept-all="true" address="0.0.0.0" auth-realm-name="admin-realm" enabled="true" name="system" port="8687" protocol="rmi_jrmp" security-enabled="false"/>

Not sure exactly which change fixed it. But this config works.

In Applications Manager you do the following(example only):

Host Name / IP Address: developer249
Port: 8687
Polling Interval: 1
JNDI Name: /jmxrmi
Authentication is enabled: X
User Name: admin
Password: password

The same thing works for the jconsole and should work for NMS's that support JMX. Enjoy.

Posted by gluck at 05:31 PM | Comments (0)

February 12, 2007

Dependency Management Choices: Maven, Ant + Maven Antlib, Ivy

We have a monolithic code base. It is something you end up with if you keep adding classes, without thought to larger modules. So how to solve the problem. Break the code up into modules. Also figure out how to combine these into applications. Then look at whether to run these together in the one JVM or separately.

One problem that comes up is tool support. Ant on its own does provide ready made solutions to this.
We needed a higher level build system on top of Ant that provides a default means of dependency management, multi-project relationships, and generation of build artifacts such as jars, wars, and ears.

A look around shows a number of approaches: Ant + Ivy, Maven, Ant + Antlib for Maven and Maven + antrun plugin.

Maven

Maven is a popular and widely used build system which handles dependency management and multi-project relationships. It operates at a higher level than ant, sort of like what Hibernate is to JDBC. Maven scripts tend to be much shorter than Ant scripts to get an amount of work done.

Maven provides:

• a dependency management system. no more “lib/*.jar” dependencies
  
• a standard for an artifact repository that handles packages and version numbers, javadocs and source code (of releases), either centralised or proxied.
  
• a ‘standard’ project lifecycle where you can ‘plug’ special processing. All projects go through: generate source, compile, package, install etc…
  
• Simple project setup that follows best practices - get a new project or module started in seconds
  
• Consistent usage across all projects means no ramp up time for new developers coming onto a project
  
• Superior dependency management including automatic updating, dependency closures (also known as transitive dependencies)
  
• Able to easily work with multiple projects at the same time
  
• A large and growing repository of libraries and metadata to use out of the box, and arrangements in place with the largest Open Source projects for real-time availability of their latest releases
  
• Extensible, with the ability to easily write plugins in Java or scripting languages
  
• Instant access to new features with little or no extra configuration
  
• Ant tasks for dependency management and deployment outside of Maven
  
• Model based builds: Maven is able to build any number of projects into predefined output types such as a JAR, WAR, or distribution based on metadata about the project, without the need to do any scripting in most cases.
  
• Coherent site of project information: Using the same metadata as for the build process, Maven is able to generate a web site or PDF including any documentation you care to add, and adds to that standard reports about the state of development of the project. Examples of this information can be seen at the bottom of the left-hand navigation of this site under the "Project Information" and "Project Reports" submenus.
  
• Release management and distribution publication: Without much additional configuration, Maven will integrate with your source control system such as CVS and manage the release of a project based on a certain tag. It can also publish this to a distribution location for use by other projects. Maven is able to publish individual outputs such as a JAR, an archive including other dependencies and documentation, or as a source distribution.
  
• Dependency management: Maven encourages the use of a central repository of JARs and other dependencies. Maven comes with a mechanism that your project's clients can use to download any JARs required for building your project from a central JAR repository much like Perl's CPAN. This allows users of Maven to reuse JARs across projects and encourages communication between projects to ensure that backward compatibility issues are dealt with. We are collaborating with the folks at Ibiblio who have graciously allowed the central repository to live on their servers.
  

Potential Issues with Maven

The historic issues with Maven and the current status are:

• Maven was poorly documented. This has been improved somewhat, particularly with a freely downloadable PDF book on Maven2.
• Is their a critical mass out there? There now is. It is becoming very common to see pom.xml's in projects. pom.xml comes up on Google in a 1:4 ratio with Ant. So it has critical mass and is not going away. Ibiblio has a very large number of projects in its repository.
• The changeover from Maven 1 to Maven 2 was untidy and confusing. Now everyone is one maven 2
• Maven has an initial learning curve. You need to dive into black magic to develop a maven plugin when you need to do custom things and this is hard. You can always call into an ant script. Developing a plugin is optional.
• The central repository may not be available. The solution is to use a local repository. See http://cocoon.zones.apache.org/daisy/documentation/g1/1164.html

Ant + Antlib for Maven

Maven 2.0 now comes with a set of Ant tasks that can be used to utilise Maven's artifact handling features from within Ant. Included are:

• Dependency management - including transitive dependencies, scope recognition and SNAPSHOT handling
• Artifact deployment - file and SSH based deployment to a Maven repository
• POM processing - for reading a Maven 2.0 pom.xml file

See http://maven.apache.org/ant-tasks.html

Remaining Problems with Ant

Ant files tend to become huge. Can be solved, as of ant1.6, by the ant import nd macrodef tasks.

Each project ends up with duplicated Ant code. Can be solved, as of ant1.6, by the ant import nd macrodef tasks, although you then need a common artifact

No standardised build structure. This can be enforced manually.

Ant + Ivy

Ivy works very much like Ant + Maven Antlib. It has arguably a better dependency management process. It does not have a central repository of projects. You have to package third party stuff up and put it in your Ivy repository. It does not provide the extra features of Maven beyond dependency management.

Making a Choice

Relative Popularity

In terms of what projects are using, one way to tell is to search google for the config files that are used by each tool.

A google search for build.xml returns 3,240,000 results.
A google search for pom.xml returns 786,000 results.
A google search for ivy.xml returns 17,900 results.

Both Ant and Maven are mainstream. Ivy is a niche player by this measure.

Thought Leaders

Matt Raible was an eager Ivy user, then moved to Ant + Maven Antlib then moved to Maven as of September 06. The Spring project is moving to Maven. Much of CodeHaus and Apache have/are moving to Maven.

I know people who use Maven and those that use Ivy. Both Maven and Ivy are part of Apache.

Bugs

The Maven project is extremely poor at fixing bugs and getting releases out. Ivy has a better reputation.

Summing Up

I have not made the move to pure Maven because I think it would take a fair bit of work. I have started to do so then run into limitations or bugs. The large numbers of unresolved bugs trouble me greatly.

If you are doing Corporate development, like using Ant and have existing projects you want to solve the dependency management problem for, I would say go either with Ant + Maven Antlib or Ant + Ivy.

At work we opted to go pure Maven and convert our existing Ant projects. I am not sure at this point that it has been worth it.

Posted by gluck at 11:28 AM | Comments (0)

December 04, 2006

ehcache-1.2.4 released

Ehcache-1.2.4 has been released.

It contains four feature and fifteen bug fixes received in the three months since ehcache-1.2.3 was released. See the changelog for details.

Eight of the changes in this release are scalability improvements contributed by users running very large ehcache deployments. The distributed caching changes stem from a user who is using ehcache in a multi-subnet cluster of 20 nodes.

The Caching and Gzip filters in the web package have been updated to work on Glassfish and Tomcat.

The distributed caching functionality in ehcache is now 6 months ago and is widely used. Moreover the maintainer has now been running ehcache distributed in production for four months without incident. This functionality can now be considered mature.

Another area that has improved in this release is a large reduction in thread use. Ehcache used to use three threads per DiskStore. Now it uses one.

Ehcache has no open bugs or patches at the time of release.

Work now turns to a meaningful JSR107 (JCACHE) implementation and perhaps pushing to fix up and then releasing JSR107. From what I have seen around there is now pent up demand for JSR107 to be released. I am planning to implement it as it is but keep a list of its shortcomings for use in negotiating some improvements with my expert group colleagues.

Posted by gluck at 05:46 PM | Comments (0)

SEDA, NIO and Grizzly/Glassfish

SEDA, or staged event driven architecture was all the rage about four years ago. (http://www.eecs.harvard.edu/~mdw/proj/seda/).

Java 1.4 added the NIO, or "new IO" package inspired by it.

The logical next step was a Java web server using the approach. I spent some time last week tuning our Glassfish application server for production. The default is 10 threads. In Orion we can peak at around 500 and have at times used over a 1000. So I was scratching my head, and then I realised that Grizzly has a connection pool with defaults in the thousands but only 10 worker threads. Connections enter a queue and then get serviced. SEDA.

In the end I decided to set the number of worker threads to the number of connections in the database connection pool (75) plus 25 threads for service from cache. Ehcache is pretty fast, so we can have a small thread pool to service requests purely from cache.

As someone with a reasonably long memory I thought it appropriate to acknowledge SEDA and reflect on its appearance in a mainstream Java application server.

Posted by gluck at 05:32 PM | Comments (0)

July 15, 2006

Tuning Memory Use in ehcache

In ehcache-1.2 the DiskStore asynchronous spooling was reworked and made much faster. It is now possible to fill the spool very quickly. This gives great cache performance but creates a new problem, temporary memory consumption in the spool thread.

The problem has arisen because data is hitting the spool so fast now that the actual number of element held in the MemoryStore and the spool of the DiskStore can easily exceed the maximum MemoryStore size. You can get memory spikes. Ultimately, I might introduce soft references, so that elements in the spool can simply be reclaimed. This would need to be a configuration option, because some apps are a bit fussy about their cache elements up and disappearing.

The memory spikes, if they are high enough, can cause OutOfMemory errors. For some reason, these would often occur in the flushSpool() method. Ehcache 1.2.0_01 and 1.2.1 contain hardening measures so that these do not cause trouble. If one occurs, that section of the spool that is being written is discarded. So it degrades down to something similar to a SoftReference solution. But it would be better to minimise the occurrence of this behaviour.

This article documents some investigations I have made and a solution I have. The test involves creating 5500 10000 byte objects and putting them in a cache. These go into the MemoryStore and overflow to the DiskStore immediately. The spool thread then takes care of writing them. Memory use spikes to about 55MB and then drops. The disk store was modified to call System.gc() every 700 Elements spooled.

Figure 1 shows the original implementation. The System.gc() calls have little effect. The memory cannot be released until the flushSpool() method completes. Though not shown, forcing a gc in the profiler after elements have been written will return memory back to about 14MB.

Figure 1: Old Disk Store

Figure 2 shows the memory profile after one small change. As the flushSpool() method iterates through the elements to persist them, set each one to null in the array holding references to them, after they are written. The System.gc() every 700 elements actually reclaims memory.

Figure 2: Old Disk Store with just dereference after array element use


Figure 3 shows an implementation which uses the technique from Figure 2 plus two more:

1. Do the work split up around 5 methods. The theory here is that some JDK implementations do not actually reclaim memory for references that are dereferenced in the same method. They wait for the method to return first.
2. ByteArrayOutputStream creates lots of temporary byte[] using System.arrayCopy(). It starts with 32 bytes, and then bit shifts to the left by one, as more memory is required. So the sequence is:
   
   32
   64
   128
   256
   512
   1024
   2048
   4096
   8152
   16304
   32608
   65216
   ...
   
   For our 10000 byte offects, 10 byte arrays are created by ByteArraytOutputStream as the stream is written, totalling 32616 bytes. We create a subclass that uses as its starting point the average size of each entry in the DiskStore.
   

This implementation's memory use is much smoother.

Figure 3: New Disk Store


The new improved implementation will be released in ehcache-1.2.0_02 and ehcache-1.2.2.

Posted by gluck at 05:16 PM | Comments (1)

July 09, 2006

How We Solved our Garbage Collection Pausing Problem

I had our main J2EE app at work with 9 second pauses. These would happen on average every 50 seconds. Needless to say this was a huge performance problem. Pauses are caused by major garbage collections. Minor garbage collections do not cause pausing. Pausing means nothing, absolutley nothing, gets done in your app. 9 seconds is a long time. The peaks were up to 15 second.

We tried quite a few garbage collection settings. They each behaved differently but could not be considered better. In the end we consulted some engineers at Sun who, after analysing our verbose gc logs, gave us the following piece of black magic:

java ... -XX:+DisableExplicitGC -XX:+UseConcMarkSweepGC -XX:NewSize=1200m -XX:SurvivorRatio=16

The reasoning for each setting is as follows:

-XX:+DisableExplicitGC - some libs call System.gc(). This is usually a bad idea and could explain some of what we saw.
-XX:+UseConcMarkSweepGC - use the low pause collector
-XX:NewSize=1200m -XX:SurvivorRatio=16 - the black magic part. Tuning these requires emprical observation of your GC log, either from verbose gc or jstat ( a JDK 1.5 tool). In particular the 1200m new size is 1/4 of our heap size of 4800MB.

What was the result? Major GCs and their attendant pauses reduced to 2 per day, from once every 50 seconds. Mean response times dropped from seconds to milliseconds. All in all, one of the best results I have achieved this year. Thanks Sun.

Posted by gluck at 07:04 PM | Comments (3)

May 02, 2006

yDoc ant target and maven plugin config

yDoc (http://www.yworks.com/en/products_ydoc.htm) is a JavaDoc doclet that adds wonderful, clickable UML diagrams to your JavaDoc.

I have been pestering these guys for a few years for an open source license which they gave me today.

As part payback I thought I would document my configuration of yDoc in my javadoc ant target and my maven pom.xml plugin config. The yDoc guys do not provide either and it can be fiddly to get going.

Ant Target

Maven reporting plugin config

You can see the results of this configuration in ehcache's online JavaDoc.

Posted by gluck at 06:31 AM | Comments (0)

March 26, 2006

Weighing in on the Gosling flame war over scripting languages

I want to way in on this flame war that James Gosling has triggered with his mild criticism of scripting languages. I think it is unfair and unhelpful. Here is my two cents worth.

I am still feeling my way on Ruby and Ruby on Rails. I spent a half day with Prag Dave, have done a ROR tutorial and have the two Ruby books. I also read a book on comparative programming languages to see what the experts thought.

I like the idea of having a few tools on the toolbelt, thus avoiding the "if all you have is a hammer, every problem looks like a nail" phenomenon.

I was not a particularly early adopter of Java. I got going with it about 1.2. The early Java was high on hype and low on class libraries and frameworks. And it was slow.

It seems to me that Bruce Tate, Dave Thomas and David Hansson amongst others are out flaming people rather than dealing with the criticisms raised.

Bruce should get over advancing Ruby by attacking Java. His beyond series seems to overlook the Tiobe stats. Java is growing in popularity, not declining. The thing that is declining is Perl. He should start thinking in terms of beyond Perl.

Dave Thomas ridicules the idea of an IDE. Perhaps when Ruby's class libraries are too big for him to keep in his head he will reconsider. One new thing that has made JavaScript and CSS nicer to do was the adding of those languages to IntelliJ's IDE. I have had enough of groundless claims that real projects can be done in Ruby 10 times faster. I know of a project done late last year which took three times as long to do in Ruby than was estimated for it to be done in Java.

David attacks Java too. I have seen complaints about verbosity which overlooked the IDE generation of same and the resulting readability. Having attended sessions on Ruby and a session given by Guido Van Rossum on Python, a common criticism against both is that there are no really large systems written in them. I am not sure if there never will be. But boring maintainability issues like readability are a factor.

This sort of behaviour turns me off in a big way. I think these guys should be mindful of the vast numbers of us that are still in the undecided camp on Ruby.

So, what will it take to convince me one way or another? Not attacks on my current language of choice, that's for sure. I need to resolve the following preliminary issue list:

1. No decent IDEs
2. very few libraries. I think there are about 500 gems compared with around 20000 Java libs in open source alone
3. no native threads
4. An active record one: Active Record does not use prepared statements and executes very poorly on a database. Now maybe a language/framework does not always need to be fast, but it should allow for efficient execution on a database, which is often shared infrastructure.
5. Slow? execution speed. I need to do my own measurement on this one but I have it on my radar as a problem.
6. The possibility of developers creating unmaintable code with clever tricks. Not sure on this one either until I have more experience.

If anyone can help out with any of those I am happy to hear from you.

On the positive side, Ruby is open source, follows the Unix way, has been around for a decade and is loved by a lot of people I respect greatly. Most open source responds well to criticism and gets better. Let's hope thats what happens with Ruby rather than Ruby not getting better but the criticisers being taken out.

As to Java, what I have personally seen in the last year is large scientific C++ applications moving to the Java platform. I was personally involved in a 12 million line one of these. The other is large mainframe COBOL and BASIC systems moving to Java. Along with the growth shown on tiobe.com, I think Java is quite safely in the mainstream of computing. On the Tiobe index Java is ranked number 1 and Ruby number 22.

Anyone for Haskell? Sounds like an interesting language. It is ranked number 70.

Having initially posted this I wondered what Hani over at The Bile Blog was making of all of this. Here it is: http://jroller.com/trackback/fate/Weblog/tss_bringing_out_the_best

Posted by gluck at 08:21 PM | Comments (2)

ehcache-1.2 beta 5 released

Today I released ehcache-1.2 beta 5. Quite a few people have been playing with the new features. I think the whole listener area has been very well road tested now. Ditto with the DiskStore performance redesign. I am also very happy that people are playing with the distributed cache stuff. Fixed a couple of bugs in that area. This release package up fixes to all known issues in the new 1.2 feature set.

What makes me enormously happy is the time and care other developers take to report bugs and patches. If they see a broken window, they know I want to fix it and so they tell me about it. And they sometimes give me the pane of glass too. Great stuff. Release early, release often.

Posted by gluck at 03:14 AM | Comments (0)

March 22, 2006

Comparative Cache Performance Numbers

Some guys have created a java cache test tool called cache4j_perfomance_tester.

The results for ehcache-1.1 and ehcache-1.2 follow.

According to their test methodology both versions of ehcache are the fastest for all three scenarios. Compared with oscache, ehcache was 4 times faster for two of the scenarios and twice as fast for one.

I find these results interesting. I had the JCS guy Aaron Smuts complaining about the old performance numbers I had up for JCS. They were quite old so I removed them. I didn't have the time to do my own analysis but I am happy that someone else has. Ehcache is in good shape.

ehcache-1.1

ehcache-1.2




Greg Luck

Posted by gluck at 12:21 AM | Comments (0)

March 16, 2006

Developing a project site with Maven 2.0

The last week or so I have been developing a new version of the http://ehcache.sf.net project web site with Maven 2.0.

Why? The old site featured a couple of toilet roll web pages, the longest of which was 2026 lines. That's a long toilet roll. With the upcoming release of 1.2 and a lot more features to write about, something had to give. I knew I wanted a nice side nav bar and to provide some extra info about the project over what I had been doing. I thought about doing all the redesign by hand or moving the site to confluence, which I have an open source license for. But I am lazy.

I met the guys from Maven 2.0 in Portland at last year's OSCON and I knew that Mergere was paying about 8 guys to work on Maven 2.0 full time. So I went to take a look at what they were up to. It looked like there was a lot of stuff you get free out of the box.

A week later and the site is done. I think it is a big improvement over what was there, and I think Maven 2.0 ended up being a pretty good choice to to it. From here on it will be very easy to maintain.

So, what did I think?

The Good

• APT, Maven 2's simple documentation adopted tag language, is simple with a very easy to learn syntax and high productivity.
• You can call into Ant for stuff that Maven does not do. Nice.
• Tons and tons of great reports and site output that are generated from your pom.xml or with a little plugin setup.
• The #maven channel at irc.codehaus.org. Thanks guys!
  

The Bad

• APT is limited. No superscripts. Difficult to combine multiple things or it gets confused.
• Maven is hard to get into.
  
• Very poor documentation. I logged a few bugs on this. Unless the maven guys want to keep getting bugged on IRC they need to write more and better doco.
  
• Maven 2 has a lot less plugins than Maven 1.

The Ugly

• Doxia barfs on APT when you have a tag mismatch with an error like "missing '>'". Try finding that in a 500 line document with no line number indicated.
  
• A general lack of useful error messages which pinpoint what is really going on. SCM and Clover plugins are partiularly bad.
• MOJO properties don't seem to have a standard naming convention. I got caught up with seemingly random variations around the_tag, theTag and thetag.

Where to now? I am not yet building ehcache on Maven. I did restructure the code to follow the Maven directory structure. The Ant build works very well however and I am wary about how much time it will take to get everything working. Maybe version 2.1.

Would I recommend it? If you like the new ehcache site, you can get the same thing for your own project , using ehcache as an example, in no time. But if you do. copy the ehcache pom.xml. Those 424 lines of config were hard won.

Posted by gluck at 08:40 PM | Comments (0)

March 04, 2006

Java vs Ruby: the keystroke effect of good IDEs

Years ago I used to use vi for all of my programming. I also used JEdit for a while. I remember being in a Sun conference about 7 years ago when the speaker asked for a show of hands as to whether the assembed programmers were using text editors or IDEs. About 90% were using text editors. She asked "Why?" The answer some audience members gave was that the Java IDEs were crap. And so they were.

However since then they have come along way. Today the rare individual you encounter who does not use an IDE is a Unix hacker who has been living in a cave. Or a Ruby fanatic. The Ruby guys tend to use TextMate, which is a nice text editor. I have been using it the past few days. Apart from the .conf files it gets its cursor out of sync on, it seems to work well. So why aren't they using IDEs? I think the answer is the same we gave at that Java seminar 7 years ago. I have tried out about 5 of them now and that is my conclusion. ActiveState's Komodo is probably the best.

Dave Thomas in his Ruby talk at OSCON dealt with the issue differently; he was openly disparaging of the whole idea of IDEs or any language that was either compiled or verbose enough to require one. His argument was that, in Ruby, you write so little code that things like autocomplete do not matter. And there is no compile to do.

In Java by contrast, I find autocomplete, syntax highlighting, quick documentation lookup and so on highly productive.

Continuing the verbosity theme, over at Loud Thinking David Heinemeier Hansson was reduced to giggles by the following Java code snippet:

UploadedFile uploadedFile = (UploadedFile) fileUpload1.getUploadedFile();

It is certainly far more verbose than the equivalent Ruby code. On the other hand it is a familiar idiom; one which is very easy to parse. It has high readability. What about its writability? Is it really that much more work? I thought I would take a similar example and count the key strokes required to produce it in IntelliJ IDEA. Here is the sample piece of code from ehcache:

CacheEventListener cacheEventListener = (CacheEventListener) iterator.next();

It contains 77 characters, including whitespace characters, starting from the first "C". Is this really 77 keystrokes? Following is the exact key sequence I used to write the above code:

C E L CTRL-SPACE RET SPACE CTRL-SPACE RET SPACE = SPACE i t CTRL-SPACE RET . CTRL-SPACE RET ; ALT-ENTER RET

That is a total of 21 keystrokes, not 77.

Ok, so it was quick to type. Is the instance variable name silly? "cacheEventListener" is the first suggestion made by IntelliJ, as would "uploadedFile" be. I don't think it is silly; I think it is readable. That you use the class name indicates to the reader that it is a CacheEventListener with no other notable qualities, thus the simple name.

There are many other code examples that require far less keystrokes to write. IDEs like IntelliJ give automation support to structures like looping and exceptions and many others. Rather than the 4:1 ratio in the above example, these can get closer to 10:1.

I like the idea of having lots of tools on my tool belt. I have had a little time lately and have been working my way through Ruby on Rails. I am pretty slick on vi and find TextMate easy enough. But every character of code is a key stroke I typed into the keyboard. And every little bit of API I had to go and lookup in my browser. So lets not kid ourselves: the Ruby community would be much more productive with a great IDE like IntelliJ IDEA.

Posted by gluck at 10:09 PM | Comments (5)

February 05, 2006

Ruby off the rails

Posted by gluck at 02:41 PM | Comments (0)

January 29, 2006

ehcache goes distributed

For the past couple of years I have been saying that ehcache will never be distributed. Ehcache grew out of uncommitted patches to the Apache JCS cache. JCS implemented the JCACHE JSR. It had lots and lots of features. The reason for the patches was that the in-memory and disk stores had memory leaks and threading problems.

The idea of ehcache was to bring it back to something very simple and then get all the kinks out. Ehcache has been successful in achieving that aim.

Over the last year there have been feature requests and offers of help to implement those features. Surya Suravarapu in particular volunteered for one day a week. He was very keen to make ehcache distributed. We decided to start our collaboration with something more prosaic: some new cache policies. He added FIFO and Least Frequently Used. In its latest incarnation it has become Less Frequently Used, for performance reasons, which is an interesting story in itself.

Steve from jofti.com then requested event listener functionality. He agreed to give input into the design and review its implementation. After a few iterations we got that done to his satisfaction and it is now being used from his Jofti cache indexer and search library.

Paul Hammant and Jo Walnes took a look at ehcache during OSCON2005. They argued that the singleton CacheManager was evil and should be replaced by constructor-based instances, for which it is the callers responsibility to maintain references. I agreed with that and so added the functionality, with a few safety provisions to prevent disk store collisions. The feature turns out to be particularly useful in testing distributed caches on a single machine.

At the conference I ran into James Strachan. James is one of the JCACHE people and the prime mover behind many projects, including the notable ActiveMQ. We discussed distribution and how it should be done. He agreed with me that multicast was fine for discovery but not for delivery. Then I ran into the guys from Sourcelabs, who are keen ehcache enthusiasts and may offer support for it. They were saying I really needed to make it distributed. There voice is of course added to the many requests for the same feature.

These ideas bounced around for a while. I was also doing some architecture work for a distributed system at the time. I had also been doing lots of reading on distributed system. In August I decided to commit the time to make it happen.

I was keen on doing it test first and keeping it as simple as possible. I decided to start with the primitives in the JDK and see where that took me.

One thing that quickly emerged was the disparate views on what makes for the best distribution mechanism. Surya was keen on a JGroups approach. Another group at BNP were interested in a JMS based approach. To accommodate these needs I decided to make the distribution framework pluggable. I am interested in using ehcache as a test bed to explore different distribution ideas. Of course, it takes at least two implementations to make sure you have your plugin framework right. I have two for discovery and one for delivery right now. Hopefully another delivery implementation will get done that can prove or force modification to the framework.

A lot of hard work and 5 months later and ehcache 1.2 beta 3 is out with distributed caching built-in. Thanks to everyone who lent a hand.

I now want to get it in the hands of as many people as possible and get the bugs out. I need to do further documentation. I need to see how distribution can best be utilised for applications like Hibernate and web pages. Ehcache-1.1 is now 13 months old. An update will be timely.

The funny thing is, ehcache now had a similar feature set to the original JCS. Perhaps it would have saved me a lot of time and effort if the JCS developers could have just accepted the patches I put in two and a half years ago.

Posted by gluck at 07:42 PM | Comments (1)

January 23, 2006

And the Programming Language of 2006 is ...

Java. Yes, Java.

http://www.tiobe.com/tiobe_index/index.htm is the monthly programming language index. It tracks the popularity of programming languages. Java won the 2006 Programming Language award because it was the language that increased in popularity the most during 2005. By 4.77%.

JDK 1.6

JDK 1.6 proved to be quite popular and caused an upswing.

Anecdotally the big news for me in 2005 was the movement of code written in C++ to Java. I spent time on a large project last year which was moving 12.5 millions line of C++ to Java. It was a performance critical desktop application. The performance gains of Java in recent releases were enough. The addition of generics was very attractive to the C++ guys. They thought the time to move to Java was "just right".

Cold Fusion lives again

One interesting phemomenon is the rising popularity of Cold Fusion in Tiobe's index. Cold Fusion is Java based an deployable as a WAR or EAR. Its big feature is productivity. I think of it as a Java based Domain Specific Language. Anecdotally Cold Fusion seems to be making headway again for web apps.

The Decline? of Dynamic Typing

The other Tiobe identified trend is the relative decline of dynamically typed languages.

Where does this trend leave the dynamically typed Ruby? From the O'Reilly conference and amongst my peers, there is quite a lot of interest in Ruby. However its popularity in the Tiobe index is only 22, though it is increasing in popularity at the expense of Perl. So, perhaps the book series should be called "Beyond Perl", rather than "Beyond Java".


Dynamic vs Static Typing is a subject of programming language textbooks. Which is why I am reading one. Some good stuff on this topic is available here. Robert W. Sebesta, in his Concepts of Programming Languages has this to say:

• Dynamic type binding causes programs to be less reliable, because the error detection capability of the compiler is diminished...
• ... languages with dynamic type binding... must be implemented with interpreters rather than compilers.
  
• Perhaps the greatest disadvantage of dynamic type binding is cost. The cost of implementing dynamic type binding is considerable.

And this on page 223:

"The penalty for static checking is reduced programmer flexibility. Fewer shortcuts and tricks are possible. Such techniques, though, are now generally held in low esteem".

In his view, Java and C# are on the ascendancy partly because they make the right set of tradeoffs in language features. They combine the right mix of readability, writability and reliability. I have been tinkering with Ruby since the O'Reilly conference last year, when I attended a half day session with Dave Thomas. I am hoping to get the time to do a full comparison of Java and Ruby in those terms.

Posted by gluck at 12:52 AM | Comments (2)

November 20, 2005

Architectural Priniciples

I am doing some work at the moment summarising an architecture that I have developed over the last 6 months. The hard part was thinking about where to start. In thinking about it I realised there were some priniciples that informed all of the architectural choices that had been made. The team I am with agree with these. But I can see that others may strenuously disagree with some of them.

Open Standards over Proprietary Products

There are two competing ideas in the software industry.

One is that a single vendor can satisfy all current and future requirements. For the past 60 years this has first meant IBM and later meant Microsoft. In this approach you embrace proprietary everything on the assumption that you will only ever work with that vendor's products. Examples of this approach are EBCDIC (a proprietary ASCII), NetBeui, IPX, RTF, DOS, .NET and Oracle.

The other idea is Open Standards. This is where you assume there will be multiple vendors. Getting everything to work requires standards open to each vendor to implement. Examples of this latter approach are TCP/IP networking, Unicode, the Internet, World Wide Web, Linux, Web Services, SQL2003 and J2EE.

Many of our customers want choice. Most of them will in the end want to use our software with leading prorietary products. We need to offer first class integration with those. But we can also service customers that want to pursue Linux and other open standards. There primary motivation is to achieve commodity pricing of technologies that have become commodities.

Integration in many cases is easy because almost all modern proprietary products implement large numbers of open standards themselves. Examples are Oracle (SQL2003), Web Services (.NET), TCP/IP (Windows 95 onwards).

Open standards can be used in by using proprietary products in standards-based ways or by using commodity priced products such as open source/cheapware such as JBoss, JRun and PostGIS.

Acquire components rather than build them

In examining the software we have built, over 98% of our runtime code is acquired rather than written in-house. When we add Eclipse and JBoss to the mix, together with tools our code drops to a fraction of a percent.

This is a healthy thing. Application development is about gluing infrastructure and plumbing together and only writing what represents the true IP of an organisation. Both initial costs and maintenance costs are kept down.

Use Open Source/Freeware/Cheapware for Commoditised components

There are three forms of commodity priced software components: Open Source , Freeware and Cheapware .

There are 16,630 Java projects (Nov 2005) on http://sourceforge.net, the most available for any language.

Some examples of commoditised components used in our project are:

* BeanShell - dynamic scripting using Java syntax
* jBPM - Java Business Process Management
* Hibernate - An object relational mapping library
* geotools - Geospatial Tools
* JBoss - embedded application server
* Java - free programming language and runtime environment
* Linux - a Unix like operating system, and one of two supported operating systems

Cross Platform by Default

Consistent with other principles everything we build is cross-platform by default. To avoid costly integration problems, some developers develop on Linux and we continuously integrate on Linux. It is easier to work on the server on Linux if the client also runs on Linux.

There is a very low cost (a few days so far) to support Linux in our process.

Go with Mainstream Approaches by Default

There are always many ways of solving a problem. We go with a mainstream solution until or unless an alternative can be justified. Some of these mainstream approaches are:

1. The full J2EE as the underlying platform
2. Stateless Session EJBs for the service layer, exposed as RMI Remote classes, Web Services Endpoints, local services for the web applications
3. Security based on J2EE Declarative Security
4. JSPs as the default web view
5. RMI for remoting to EJBs
6. JMS for parallel execution on the server
7. Eclipse application clients

Agile Approach

Detailed upfront design usually fails because:

1. there is insufficient understanding of the problem and the technologies upfront
2. changes occur that invalidate the design

In Agile, you do architecture and design just in time. What do you do about varying skill levels and keeping the architecture consistent? You create the the first-ofs on a just in time basis which then allows other groups to copy these as templates.

Another aspect is that implementations change as more requirements are implemented. Care must be taken to avoid fragilities in the architecture for this to succeed. In practice this means monitoring and correcting technical debt through continous refactoring.

Simplicity

Complexity must pay for itself.

Simplicity is subjective. The following is a list of things which can be considered simple:

* 10 lines of code versus 100 lines of code
* code that is expressed in terms of a well-known pattern rather code that does not follow a pattern
* Use of a library or framework understood by the majority of the developers, compared with one that is not
* Documented code and libraries compared with those that are not.
* Reuse of code compared with single use.

Pure Java by default

Being pure Java supports simplicity, cross-platform operation and an Agile approach. Callouts to other languages should be carefully managed.

Posted by gluck at 03:33 AM | Comments (0)

August 28, 2005

Eclipse RCP/OSGI Classloader HOWTO

I am working on a new desktop application using Eclipse Rich Client Platform (RCP). Eclipse RCP applications are comprised of plugins. The plugins are not any ordinary old plugins. They are Open Standard Gateway Initiative (OSGI) plugins.

One feature of OSGI plugins is classpath separation. One plugin can be running dom4j 1.5 and another dom4j 1.6, and there is no conflict. A nice idea. Unfortunately, a combination of an extraordinarily poor user interface in the Eclipse IDE coupled with a classloader that goes out of its way not to load classes, makes it very difficult for the newly initiated to get going.

While I was away at OSCON2005, a developer on the project spent 8 full work days trying to figure out how to get our plugin to a load classes from a jar. While I was at OSCON I met Scott Delap. Scott is the maintainer of clientjava.com and knows a lot about RCP. Having taken a quick look at the problem myself and finding no documentation on to get the Eclipse IDE to do the right magic I called Scott. He helped through a fairly involved process, and one that is dependent on order and I was then able load jars!

With agreement from Scott, I am making the HOWTO available below. It covers:

• Loading classes from jars.
• Exporting classes to make them visible to other plugins.
• Swapping in another classloader to load resources.

Eclipse 3.1 OSGI classloading HOWTO

Posted by gluck at 10:00 AM | Comments (0)

August 16, 2005

hloader 0.7 released - easy object loading for Hibernate3

Hloader is a tool for persistent object instances, expressed in XML, into a database using Hibernate. It is a Hibernate loader, thus the name.

It can be used for:

• loading reference data
• loading bootstrap data
• loading test setup data

More than anything, hloader extends an near zero cost of change to this data, thus supporting the Agile approach of evolving your database design along with you code.

This version brings support for Hibernate 3 and much comprehensive documentation.

See hloader.sf.net.

Posted by gluck at 12:29 PM | Comments (0)

June 20, 2005

Jpam 0.5 released - Linux, Solaris and Mac OS X supported

Jpam 0.5 released with source and binaries for Linux x86, Linux x86_64, Mac OS X and, new in this release, Solaris.

Jpam is the first general purpose, multi platform Java-PAM bridge. PAM, or Pluggable Authentication Modules, is a standard security architecture used on Linux, Solaris, Mac OS X and other Unix systems. JPAM permits the use of PAM authentication facilities by Java applications running on those platforms. These facilities include:

1. account
2. auth
3. password
4. session

Both the JAAS API and a direct method call API are supported.

Java developers, wary of writing native code, have relied on security schemes based on XML configuration files, JNDI and JDBC. With hardware token based security schemes this is not possible. JPAM allows seamless access to more than 70 PAM modules, including those for SecurId and Radius. All hardware token vendors release PAM modules for their products.

This is a stable release. JPAM has been now been used in production for more than five months and extensively tested on different platforms.

On Linux, this release has been tested on Red Hat AS 3 and 4, Fedora Core 3 and 4, SuSe Linux 9 and Novell Linux Desktop. On Mac OS X it has been tested on Mac OS X 10.4.1 Tiger. On Solaris it has been tested on Solaris 9.

Posted by gluck at 02:20 AM | Comments (0)

May 20, 2005

Oracle Streams and JMS: A dead combination?

Oracle Streams is supposedly a JMS implementation. After a month of waiting on Oracle support as to why some of the basic examples did not work I got: Note:154777.1 Subject: Unable to Create Receiver on a Raw Queue Type: PROBLEM Status: PUBLISHED Content Type: TEXT/X-HTML Creation Date: 20-JUL-2001 Last Revision Date: 13-AUG-2001 * fact: Oracle Server - Enterprise Edition 8.1.7 * fact: Advanced Queuing (QUEUE) * symptom: Runtime error using the Java Messaging Service (JMS) * symptom: java.lang.NullPointerException * symptom: oracle.jms.AQjmsSession.createReceiver(javax.jms.Queue) * cause: Access to queues with a RAW payload via the JMS API is not supported. fix: 1. Use the native Java API - oracle.AQ.* to access the RAW queue. 2. Use one of the predefined payload types for JMS. In this case SYS.AQ$ _JMS_TEXT_MESSAGE was used. For more information on JMS and the Java AQ API, refer to : Oracle8i Application Developer's Guide - Advanced Queuing. The native API is not deprecated. So all of this adds up to Oracle Streams and JMS being a dead combination on Oracle.

Posted by gluck at 07:07 AM | Comments (0)

May 10, 2005

Running/Debugging JBoss within IntelliJ IDEA

I usually use Orion, and find debugging it from within IntelliJ is great for debugging. You can set breakpoints in EJBs or other server side code, start it with the Debug button and then wait for your breakpoints to be reached. You can then evaluate expressions, recompile classes and so on.

I set out to do the same thing with JBoss, which I am using for my current project and got it working. It works even better than Orion, because Orion's classes are obfuscated, whereas JBoss' are not.

Step 1 Create a JBoss module

Open File -> Settings -> Paths and create a new Java module as shown below:

(Click to expand)

Step 2 - Add a new Run/Debug Configuration

Add a configuration as shown below.

(Click to expand)

Note that the VM parameters do not show up fully in the screen shot. Copy and past the following:

 -Xms128m -Xmx512m -Dprogram.name=run.sh -Djava.endorsed.dirs=/convergence/configunit/jboss/lib/endorsed

Using it

1. Make sure that JBoss is not running on your machine
2. Open Run -> Debug and select the Application tab. Select JBoss and then click on the Debug button. You should see JBoss' start up messages appearing in the IDEA console.

Update: I had to set this up on a new project and got some errors. After playing with I discovered that it is important to exclude any other modules and classes from the JBoss module. Then it works.

Posted by gluck at 03:34 AM | Comments (1)

Integrating with Oracle 11i: Figuring Out the APIs

Introduction

I am working on JMS integration with Oracle 11i. It looks like multiple approaches are possible. I went with implementing an adapter in JBoss using a Message Driven Bean receiving messages from a topic and then using Oracle 11i's public stored procedure or other APIs to update data.

Sound simple? It should be. A lack of Oracle developer documentation turns this into a nightmare. The intent here is to remedy, in a small way, that lack of documentation.

The techniques are illustrated with an exam ple. It is using getOrganizationRec in Oracle's HZ_PARTY_V2PUB package.

The APIs

The HZ_PARTY_V2PUB Package

This is the PL/SQL API used for integrating with Oracle Financials. Documentation about these API's and record structures can be found in Chapter 3 of the Oracle Trading Community Architecture Technical Implementation Guide located here : http://download-west.oracle.com/docs/cd/B16981_02/current/acrobat/115hztig.pdf.

get_organization_rec, shown below, uses a ORGANIZATION_REC_TYPE.

PROCEDURE get_organization_rec (
    p_init_msg_list                    IN      VARCHAR2 := FND_API.G_FALSE,
    p_party_id                         IN      NUMBER,
    p_content_source_type              IN      VARCHAR2 := G_MISS_CONTENT_SOURCE_TYPE,
    x_organization_rec                 OUT     NOCOPY ORGANIZATION_REC_TYPE,
    x_return_status                    OUT NOCOPY     VARCHAR2,
    x_msg_count                        OUT NOCOPY     NUMBER,
    x_msg_data                         OUT NOCOPY     VARCHAR2
);

An ORGANIZATION_REC_TYPE is a RECORD. You cannot use pl/sql RECORD types in jdbc calls: PL/SQL TABLE, BOOLEAN, and RECORD Types. You’ll have to create a sql type for the record, then you can use it using a jdbc struct. Note that this limitation is present in both the thin and oci8 versions of Oracle's drivers.

See:

1. http://download-west.oracle.com/docs/cd/B10501_01/java.920/a96654/ref.htm#1005988
2. http://download-west.oracle.com/docs/cd/B10501_01/java.920/a96654/oraoot.htm#1039477

TYPE organization_rec_type IS RECORD(
    organization_name               VARCHAR2(360),
    duns_number_c                   VARCHAR2(30),
    enquiry_duns                    VARCHAR2(15),
    ceo_name                        VARCHAR2(240),
    ceo_title                       VARCHAR2(240),
    principal_name                  VARCHAR2(240),
    principal_title                 VARCHAR2(240),
    legal_status                    VARCHAR2(30),
    control_yr                      NUMBER,
    employees_total                 NUMBER,
    hq_branch_ind                   VARCHAR2(30),
    branch_flag                     VARCHAR2(1),
    oob_ind                         VARCHAR2(30),
    line_of_business                VARCHAR2(240),
    cong_dist_code                  VARCHAR2(2),
    sic_code                        VARCHAR2(30),
    import_ind                      VARCHAR2(30),
    export_ind                      VARCHAR2(30),
    labor_surplus_ind               VARCHAR2(30),
    debarment_ind                   VARCHAR2(30),
    minority_owned_ind              VARCHAR2(30),
    minority_owned_type             VARCHAR2(30),
    woman_owned_ind                 VARCHAR2(30),
    disadv_8a_ind                   VARCHAR2(30),
    small_bus_ind                   VARCHAR2(30),
    rent_own_ind                    VARCHAR2(30),
    debarments_count                NUMBER,
    debarments_date                 DATE,
    failure_score                   VARCHAR2(30),
    failure_score_natnl_percentile  NUMBER,
    failure_score_override_code     VARCHAR2(30),
    failure_score_commentary        VARCHAR2(30),
    global_failure_score            VARCHAR2(5),
    db_rating                       VARCHAR2(5),
    credit_score                    VARCHAR2(30),
    credit_score_commentary         VARCHAR2(30),
    paydex_score                    VARCHAR2(3),
    paydex_three_months_ago         VARCHAR2(3),
    paydex_norm                     VARCHAR2(3),
    best_time_contact_begin         DATE,
    best_time_contact_end           DATE,
    organization_name_phonetic      VARCHAR2(320),
    tax_reference                   VARCHAR2(50),
    gsa_indicator_flag              VARCHAR2(1),
    jgzz_fiscal_code                VARCHAR2(20),
    analysis_fy                     VARCHAR2(5),
    fiscal_yearend_month            VARCHAR2(30),
    curr_fy_potential_revenue       NUMBER,
    next_fy_potential_revenue       NUMBER,
    year_established                NUMBER,
    mission_statement               VARCHAR2(2000),
    organization_type               VARCHAR2(30),
    business_scope                  VARCHAR2(20),
    corporation_class               VARCHAR2(60),
    known_as                        VARCHAR2(240),
    known_as2                       VARCHAR2(240),
    known_as3                       VARCHAR2(240),
    known_as4                       VARCHAR2(240),
    known_as5                       VARCHAR2(240),
    local_bus_iden_type             VARCHAR2(30),
    local_bus_identifier            VARCHAR2(60),
    pref_functional_currency        VARCHAR2(30),
    registration_type               VARCHAR2(30),
    total_employees_text            VARCHAR2(60),
    total_employees_ind             VARCHAR2(30),
    total_emp_est_ind               VARCHAR2(30),
    total_emp_min_ind               VARCHAR2(30),
    parent_sub_ind                  VARCHAR2(30),
    incorp_year                     NUMBER,
    sic_code_type                   VARCHAR2(30),
    public_private_ownership_flag   VARCHAR2(1),
    internal_flag                   VARCHAR2(30),
    local_activity_code_type        VARCHAR2(30),
    local_activity_code             VARCHAR2(30),
    emp_at_primary_adr              VARCHAR2(10),
    emp_at_primary_adr_text         VARCHAR2(12),
    emp_at_primary_adr_est_ind      VARCHAR2(30),
    emp_at_primary_adr_min_ind      VARCHAR2(30),
    high_credit                     NUMBER,
    avg_high_credit                 NUMBER,
    total_payments                  NUMBER,
    credit_score_class              NUMBER,
    credit_score_natl_percentile    NUMBER,
    credit_score_incd_default       NUMBER,
    credit_score_age                NUMBER,
    credit_score_date               DATE,
    credit_score_commentary2        VARCHAR2(30),
    credit_score_commentary3        VARCHAR2(30),
    credit_score_commentary4        VARCHAR2(30),
    credit_score_commentary5        VARCHAR2(30),
    credit_score_commentary6        VARCHAR2(30),
    credit_score_commentary7        VARCHAR2(30),
    credit_score_commentary8        VARCHAR2(30),
    credit_score_commentary9        VARCHAR2(30),
    credit_score_commentary10       VARCHAR2(30),
    failure_score_class             NUMBER,
    failure_score_incd_default      NUMBER,
    failure_score_age               NUMBER,
    failure_score_date              DATE,
    failure_score_commentary2       VARCHAR2(30),
    failure_score_commentary3       VARCHAR2(30),
    failure_score_commentary4       VARCHAR2(30),
    failure_score_commentary5       VARCHAR2(30),
    failure_score_commentary6       VARCHAR2(30),
    failure_score_commentary7       VARCHAR2(30),
    failure_score_commentary8       VARCHAR2(30),
    failure_score_commentary9       VARCHAR2(30),
    failure_score_commentary10      VARCHAR2(30),
    maximum_credit_recommendation   NUMBER,
    maximum_credit_currency_code    VARCHAR2(240),
    displayed_duns_party_id         NUMBER,
    content_source_type             VARCHAR2(30) := G_MISS_CONTENT_SOURCE_TYPE,
    content_source_number           VARCHAR2(30),
    attribute_category              VARCHAR2(30),
    attribute1                      VARCHAR2(150),
    attribute2                      VARCHAR2(150),
    attribute3                      VARCHAR2(150),
    attribute4                      VARCHAR2(150),
    attribute5                      VARCHAR2(150),
    attribute6                      VARCHAR2(150),
    attribute7                      VARCHAR2(150),
    attribute8                      VARCHAR2(150),
    attribute9                      VARCHAR2(150),
    attribute10                     VARCHAR2(150),
    attribute11                     VARCHAR2(150),
    attribute12                     VARCHAR2(150),
    attribute13                     VARCHAR2(150),
    attribute14                     VARCHAR2(150),
    attribute15                     VARCHAR2(150),
    attribute16                     VARCHAR2(150),
    attribute17                     VARCHAR2(150),
    attribute18                     VARCHAR2(150),
    attribute19                     VARCHAR2(150),
    attribute20                     VARCHAR2(150),
    created_by_module               VARCHAR2(150),
    application_id                  NUMBER,
    do_not_confuse_with             VARCHAR2(255),
    actual_content_source           VARCHAR2(30) := G_SST_SOURCE_TYPE,
    party_rec                       PARTY_REC_TYPE:= G_MISS_PARTY_REC

);

The HZ_PARTY_V2PUB_JW Package

This package is the same as the HZ_PARTY_V2PUB package except that the procedure signatures are changed to work with JDBC. The "_JW" suffix stands for Java Wrapper.

The problem with this API is that it is extremely verbose, for our purposes. The get_organization_rec has about 130 parameters.

The oracle.apps.ar.hz.v2api.HzPartyV2Pub Package

This is a Java package. We have created a jar called hz.jar. Oracle distributes the Java classes as classes in an exploded directory structure under the $OA_JAVA directory of an Oracle Financials installation. We use the smike dev environment for obtaining the classes.

To recreate the jar:

1. login to a server with Oracle 11i installed on it
2. cd to $OA_JAVA
3. jar -cvf hz.jar oracle/apps/ar/hz

Inspecting the classes shows that the use the HZ_PARTY_V2PUB_JW. They provide a convenient wrapper.

Summary

The steps to figuring out how to use a PL/SQL package in Oracle 11i are:

1. Find the PL/SQL package and procedure using Oracle Enterprise Manager or similar
2. Check the parameters to see if any are unsupported by the Oracle JDBC driver e.g. PL/SQL TABLE, BOOLEAN, and RECORD Types. If there are none go ahead and use it.
3. If there are unsupported parameters Oracle should have a PL/SQL package of the same name with a "_JW" appended to the end. This is the Java Wrapper package. It will contain the same PL/SQL functions and procedures but with exploded parameters supported by JDBC.
4. You can use the PL/SQL directly or look for a convenience Java API. To do so do a recursive grep under $OA_JAVA. When you find a hit, create a jar of that package and add it your classpath. You should find a Java class which calls the PL/SQL.

Posted by gluck at 02:35 AM | Comments (1)

May 03, 2005

The Horror of Oracle's OCI driver

It has been about two years since I last used Oracle's OCI driver. I remember it was painful to get going. I wish I had maintained a blog then, because I would have written the steps down and avoided a few hours of pain today. Which is why I am going to document what needs to be done. Hopefully some other poor soul can avoid the experience.

Step by step

1. Download the instance client and unzip it into a directory. Let's assume that directory is /u01/instantclient.
2. Copy the exact ojdbc14.jar in /u01/instantclient to your project's lib directory. The exact jar, and not a similar one needs to be in your classpath when you execute your java code.
3. In your profile, say .bash_profile, add

   export LD_LIBRARY_PATH=/u01/instantclient

   . This lets Oracle's shared objects know where to find its other shared objects.
4. Add

   -Djava.library.path=/u01/instantclient

   to your Java launch command. This tells java to add the path to its native library search path.

Gotchas

Oracle is silent on two of the three requirements. They mention the LD_LIBRARY_PATH but not the other two.

The error messages when something goes wrong are not very informative.

Posted by gluck at 09:08 AM | Comments (1)

May 02, 2005

HOWTO: Java 5 on Mac OS X 10.4 Tiger

Mac OS X 10.4 aka Tiger does not come with Java 5. However Java 5 was released as a download the day OS X 10.4 was released: 29 April 2005. This post is about getting and using Java 5 on Mac OS X 10.4.

Getting It

You can download Java 5 for Mac OS X 10.4 here.

Configuration

After installation Java 1.4.2 remains the default. You can change the default for Applications and Applets to Java 5 through the new utility:

/Applications/Utilities/Java/J2SE 5.0/Java Preferences

With these changes command line invocation remains at Java 1.4.2. I changed this by making some changes to my shell profile. I use bash, so I changed my .bash_profile in my home directory as follows:

export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home
export ANT_HOME=~/apache-ant-1.6.2
export PATH=PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$PATH

To get the new profile to take immediate effect, I typed

. .bash_profile

.

IntelliJ Idea on Java 5

IntelliJ Idea 4.5 requires Java 1.4.2. Changing the application default to Java 5 (using the new Java 5 preferences utility discussed above) breaks Idea.

Fortunately, the latest Idea 5.0 beta now supports Java 1.4.2 and Java 5. Get it from http://intellij.net/eap (free registration required).

Performance

To test performance, I used JPam, an open source Java-PAM bridge I have been working on lately. The default Ant target uses javac, javah, jar, gcc, javadoc and runs many jUnit tests, some of which are concurrency tests using 50 threads.
I ran three tests using each JDK and got similar minimum, maximum and mean run times. In this test at least, there was no significant differences to report.

More Information

Here is a list of useful Apple resources for Java 5:
1. Mailing List Archive - java-dev
2. Java Development Guide
3. Java Release Notes

Conclusion

I am happy to have Java 5 on Mac OS X. It seems to be solid and ready for use. Now that it is here it will simplify the lives of tool makers who were holding of supporting Java 5.

Posted by gluck at 02:53 AM | Comments (0)

March 10, 2005

Java Webstart with Eclipse Rich Client Platform and Linux

With Linux

I run Fedora Core 3. Something in Fedora Core 3 broke the Java Webstart which ships with JDK1.4.2 and JDK1.5.0. It is not fixed in JDK1.5.0 update 1, but is scheduled to be fixed in update 2.

javaws uses all available cpu and does not exit.

Workaround

A Sun engineer has kindly provided a special release which works. It is attached to the Sun bug. See http://bugs.sun.com/bugdatabase/view_bug.do;:WuuT?bug_id=6195591. I reproduce the excerpt for convenience below. Get this version of javaws http://www.jbg.f2s.com/javaws-1_2_0_06-linux-i586-i.zip
1. unzip in a suitable location (not the current java directory)
2. execute install.sh to install this version of javaws.
3. Now open a web start link (eg one from here http://java.sun.com/products/javawebstart/demos.html but specify the newly installed javaws as the program to execute rather than the old (not working one) Note, if you have already set the default web start app in FireFox then just go to Edit->Preferences->Downloads and edit the entry for JNLP file types.

With Eclipse Rich Client Platform (RCP)

We, after many days of effort could not get RCP and JNLP to work. There is a open source project out there called webrcp which seeks to get the two working. See http://webrcp.sf.net. It worked perfectly on Linux. It turned out to have a showstopper bug which stopped it working on Linux. We have submitted a bug and patch to the project for those that come after us. With the patch installed, it worked.

Posted by gluck at 11:14 AM | Comments (0)

March 04, 2005

First Thoughts on Eclipse Platform

This week I have been part of a team working on an Eclipse Platform application to be launched from JNLP, or Java WebStart, as it is better known. We are using Eclipse 3.0.

Problems

My complaints are as follows:

build.xml is being deleted

Whenever I build my project from the Eclipse IDE it deletes my build.xml. This is happening to each developer. Not sure where this coming from or how to stop it.

Null Pointer Exception

We are developing an eclipse plugin. You export it to a jar. It optionally creates an Ant script, so you can do the same thing in your Ant build. Unfortunately is does not create a taskdef. There is no documentation that my colleague could find in two hours of searching on how to do this. We independently got it half working, both to be stopped by an informative null pointer exception! We ended up looking at what was created and doing it from Ant with Ant built-in commands.

Mac Support?

The Mac support is second class at best. Including the carbon lib in your library path at application start up causes an exception. Including the windows libs on Linux or the Linux libs on Windows does not. On the Mac you cannot start an eclipse app with Java. You need java_swt, which comes with eclipse.

Raw

I understand that 3.0 is the first version of Eclipse designed to easily run plugins outside of the IDE. It shows.

No Documentation

There is very scant documentation of an extremely poor quality. This feels like a backyard doco effort.

The Promise

I had heard lots of good things about Eclipse Platform. Perhaps what we are going through settles down to assumed knowledge after a time. I would welcome any commenters views. We are presently seriously wondering how widely it can be used given the problems we are seeing.

Posted by gluck at 12:57 PM | Comments (1)

January 30, 2005

Performance Benchmark Results: JVM vs CLR

I was talking with someone the other day about Java vs .NET performance. I have never thought that much about it. He said that .NET was fast because it had a Just In Time compiler ("JIT"). I choked on my coffee. Java's JVM has had a JIT for about 6 years now. Not only that, but I had never heard that .net was faster.

So, I decided to go and run some benchmarks for myself. Rather than dirty my machine by installing Windows, I fired up VMWare and installed it on a disposable file. I then started the arduous and frustrating experience of installing .net onto my VMWare pack of XP. Reading the EULA, I discovered that Microsoft had already foreseen the threat of benchmarks and banned disclosure of any benchmark information without written permission from them.

So, If I were to tell you that Java JVM based on JDK1.5 was 42% faster than .net's CLR, based on a benchmark I had, that would be breaching the EULA. In reality I cannot even get started because of the mind boggling activation process required. Currently my 60 day trial activation key has been used too many times and my Visual Studio is disabled. I am enjoying the experience already!

There are some braver souls than me around. See http://www.shudo.net/jit/perf/. There benchmarks are current. They were last updated in December. They show that on SciMark, Java scored 324 vs .NET of 240. On Linpack, Java scored 413 vs .net's 282. Adding the scores together gives 737 for Java and 522 for .net.

On their site there is a lot of detail and a lot of different versions. Check it out.

So, in complete agreement with the EULA, I can tell you that Java's VM is 42% faster than .net's CLR.

And the next time someone claims that .net is faster because it "has a JIT compiler" point them at the site.

Posted by gluck at 03:08 AM | Comments (8)

December 04, 2004

SimonSays: A stretch reminder program to cure those stiff necks

I finished the first Java version of a program recently which reminds programmers and other continuous computer users to stop and stretch. Versions are available for Mac OS X, Linux and Windows. It is the Java reincarnation of a commercial program I wrote years ago and sold quite a few copies of.

It took me three years of part-time effort to complete the new Java version. It was started on JDK 1.1 and finished on JDK 1.4.2. Over that time Java has matured enormously, and made it a lot easier to create a smooth desktop application. Simon has been developed primarily on Mac OS X and Linux. Mac OS X has also matured as a Java platform and I finished the application on IntelliJ.

If you supervise people there is a problem getting them to stretch regularly. On our project we now have four people who have their own chairs and regularly see physiotherapists. Yet a better approach is to develop healthy habits. These include posture, particularly seating position, and regular stretching.

Simon Says pops up at predetermined time intervals. Simon, our cartoon character then guides you through an animated series of stretching exercises . It logs your exercises, so when you get that stiff neck you can go back and check whether you have been streching, or not. When the exercise program is completed it sleeps for a set time after which it will pop up again with another program to run.

There are more than 100 exercises which exercise most parts of the body organised into programs. For example, stretches can be selected for cervical spine,