Kerberos is cool

The last few weeks I have been working on a single sign on/Kerberos project.
For anyone for whom either of those term is new, here is some food for thought. Windoze, Linux, Mac OS X (10.4), Firefox, IE, Apache, ssh… has, in the past 10 years been Kerberised. Rather than Microsoft’s Embrace Extend Annihilate being the death knell for Kerberos, their endorsement legitimised it.
Add in SPNEGO, another M$ innocation, which added browsers into single sign on, and here we are. Much of everything we need is Kerberised. An overnight success which took a decade. It is a very good time to adopt Kerberos.
Over the next little while I intend creating a Kerberos/Glassfish HOWTO, showing how to add a Glassfish security realm to a Kerberos realm. If you look in your JAVA_HOME/bin directory, you will see kinit and klist, Yes Java was Kerberised in 1.4.2. There is a Kerberos Login Config: com.sun.security.auth.module.Krb5LoginModule

I will be presenting at JavaOne 2007 on ehcache

For anyone attending JavaOne 2007 at the Moscone Centre in San Francisco I will be presenting a session entitled “Distributed Caching, Using the JCACHE API and ehcache, Including a Case Study”.
Abstract follows:
Title: Distributed Caching, Using the JCACHE API and ehcache, Including a Case Study on Wotif.com
Abstract: Java EE web applications are typically used in e-commerce systems. They generally involve multiple application servers communicating with a database. Caching can play an important role in enabling scaling out. Caching can be applied to servlet responses, collection caching, and Java Persistence API caching. JSR 107 (JCACHE) is an effort to standardize the cache API. It benefits users by providing a lowest common denominator for caches and by reducing the cost of change of caching implementations. This presentation looks at the features of JCACHE and the ehcache implementation of JCACHE (ehcache-1.3) in particular. It also includes a case study of wotif.com, the third-largest e-commerce site in Australia by revenue, which does a large amount of caching at each of the levels mentioned above. The case study shows how wotif.com achieved horizontal scaling by using the distributed caching features available in ehcache–all running on the GlassFish project application server. It demonstrates how to replicate servlet responses across a cluster, so that regardless of the view generator–whether it be JavaServer Pages (JSP) software, Velocity, or XML responses to Ajax calls–the servers can share a cache of responses. Next the presentation looks at how to cache computed results represented as collections. It examines pull-through-cache patterns and gives examples. Last, it discusses caching with the Java Persistence API. Wotif.com uses Hibernate, which enables pluggable second-level caching, and the presentation shows how the site uses Hibernate with a distributed cache to minimize database access. The session closes with a discussion of the timeline for release of the JCACHE specification and how to get started.

Opening Up Glassfish to remote JMX monitoring

We are doing monitoring of our apps using ManageEngine Applications Manager. It supports JMX remoting. The latest patch release supports composite types. Very nice.
Right now Glassfish is the new kid on the block. Support for it is not yet baked in to Applications Manager (but will be, according to their product manager). In the meantime you can use generic JMX monitoring to monitor it.
The glassfish domain defaults do not work for remote monitoring. After a bit of playing around I got it to work.
You need to add the following to your domain.xml configuration in Glassfish:
Add these to the java-config element:
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
Add this to the admin-service element.
<jmx-connector accept-all=”true” address=”0.0.0.0″ auth-realm-name=”admin-realm” enabled=”true” name=”system” port=”8687″ protocol=”rmi_jrmp” security-enabled=”false”/>
Not sure exactly which change fixed it. But this config works.
In Applications Manager you do the following(example only):
Host Name / IP Address: developer249
Port: 8687
Polling Interval: 1
JNDI Name: /jmxrmi
Authentication is enabled: X
User Name: admin
Password: password
The same thing works for the jconsole and should work for NMS’s that support JMX. Enjoy.

Wotif.com live on Glassfish

Wotif.com, my employer, went live on Glassfish in mid January. I was the driving force behind the move. You can read about it on the Glassfish site here: http://blogs.sun.com/theaquarium/entry/how_to_run_a_top .
There is a questionnaire linked from the entry that is the raw question and answer source for the entry. You can see it here: http://blogs.sun.com/stories/resource/wotif/wotif_responses.html
Though not one of the main reasons for the change, we find ourselves on the JEE5 platform, and liking it. XDoclet boiler plate generation and EJB in-container testing are things of the past. It took a few days to move our code from EJB2 to EJB3. The end result is that we find ourselves early adopters of JEE5. The opportunitites this presents and some neat ways of capitalising on them are being documented on Robert Watkin’s blog. See http://twasink.net/blog/ .

Dependency Management Choices: Maven, Ant + Maven Antlib, Ivy

We have a monolithic code base. It is something you end up with if you keep adding classes, without thought to larger modules. So how to solve the problem. Break the code up into modules. Also figure out how to combine these into applications. Then look at whether to run these together in the one JVM or separately.
One problem that comes up is tool support. Ant on its own does provide ready made solutions to this.
We needed a higher level build system on top of Ant that provides a default means of dependency management, multi-project relationships, and generation of build artifacts such as jars, wars, and ears.
A look around shows a number of approaches: Ant + Ivy, Maven, Ant + Antlib for Maven and Maven + antrun plugin.

Maven

Maven is a popular and widely used build system which handles dependency management and multi-project relationships. It operates at a higher level than ant, sort of like what Hibernate is to JDBC. Maven scripts tend to be much shorter than Ant scripts to get an amount of work done.
Maven provides:

  • a dependency management system. no more ?lib/*.jar? dependencies
  • a standard for an artifact repository that handles packages and version numbers, javadocs and source code (of releases), either centralised or proxied.
  • a ?standard? project lifecycle where you can ?plug? special processing. All projects go through: generate source, compile, package, install etc?
  • Simple project setup that follows best practices – get a new project or module started in seconds
  • Consistent usage across all projects means no ramp up time for new developers coming onto a project
  • Superior dependency management including automatic updating, dependency closures (also known as transitive dependencies)
  • Able to easily work with multiple projects at the same time
  • A large and growing repository of libraries and metadata to use out of the box, and arrangements in place with the largest Open Source projects for real-time availability of their latest releases
  • Extensible, with the ability to easily write plugins in Java or scripting languages
  • Instant access to new features with little or no extra configuration
  • Ant tasks for dependency management and deployment outside of Maven
  • Model based builds: Maven is able to build any number of projects into predefined output types such as a JAR, WAR, or distribution based on metadata about the project, without the need to do any scripting in most cases.
  • Coherent site of project information: Using the same metadata as for the build process, Maven is able to generate a web site or PDF including any documentation you care to add, and adds to that standard reports about the state of development of the project. Examples of this information can be seen at the bottom of the left-hand navigation of this site under the “Project Information” and “Project Reports” submenus.
  • Release management and distribution publication: Without much additional configuration, Maven will integrate with your source control system such as CVS and manage the release of a project based on a certain tag. It can also publish this to a distribution location for use by other projects. Maven is able to publish individual outputs such as a JAR, an archive including other dependencies and documentation, or as a source distribution.
  • Dependency management: Maven encourages the use of a central repository of JARs and other dependencies. Maven comes with a mechanism that your project’s clients can use to download any JARs required for building your project from a central JAR repository much like Perl’s CPAN. This allows users of Maven to reuse JARs across projects and encourages communication between projects to ensure that backward compatibility issues are dealt with. We are collaborating with the folks at Ibiblio who have graciously allowed the central repository to live on their servers.

Potential Issues with Maven

The historic issues with Maven and the current status are:

  • Maven was poorly documented. This has been improved somewhat, particularly with a freely downloadable PDF book on Maven2.
  • Is their a critical mass out there? There now is. It is becoming very common to see pom.xml’s in projects. pom.xml comes up on Google in a 1:4 ratio with Ant. So it has critical mass and is not going away. Ibiblio has a very large number of projects in its repository.
  • The changeover from Maven 1 to Maven 2 was untidy and confusing. Now everyone is one maven 2
  • Maven has an initial learning curve. You need to dive into black magic to develop a maven plugin when you need to do custom things and this is hard. You can always call into an ant script. Developing a plugin is optional.
  • The central repository may not be available. The solution is to use a local repository. See http://cocoon.zones.apache.org/daisy/documentation/g1/1164.html

Ant + Antlib for Maven

Maven 2.0 now comes with a set of Ant tasks that can be used to utilise Maven’s artifact handling features from within Ant. Included are:

  • Dependency management – including transitive dependencies, scope recognition and SNAPSHOT handling
  • Artifact deployment – file and SSH based deployment to a Maven repository
  • POM processing – for reading a Maven 2.0 pom.xml file

See http://maven.apache.org/ant-tasks.html

Remaining Problems with Ant

Ant files tend to become huge. Can be solved, as of ant1.6, by the ant import nd macrodef tasks.
Each project ends up with duplicated Ant code. Can be solved, as of ant1.6, by the ant import nd macrodef tasks, although you then need a common artifact
No standardised build structure. This can be enforced manually.

Ant + Ivy

Ivy works very much like Ant + Maven Antlib. It has arguably a better dependency management process. It does not have a central repository of projects. You have to package third party stuff up and put it in your Ivy repository. It does not provide the extra features of Maven beyond dependency management.

Making a Choice

Relative Popularity

In terms of what projects are using, one way to tell is to search google for the config files that are used by each tool.
A google search for build.xml returns 3,240,000 results.
A google search for pom.xml returns 786,000 results.
A google search for ivy.xml returns 17,900 results.
Both Ant and Maven are mainstream. Ivy is a niche player by this measure.

Thought Leaders

Matt Raible was an eager Ivy user, then moved to Ant + Maven Antlib then moved to Maven as of September 06.
The Spring project is moving to Maven.
Much of CodeHaus and Apache have/are moving to Maven.
I know people who use Maven and those that use Ivy. Both Maven and Ivy are part of Apache.

Bugs

The Maven project is extremely poor at fixing bugs and getting releases out. Ivy has a better reputation.

Summing Up

Online Site Trends

Luke Welling of Hitwise showed up some trends using the Hitwise database. It is mostly site visits. Interesting to compare this with Google Trends and Tiobe.
Mailing lists seem to be getting replaced by feeds.
Ubuntu matching Debian in May 06. Redhat is still getting more visits than all of the others combined.
PostGres is gaining on MySQL and Oracle is in slow decline.
PHP continues its dominance over Perl and other dynamic languages.
Digg.com eclipsed slashdot.org in 2005.
Wikipedia is huge.
DEL.ICIO.US went crazy last year.
feedster.com dropped off after site difficulties and is now coming up.

Kiwi Foo – First two days

Distributed Stuff

One was on distributed caching and file systems. We talked about memcached, ehcache and a new file system from Live Journal – Mogile FS. It is user space, so not really a true file system. It comes with a mod_mogile for Apache. Must take a look at that. It looks like use of memcached is settling on page caching. Serialization issues make it a bit expensive for object caching.

Emerging Internet Security Standards

This was a wide ranging discussion on whether any standards are emerging in Internet security after 10 years of nothing more than server SSL certs being taken up.
In Instant Messaging it looks like OTR, which stands for Off the Record, is emerging. Adium for Mac, which had a version 1 release today, has it built right in. Many others have plugins. You click a lock and it asks the other party to go off the record. If they have the software they can elect to. If not, the handshake does not complete and it stays in plain text.
OpenID is the other big one. It is a single sign on system for the web. Technorati and Wikipedia are moving to it. Firefox 3 will support it (Ben Goodger confirmed it is on the wish list). Firefox 3 is also down to support SAML, the Oasis standard. Sun’s Liberty Alliance specifies SAML. So hopefully something is going to happen here.
As to email, it doesn’t look like anything is going to happen. We speculated that it will take someone like Google Mail to introduce a standard. Maybe S/MIME, maybe something else. Interestingly lots of ISPs are supporting using SSL for MTA. I tested mine and it does not but apparently lots of Australian ones do. One tick for confidentiality.

Ruby Roundup

There wasn’t much on Ruby in the sessions. But I got around the rubyists. I met the guy who decided not to accept a patch for Oracle prepared statements. “Our user community does not really use Oracle and prepared statements can be harmful to PostGres performance and are neutral for MySQL”. One guy is an ex-Rubyist. His complaint was the community. Naive implementations solving problems that were elegantly solved 10 years ago, with the implementers thinking how cool they are. In the next breath I heard how banks and big enterprises are no longer interested in Java and are all doing Ruby. I look forward to them changing over to MySQL because the Ruby community does not care about Oracle.
There seems to be high respect for the JRuby project. It looks like it will imminently be able to run Rails. There are a few people showing how to deploy it on Glassfish. The Netbeans guys are making much of adding not just Ruby but Perl, Python and PHP language support. And IntelliJ IDEA released just a few days ago their Ruby plugin through their plugin manager. I played with it again (the last time was a few months back). It is ok. It has RDoc with CTRL-Q, syntax highlighting and minimal keyword auto completion. It needs to be much better to get me enthused. Allan Odgaard, the TextMate author, pointed out that Ruby autocompletion has just been added as a plugin to TextMate. (His focus is on supporting 100 languages, so he himself does not do too much that is language specific – he likes the Emacs analogy).
Tiobe announced that Ruby was the language of 2006, because it had the most growth. The latest Tiobe shows a leveling off of Ruby. Not sure if this is temporary or whether it will level off below Python.
Overall the super buzz around Ruby seems to be dying down.

Perl

Allison, a Parrot VM architect was here. I asked her about whether an open source JVM might get used by Perl and others. She thinks maybe, but the independent efforts, which are designed for dynamic languages will continue.

Segway

We all had a go on a Segway that the LiveJournal guy brought with him. It has a black key and a red key. Red is for fast. We all graduated from black to red and a had a great time. You delegate balancing to the machine. The rest is easy.

Google

The google guys were conspicuous by their t-shirts and tight-lippedness. Google’s share price has taken a bit of a tumble, which was the main discussion. Chris De Bono, helpfully “does not comment on future share prices”. This is very similar to his comment from OSCON two years ago in answer to questions about better Linux support with he “does not comment on future product announcements”. With conversation that exciting I moved on. Interestingly, given that there now a 1/googol chance of any new hires making anything out of Google employee optios, all the Googlers are emitting a new company line that Google has found new creative ways to entice developers to join. I am up to three recruitment approaches from Google now in the past two years and am probably now on the do not call list.
I did manage to meet Lars Rasmussen who was on the team that did Google Maps. Due to its Cuba policy the USA has lost Lars and his Cuban wife to a country (Australia) that has normalised relations with Cuba. A good win for Australia.

Java

Lots of Java people haunting the corridors. Thankfully they were all IntelliJ users, so we could escape the oppression of the Eclipse nazis for a short time. I met the guy who did the TestNG plugin. JUnit 4 is at the end of the day not that exciting. I might take a look at TestNG. I spread the word about Glassfish which I am now using at work in production. I think it will be big. It seems clear that Glassfish is the platform to be on for new experiments. Some of the non Java guys, who do not exactly keep up with Sun news, were interested to hear that Sun had done an open source Java app server. And the open sourcing of Java is big. Glassfish is coming on to the Linux distros (Ubuntu already announced) so they will see it.

Linux

It was agreed that Linux newbies prefer Ubuntu these days. A grizzled Debian veteran still prefers the original formula. I showed off my Macbook running FC6 on VMWare Fusion in answer to questions about whether I had used Parallels.

Politics

Lots of it at the conference. We even had the NZ Communications Minister with us. He seemed very well informed on IP issues. NZ is doing a new Copyright Act and trying to avoid the disgraceful Australian precedent done as part of the Australia – USA bilateral trade deal where we let the US export their IP laws to us in exchange for a little bit of market access for our farm products.
He suggested that patent infringement lacked mens rea. I suggested that it is now hard to write a line of code without infringing some patent. So we all have mens rea, but we have no idea what patent we are infringing.
Corporates still seem best served by building up patent war chests. A bit hard for those of who do open source.

Trip to New Zealand

I am in New Zealand for the Kiwi Foo Conference and to do the Tongerara track.

The Flight OverAuckland

New Zealand has become very successful at attracting foreign students, mainly chinese speakers. My hotel in Auckland actually turned out to be a foreign student accommodation tower. The front desk did not speak English well at all. My check-in was finally accomplished by handing over my voucher. A China town has formed between Queens St and the freeway to the east – about 6 blocks. It sort of felt like being in Hong Kong, with lots of packed towers and high energy.
I gained some temporary respite by catching a live gig at an Irish pub a bit further down Queen St.

Warkworth

Driving north out of Auckland it is striking, to an Australian, how effortlessly green everything is. The area strongly reminds me of the Tillamook region of Oregon. Rolling green hills and bucholic bliss. Warkworth is a lovely town about an hour north. A freeway from Auckland is steadily making its way north. One day Warkworth may end up being a dormitory suburb for Auckland, but not yet. Warkworth is the home of Nat Torkington, OSCON and Kiwi Foo organiser.

The New Worry

New Zealanders lose people each year to Australia. A lot of them are men. New Zealand now has a sex imbalance and is becoming a bit matriarchal. At the conference the Mahurangi College has taken over the male staff toilet and turned it into a female one. So there are two female toilets side by side. The make toilet is outside across the courtyard in a converted store room.
The new worry though, is that with Australia rapidly drying out due to global warming, Australians might start migrating en mass to New Zealand. New Zealand seems blissfully unaffected by Global warming.

Internet

The Internet is expensive in NZ. Hotels do not offer it free. Hot spots are rare. I will never complain about the Australian Internet again. Fortunately for us we had our own Internet company sponsor. One satellite on the roof later and the conference was humming. Some of the conference topics reflected the frustration. One session was on how to use cooking equipment to create parabolic antennas to extend WIFI range. Another was on a commercial effort to create controlled hot spots using your own personal Internet service and WIFI card.
Apparently Google Maps does not do maps for New Zealand – yet. And google itself has been known to become inaccessible from New Zealand for lengthy periods.