ehcache goes distributed

For the past couple of years I have been saying that ehcache will never be distributed. Ehcache grew out of uncommitted patches to the Apache JCS cache. JCS implemented the JCACHE JSR. It had lots and lots of features. The reason for the patches was that the in-memory and disk stores had memory leaks and threading problems.

The idea of ehcache was to bring it back to something very simple and then get all the kinks out. Ehcache has been successful in achieving that aim.

Over the last year there have been feature requests and offers of help to implement those features. Surya Suravarapu in particular volunteered for one day a week. He was very keen to make ehcache distributed. We decided to start our collaboration with something more prosaic: some new cache policies. He added FIFO and Least Frequently Used. In its latest incarnation it has become Less Frequently Used, for performance reasons, which is an interesting story in itself.

Steve from jofti.com then requested event listener functionality. He agreed to give input into the design and review its implementation. After a few iterations we got that done to his satisfaction and it is now being used from his Jofti cache indexer and search library.

Paul Hammant and Jo Walnes took a look at ehcache during OSCON2005. They argued that the singleton CacheManager was evil and should be replaced by constructor-based instances, for which it is the callers responsibility to maintain references. I agreed with that and so added the functionality, with a few safety provisions to prevent disk store collisions. The feature turns out to be particularly useful in testing distributed caches on a single machine.

At the conference I ran into James Strachan. James is one of the JCACHE people and the prime mover behind many projects, including the notable ActiveMQ. We discussed distribution and how it should be done. He agreed with me that multicast was fine for discovery but not for delivery. Then I ran into the guys from Sourcelabs, who are keen ehcache enthusiasts and may offer support for it. They were saying I really needed to make it distributed. There voice is of course added to the many requests for the same feature.

These ideas bounced around for a while. I was also doing some architecture work for a distributed system at the time. I had also been doing lots of reading on distributed system. In August I decided to commit the time to make it happen.

I was keen on doing it test first and keeping it as simple as possible. I decided to start with the primitives in the JDK and see where that took me.

One thing that quickly emerged was the disparate views on what makes for the best distribution mechanism. Surya was keen on a JGroups approach. Another group at BNP were interested in a JMS based approach. To accommodate these needs I decided to make the distribution framework pluggable. I am interested in using ehcache as a test bed to explore different distribution ideas. Of course, it takes at least two implementations to make sure you have your plugin framework right. I have two for discovery and one for delivery right now. Hopefully another delivery implementation will get done that can prove or force modification to the framework.

A lot of hard work and 5 months later and ehcache 1.2 beta 3 is out with distributed caching built-in. Thanks to everyone who lent a hand.

I now want to get it in the hands of as many people as possible and get the bugs out. I need to do further documentation. I need to see how distribution can best be utilised for applications like Hibernate and web pages. Ehcache-1.1 is now 13 months old. An update will be timely.

The funny thing is, ehcache now had a similar feature set to the original JCS. Perhaps it would have saved me a lot of time and effort if the JCS developers could have just accepted the patches I put in two and a half years ago.

And the Programming Language of 2006 is …

Java. Yes, Java.

http://www.tiobe.com/tiobe_index/index.htm is the monthly programming language index. It tracks the popularity of programming languages. Java won the 2006 Programming Language award because it was the language that increased in popularity the most during 2005. By 4.77%.

JDK 1.6

JDK 1.6 proved to be quite popular and caused an upswing.

Anecdotally the big news for me in 2005 was the movement of code written in C++ to Java. I spent time on a large project last year which was moving 12.5 millions line of C++ to Java. It was a performance critical desktop application. The performance gains of Java in recent releases were enough. The addition of generics was very attractive to the C++ guys. They thought the time to move to Java was “just right”.

Cold Fusion lives again

One interesting phemomenon is the rising popularity of Cold Fusion in Tiobe’s index. Cold Fusion is Java based an deployable as a WAR or EAR. Its big feature is productivity. I think of it as a Java based Domain Specific Language. Anecdotally Cold Fusion seems to be making headway again for web apps.

The Decline? of Dynamic Typing

The other Tiobe identified trend is the relative decline of dynamically typed languages.

Where does this trend leave the dynamically typed Ruby? From the O’Reilly conference and amongst my peers, there is quite a lot of interest in Ruby. However its popularity in the Tiobe index is only 22, though it is increasing in popularity at the expense of Perl. So, perhaps the book series should be called “Beyond Perl”, rather than “Beyond Java”.

Dynamic vs Static Typing is a subject of programming language textbooks. Which is why I am reading one. Some good stuff on this topic is available here. Robert W. Sebesta, in his Concepts of Programming Languages has this to say:

  • Dynamic type binding causes programs to be less reliable, because the error detection capability of the compiler is diminished…
  • … languages with dynamic type binding… must be implemented with interpreters rather than compilers.
  • Perhaps the greatest disadvantage of dynamic type binding is cost. The cost of implementing dynamic type binding is considerable.

And this on page 223:

“The penalty for static checking is reduced programmer flexibility. Fewer shortcuts and tricks are possible. Such techniques, though, are now generally held in low esteem”.

In his view, Java and C# are on the ascendancy partly because they make the right set of tradeoffs in language features. They combine the right mix of readability, writability and reliability. I have been tinkering with Ruby since the O’Reilly conference last year, when I attended a half day session with Dave Thomas. I am hoping to get the time to do a full comparison of Java and Ruby in those terms.

Relative Browser Market Share Trends

It is hard to work out exactly which browser market share statistic to pay attention to. The numbers vary depending on whether people mainly access from work and by region around the world, as well as the type of site. Be that as it may I think the trend line is interesting and perhaps more relevant.

Below is a chart for Firefox/Mozilla and miscellaneous browsers compared with IE for the past three and a half years as measured by the famous w3schools.com browser market share survey.

The absolute percentage for Firefox/Mozilla et al for January 2006 is 33.60%. This is one of the highest percentages reported. w3schools is generally a leading indicator, because the web cognoscenti and web designers and authors frequent it. More interesting than that is the trend.

Having been involved with many project decisions about which browsers to support, I always felt that 10% non IE was necessary to ensure sites were built to standards and interoperability. That 10% was in danger in 2001/2002. The only thing that saved one high profile site I worked on was the concern about journalist reviews, who were a strong Mac community. Mozilla had not been doing much to capture new users, though I used it right through from version 0.90. Firefox came along just in time really.

Where it will end? I think most users will always want to go with just what their OS gives them. I tend to. On Linux it is Firefox and on Mac OS X it is Safari, which I mainly use. I also have IE on CrossOver Office running on Linux and Firefox and Mozilla on Mac. According to w3schools, operating system visits for Mac/Linux went from 4% to 6.8% over the past three years. This represents a hard base of non IE users. Add to that what can be captured from Windows users. Much depends on whether manufacturers bundle it. This is starting to happen. In October HP announced that users would be able to choose Netscape at install time on and in December Dell announed bundling of Firefox.

No matter where it ends there is now enough non IE marketshare to assure web standards and to avoid Microsoft domination of standards. RSS is an example of something that came out of the community and is now a standard browser feature. Another is tabbed browsing and yet another is pop-up blocking.

The real worry is the low non-Windows market share. There are sites like WebEx and Google that offer degraded or no services to non Windows users. (Try using Google Video as a Linux user for example.) Hopefully the new Macs will help build that share up.

The Sun and Moon: Thailand’s Suriyana and Chantra Open Source Windows Distributions

Pantip Plaza in Bangkok is the mecca for software pirates. It is 6 stories of booths, probably around 500 500 in total, selling up to the minute versions of software for 100 Baht (USD2.50) per CD or DVD. International travellers from all over the world flock there in droves.

I picked up one piece of software at Pantip when I was there in late December and it was not pirated. There was a crowd around one booth in particular near the doors on the ground floor. There was a yellow banner as backdrop, as shown below.

Yes, the buzz was about a Thai open source distribution for Windows: Chantra. The version I picked up was 1.0, which has now been followed up by version 1.1. Thai’s but not internationals were providing their email addresses and names and picking up a free CD and brochure.

Chantra, which is Thai for Moon was released in November 2005. It includes, inter alia, the following:

  1. OpenOffice 2.0 Thai version
  2. Dia
  3. FreeMind
  4. Firefox
  5. Thunderbird
  6. BitTorrent
  7. Gaim
  8. Gimp
  9. Inkscape
  10. VLC
  11. Audacity
  12. 7-Zip
  13. ClamWin
  14. Nvu
  15. FileZilla
  16. PuTTY
  17. Notepad2
  18. AppServ
  19. Subversion
  20. Tux Racer

In the 20 minutes I was there about 30 people walked away with CDs. Chantra is distributed by an organisation called SIPA, part of the Thai government. They operate an open source portal at thaiopensource.org, which was showing 411,637 hits after a month of operation.

SIPA are also planning to release the Suriyanâ server software distribution. Suriyanâ is Thai for Sun. So we have the Sun and the Moon. Nice.

It is very encouraging to see this type of thing going on. I suspect it is a reaction to pressure on Thailand to cut down on piracy. Hopefully, efforts like these will mean that piracy will be substituted for open source use, which will help to drive the open source movement.

Syncing your Mac with your Blackberry

I bought myself a BlackBerry 7100 a few weeks back. At the time someone said you could add it to iSync using PocketMac.

I got around to looking into this yesterday. Talk about great timing. At Mac World 2006 San Francisco RIM announced they had licensed PocketMac for all BlackBerry users.

While it was announced that the free version would be up on blackberry.com in February, you can download it now from PocketMac. They have changed the price to $0. Make sure you use the credit card option. It will skip the credit card screen and take you straight to downloads. If you use PayPal it causes an error. Obviously the store is not normally used for charging nothing.

PocketMac seems to work fine as an iSync conduit.

Now if only I could something working for Lotus Notes syncing to iCal…

OpenSuSE: I’m lovin’ it!

I have been running RedHat and then Fedora as my desktop for work since RedHat 5 in December 1997. Back then it was mostly about becoming familiar with Unix so that I could administer the various Unices I had inherited as part of a new job. Having gotten used to the power of the shell and the many developer tools built in, as I transitioned from IT Management to development, it was natural to stay on the platform.

In the 8 years I have been running Linux I have never had a virus. An unpatched version of e-smith which I was using as a home gateway was once hacked but not damaged. That wakeup call made me pay attention to regular patching even in my home systems. Along with the lack of security hassles, it has been a very stable platform. Consequently losing work or productivity is pretty much a dim memory from my days on Windows.

Over the 8 years desktop Linux has steadily improved. Gnome and KDE came along. Sun decided to support Linux for Java and almost all enterprise software companies support it too. Life on Linux became easier and easier. Indeed things have become so good that in the last few years whole development teams have been running Linux.

When RedHat decided most of its money was to be made on the server they decided to create the Fedora project. The initial release was of the same quality as RedHat 9. Over time that has slipped. Fedora Core 4 is pretty good, but there is stuff that used to work better that now has problems. One example is the networking configuration. Originally I did this with text files. Then graphical config tools came along. In Fedora Core 4 it is necessary to delete a network configuration and then create a new one to make a change. For me it is a simple workaround, but a painful one to first discover.

What it amounts to I think, is the QA difference between a beta quality product and a production ready one. One of the reasons I enjoy Linux is the stable and productive base for development it gives me. Niggling hassles which have come in to the distro destroy part of that utility.

Before anyone suggests Mac OS X, know that I have been using it as my home digitial hub for three years now, and just this week upgraded my PowerMac to a dual core G5. (Yes I know about the move to Intel but the performance is comparable, the PowerMac is here now, and my old one got fried in a storm). Mac OS X is good to develop on, and I do so at home. However for a Java developer it is a second tier platform. Apple do their own JDK which typically lags 6 months to a year behind. I also like to develop on the platform that will be used for production. Mac OS X Server is not really in the mainstream and so far no clients of mine have been interested in running it.

I could also move back to Windows. However every time I use it, I feel like I have been neutered. No decent shell, no workspaces, a horrible registry and so on. (Sources at Microsoft tell me that there have been 3 so far unsuccessful attempts to rid the world of the registry). Philosophically, I also have a problem with monopolies, part of a long family tradition going back to our time as Huguenots.

So, I was pretty sure I wanted to stay on Linux, was fed up with RedHat’s Fedora policy. The number two Linux system for enterprise has always been SuSE, so I thought that was worth a look. Also, I did a stint at Novell last year and witnessed a company wide migration that was going very smoothly. Seeing my Novell Linux Desktop neighbours working with it, from programmers to project managers, and no one complaining, I thought it was worth a look. Still, I could not fathom Novell’s policy toward its free edition. SuSE 9 was the last version on the mirrors that I usually download from. References to it were buried deep in the labyrinthine Novell website. So I remained probably the only Fedora user on the Novell network. I did however complain long and hard to anyone that would listen. My main argument was that developers select the OS which when the software is done tends then to result in purchase of a commercial version for production. That is how things have tended to work. So, even though you do not make any direct money from a free version of your distro, it leads directly to revenue 6 months to a year later. My comments found a lot of support within Novell and trickled through to the product manager.

Anyway for whatever reasons Novell decided to release OpenSuSE and make a full effort of it. My main reason for not trying it went away. It has taken me until laptop changeover time to take a closer look at it. I have been using it for a few weeks now and am mightily impressed. It is up to date, slick and has the feel of a well tested and polished product. While a little foreign to me, the idea of only releasing security patches to a version, not updates, strikes me as a reasonable compromise. You get to enjoy production quality while trading the very latest. Given the maturity of Linux now, there is often little reason to want to go to a new version outside of major releases. That was not true in the past. If I do want to get the latest stuff I can always add a YaST source for it. The biggest problem for me is lack of familiarity which would be the issuse with any change, and the fact that Novell’s year away from promoting a free version has left a bit of a gap out there. You tend to find lots of packages for SuSE 9 but not for OpenSuSE 10. Having said that. everything is set up, all packages I need I have and everything works great. Stuff like CrossOver Office made the transition seamlessly. I just created a bottle (CrossOver speak for a self-contained file system) on my old system, archived it and brought it across.

It seems that other Linux users have been having similar thoughts. distrowatch.com reports on Linux distributions. OpenSuSE is number three on the top 10 list, ahead of fedora. It is beaten by Ubuntu and Mandriva, neither of which have enterprise server usage. Looking at the page hits metric over different time periods, it looks like OpenSuSE may move into second position. When looking at the last month it edges out Mandriva. Where have the new users come from? What is the distro in decline? Fedora. Lookout RedHat. If my theory is sound we will see the commercial versions of SuSE start to get more marketshare later this year. Of course RedHat could always improve the quality of Fedora. Lets hope they do so and the virtuous circle is completed.

The Weather Makers

The last book I read on climate change was The Skeptical Environmentalist, by Bjorn Lomborg. He is a statistician and argued that the global warming we witnessed up to 2000 could have alternate explanations than greenhouse gases, such as increased sunspot activity. He was lambasted at the time and investigated by the Danish Committee on Scientific Dishonesty. He was cleared of dishonesty in 2004.

The point I talk away from his book was that, at the time of writing, there was not sufficient data to conclusively prove that global warming was caused by greenhouse gases.

However, since that time there has been a lot more science done, spurred on I suspect by Lomborg’s and President Bush’s doubts. A great summary of where things stood mid 2005 is provided by The Weather Makers, by Tim Flannery. Tim is a conservationist from way back. He is famous for The Future Eaters. His new book promises to be the Silent Spring of climate change.

Where things stand does not look good. Carbon Dioxide has risen from 2 parts per 10000 to 3 parts per 10000. The role of Carbon Dioxide in retaining the Sun’s rays has been understood since the 1800s. The evidence now strongly suggests that the warming is due to increased Carbon Dioxide and to a lesser extent Methane.

To figure out exactly what will happen, scientists model the climate. This is complicated and frequently revised as new data comes in and there is new understanding of the interactions between the many things that make up the climate. Some things that a likely to happen are:

  • Rising of the sea level due to thermal expansion and melting of ice which is on land.
  • Destruction of the Great Barrier Reef, due to coral bleaching due to warmer water due to warmer air. 82% destruction for a 1 degree rise; total destruction for 3%. Total destruction is looking likely.
  • Mass extinction
  • A Semi-permanent El Nino affecting Eastern Australia.
  • Permanent strong hurricane seasons for the Western Atlantic.

Anecdotally, where I live in Brisbane we are in our worst drought ever and 2005 was the hottest ever year. The hurricane season in the US was the worst ever. The Great Barrier Reef is bleaching again.

So what to do? The latest number we need is 70% reduction of 1990 levels of greenhouse gas emissions.

Kyoto is the international effort. But Australia, the US and the developing world are not signed up, and its goals are too modest. Still, there has been a previous international success with CFCs and the Montreal Protocol. It started with only part sign up but then gathered steam. The depletion of atmospheric ozone has now stopped and we can hope for return of our sunscreen.

Tim Flannery points out that each individual can fairly easily get to 70%. Here’s how:

  • Tick the renewables box on your electricity bill.
  • If you have a petrol car, change to a diesel or hybrid one, or to a smaller one
  • Use gas if you are on town supply. (1 atom of carbon compared with 4 four coal)
  • Use a solar hot water system if it makes sense where you live

In my own situation, I drive a diesel, use town gas for water heating and am going to tick the renewables box.

Some other things to expect that should happen:

  • Startups trying to solve parts of this problem.
  • Gas rules supreme
  • Nuclear regains its past popularity
  • Wind, tidal, solar, geothermal and others get more popular
  • perhaps a worldwide carbon pricing and trading scheme

Australians per capita are the worst carbon polluters in the world. This is becase we use a lot of energy and we use 90% coal to produce that energy. Expect Australia to end up an international pariah before action gets taken here.

Some things you can do to get involved:

  • Read the book.
  • Get two other people to tick the renewables box on their electricity bill

Culture Shock: An Australian living in the US

In 2005 I went to the US with my family on an exchange program. We lived in Chicago, San Jose, Salt Lake City, Provo and San Diego; and visited many more places.

The experience was much different to what I expected, not withstanding some 6 previous short visits to the US. Most Australians think that our culture is about half way between Britain and the US. I thought that too. Now, however I believe Australia is mostly European in its culture.

Following is a table of comparisons, of large and small things.

Australia USA
Bacon Pork Loin (20% fat) Speck, or sliced Belly Fat (80% fat)
Sugar Cane Sugar High Fructose Corn Syrup
Common oil Canola (Rapeseed) Palm Oil
Best Selling Car Toyota Corolla 4 cylinder Ford F150 V8
Driving Side Left Right
Voltage 240 120
Language Commonwealth English American English, Spanish, AAVE
Basis of most visits to other countries Tourists Military
Religious Observance Low High
Universal Health Insurance Yes No
Average Waiting Time to See a Doctor 4 hours 4 weeks
Coffee Cups in Coffee Shops,
when eating in
China, Porcelain Always styrofoam or paper
(except two places)
Indicate to change lanes Always Rarely, even for the police
Approach to Illegal Immigration The Pacific Solution Government – blind eye, Minutemen – vigilantism
Number of Illegal Immigrants thousands tens of millions
Refugees admitted 42 per 100,000 29 per 100,000
Murders per year 150 15000
Obesity 25% 39%
Standard Dinner Plate 10 inches 12 inches
Measures Metric Imperial
Common Soft Drink Size 375ml 600ml
Bread Wheat, Yeast, Water, Salt Anything but
Number of Biscuit Types hundreds Four – all cookies
Dream The European Dream The American Dream
Percent of time I spend speeding 5%, and only to 10% over the limit 90%, often by more than 25%
Number of times booked for road infractions 8 0
Meat Pies Beef, Chicken, Vegetable Chicken only
Sausages Plain beef, pork and chicken. Many types of continental and Asian sausages Hot Dogs, Bratwurst or Polish
Social Safety Net Comprehensive Minimal
Average Mileage for Cars 20.5MPG 24.7 MPG

The result of these differences was a discomfort that grew through the year as our knowledge of the differences deepened. We gave up eating bread and bacon, and started shopping at organic and foreign food stores. We found that most restaurants were chains, and most chains were supplied by just one supp
lier, Sysco. So, we could either choose to go out and eat at Sysco, or source our own much wider array of groceries and cook at home, which we mostly did. The result was a 2kg net weight loss for the year.

We lived well in the United States but were constantly worried about what could go wrong. Two movies which illustrate the very fears we had are Fun with Dick and Jane, and Crash.

Toward the end of my stay I read a book by Jeremy Rifkin, the famous American social commentator, called ‘The European Dream’. He argues the American Dream is about “rags to riches”, self reliance, risk taking and exploitation of resources. The aim is material wealth. The European Dream is about quality of life, diversity, community and sustainability. The aim is quality of life, which is partly attained through sacrificing some income for more leisure time and a caring community. It seemed to me that this book explained much of what I experienced. Australia is mostly European Dream, except that we share with the US pioneering ideas about exploitation of natural resources, which for both countries at one time appeared unlimited.

It feels good to be home again. It seems others share my view. The Economist Intelligence Unit ranked all five major Australian cities in the top 11. The first US city is at 26. The Pew Global Attitudes Project for 2005 listed Australia and Canada in first place for most desired country to emigrate to, with the US in tenth place.