Greg Luck's Blog

OSCON 2005 kicks off tomorrow 1 August 2005.

I am here all week and am planning to update this blog entry with news and other tidbits.

Wednesday – Keynotes and Sessions

Tim O’Reilly

Tim has some interesting visualisation tools for book sales. Of interest to me personally was that C# declined last year and Java gained. The ratio of Java books to C# books sold is about 10:1. On job advertisements, Java is 16% to C#’s 4%. The irony in all this is that I now have two C# books, both O’Reilly: one called C# in a Nutshell, and a Mono Workbench book handed out in yesterday’s session.

Perl continues its long decline. Python continues to slowly build. Ruby has appeared just in the last 6 months and is growing rapidly.

Kim Polese

Kim was the original project manager for Java, while at Sun, and has been famous ever since. She is now at SpikeSource and was talking about the corporate use of runtime Java classes. She was surprised that a small number of components (20-30) are used by most companies. I was pleased to see one of my own contributions (ehcache) appear in her slides. Having surveyed this myself, I am not. SourceLabs, another open source services company has recognised the same thing. Both are providing testing and certification of predefined and custom stacks.

Andrew Morton

Andew, a fellow Aussie, is now at OSGI. He is the maintainer of the 2.6 kernel. He took a swipe at Sun for persisting with Solaris beyond its used by date.

Craig Mclanahan – Comparison of Web Frameworks/Matt Raible Spring

Struts, WebWork, Tapestry, Spring and JSF. Craig who knows a little bit about this topic, being behind Struts and JSF. A show of hands indicated most people were (still) using Struts. Spring is the current hot framework. A lot of people think its XML is too voluminous. Spring is a good example of the new frameworks that allow choice. They are platforms rather than monolithic applications. A promising new web framework which uses Java 5 annotations rather than XML is beehive currently in the Apache incubator.

It is a little clear what will happen with JSF. It has been coming for a long time, but only one person in the audience was using it.

SiteMesh

Jo Walnes, a colleage of mine, presented SiteMesh. SiteMesh is a layout manager which aims to separate “functional” from “presentation” html. It uses a filter to mesh the two together.

Tuesday Tutorials – Ajax and Mono

Ajax

This was a very enoyable tour through the fascinating world of AJAX. The letters stand for Asynchronous, JavaScript and XML. For those of you who are not familiar with AJAX, check out Google maps. Asynchroous requests are made outside the main GUI thread you interact with. This approach is how most desktop software works. The user always retains the locus of control. So your browser pages act more like a desktop app.

Of course doing this requires mindbending JavaScript. I personally hate JavaScript for lack of decent tools and consequent difficulty in developing it. DojoToolkit can do the hard bits for you. IntelliJ now supports the editing of JavaScript which also lowers the pain point. I am very keen to do my first AJAX app.

Mono

Mono is an open source implementation of C# and the CLR. I have never been able to install Monodevelop on Redhata or Fedora distros. Mono comes from Ximian, now part of Novell. Ximian has always had its own Gnome distribution, and naturally builds Mono on that. Happily you can now download an installer which works. I played with Monodevelop yesterday. It is no IntelliJ but is useable.

Mono plays very well with Linux. It is ridiculously easy to build GUI apps using Glade and Mono. You design in Glade, save it as XML and bind to it at runtime.

iKVM is shortly to be added to the standard Mono distro. It tranlates byte code to intermediate code and vice versa. So you can use Java libraries from Mono and any .Net assemblies from Java. Apparently JBoss, Weblogic and Eclipse all work fine running in the Mono CLR. Under the covers it uses GCJ and Classpath, so the limitations of iKVM are the same.

Monday 1 August – Tutorials Day 1

I went to two tutorials – Ruby with Dave Thomas and an Eclipse Session

Ruby

This session was well attended by first timers. Dave Thomas whizzed through the language features and then onto some applications of Ruby.

Who needs IDEs?

I spent some time during the session checking out IDEs for Ruby. I played with Arachno Ruby, FreeRide (some guys who write it are there), the Eclipse plugin and some others that needed the Fox GUI toolkit, which I could not install. They all seem a bit raw to me. The Eclipse plugin is promising but gives errors.

Dave Thomas uses a text editor. He is disparaging of the whole idea of an IDE. Indeed he was disparaging of the need of any feature not currently available in Ruby. Guido Van Rossum, the Python guy, has a similar attitude. Having bought Dave’s book, I also discovered irb, the interactive ruby shell.

Ruby on Linux

Ruby on Linux seems to be in an interesting state. The yum install for Fedora Core 4 does not work properly. I eventually downloaded and compiled the source. I then needed to reinstall RubyGems to get it all working. With that done it was easy to go to RubyForge and get some other stuff like ActiveRecord. Dave uses a Mac.

The Dynamic Language Wars

During the session Dave made a few criticisms of Python and Perl. At a session given by Guido at PARC in January he criticised Ruby. It seems to me that the dynamic languages are competing for hearts and minds.

Ruby seems to be targeting itself as a general purpose OO language. Rails is seen as a breakthrough in popularising it. Any missing features are glossed over. e.g. In answer to the lack of OS threads in the current Ruby Dave’s answer was “it only matters if you on a multi cpu machine…” to “..use multi process which is better anyway” to finally “it is coming in a future version”. ActiveRecord, which Dave calls a Table abstraction had limitations but “there cannot be more than two days work to make it do what Hibernate does”. Gavin King will have to slap himself for the three years it has taken him to do the original.

So will I use it?

In the end I was sufficiently impressed with Ruby to buy the book and make a committment to learning it. Hopefully I will be able to write a few useful scripts here and there.

Introduction to Eclipse

Bill Dudnin walked us through a tutorial on Eclipse. It sort of became an intro on Java/Advertorial for his company so I did not stay right to the end. (I am learning Ruby instead). However in the break Bill gave me some good answers to some problems I have been having as I have started using the Eclipse Platform.

OSGI

On Friday afternoon I was working on adding some libraries to an exisitng plugin I set up earlier in the week. All the quick attempts to get it in the classpath failed, including adding it to jre/lib/ext. Very strange. Anyway the explanation is that Eclipse implements the ideas of osgi.com, including the seemingly strange class loader behaviour.

The Usability problem

I find the Eclipse IDE very unintuitive and frustrating to use. I have seen others have the same problem. Bill assures me that is normal. Suggestion to IBM – do some usability testing, or maybe just try IDEA occasionally.

Mac OS X

The new app we are doing is in Java 5. The eclipse launchers for Windows and Linux seem fine with that. Not the Mac launcher. It wants 1.4.2. I tried hacking its package contents the other night with no success. Bill does not know about 1.5 on Eclipse and neither do I (yet).

More generally Bill said that there is quite a lot of stuff that does not work that well in the Eclipse Platform for the Mac.

The Ant afterthought

The Eclipse project does not use Ant at all, apart from one developer, who does the Ant integration. It shows. Bill suggested that anyone who was Ant centric might consider using Netbeans.

What documentation

So where do you go for documentation on all of these vexing issues? Bill reassured me that there is pretty much no decent documentation out there. In fairness, I found the RCP tutorial part 1 actually worked this time, whereas it was a real struggle the last time I played with it in January. Parts 2 and 3 remain a year out of date and looked too unlikely to work for me to go there.

Summary

I see usability and documentation as the two biggest issues with Eclipse and the Eclipse platform. My current project will require me to tame the beast. If I can learn to use Lotus Notes, I can deal with any lack of usability.

Sunday 31 July

OSCON seems to have serendipitously coincided with a beer festival on down at the river here in Portland. I am awaiting the arrival of Paul Hammant and Jo Walnes, at which time I will have a few pots. (Queensland speak). I spent the afternoon tacking down and squashing an elusive timing bug in ehcache-constructs AsynchronousCommandExecutor , an asychronous, fault tolerant, VM restart tolerant implementation of the Command pattern.

It is a hot steamy day here in Portland. Every street corner in the downtown is occupied by suspicious looking characters who approach you seemingly to panhandle, only at the last instant they don’t. It was been explained to me by the local Peet’s Coffee that these are drug dealers. Apparently they are mostly harmless. However because of the threat they will shoot up in the toilets, in the last decase all of the coffee shops closed their restrooms.

Tomorrow I am hoping for some other valuable insights into Ruby (Dave Thomas) and Eclipse (Bill Dudney). I have been working on a cross-plaform Eclipse Platform 3.1 app the last few weeks. I find it greatly improved over 3.0, however the plugin classloader appears to be incomprehensible. Sadly, the Mac launcher is 1.4.2 only, and I have not yet figured out how to launch JDK1.5 apps with it, although they work fine on Linux and that other very non open source OS.

Update 7:02pmPaul Hammant is here. I can taste the beer.

In the past, open source Java projects were largely independently distributed, often with nothing more than a sourceforge site. In the past few years branded collections of projects have emerged. The best known is the Jakarta collection at Apache. Some others are http://opensymphony.com, http://codehaus.org, and http://opensource.thoughtworks.com . The new thing emerging is commercial support and packaging into distributions “stacks” of the most popular projects.

JBoss, SourceLabs, Spikesource, Gluecode (now IBM) and others are making money out of of supporting Java open source. What you see in each case is that they only support a narrow selection of open source projects that make it into in their distributions. That the distributions are “stacks” further clarifies that there is a narrow selection at each level – often only one. They tend to go with the sweet spot: a configuration where most of the users are. The effect of this selection in turn reinforces the popularity of the selected projects.

Most users of Linux distributions or JBoss distributions stick to the projects included, because the work is done and it is easy. In the old days people would put their own together. I think we are seeing a similar change occurring in Java tools and libraries. An example of this in the Java space is http://jpackage.org/. I had an email just this weekend from someone asking about updating the ehcache packages there, because his company only used libraries that were in jpackage. I cooperated with him. As the method of distributing open source Java changes, individual projects need to support the new distribution approaches or suffer a decline in use.

A lesson in what can happen once you get dropped by a distribution is the XFree86 project. It was the standard Linux X server, a windowing system, used on millions of machines worldwide. There was a disagreement over licensing last year. Within a few months Red Hat had dropped them in favour of the X.org project. I do not know of anyone who has bothered downloading and installing XFree86 who is using an X.org system.

For those of us that write open source software, the main satisfaction is in having others use your code. The more people that use your code the greater the satisfaction. As a non-aligned open source contributor, I think there is a need to be aware of these changes in distribution to remain widely used.

For the past 5 months I have been talking to SourceLabs about open source. They are a promising startup offering support for commonly used open source. The research I have read consistently shows lack of support as the largest inhibitor of open source adoption.

To make themselves known to the open source community, SourceLabs decided to give something back. That something is swik. Swik is a little hard to describe, thus the title of this post – have you swikked yet? It is a combination of del.icio.us, freshmeat, and an open source themed wikipedia. Its aim is to be a repository of knowledge about open source projects. Like wikipedia, anyone can edit anything. Like freshmeat, it has information on open source projects. Like del.icio.us it is a social network. The hope is it will provide some much needed meta information about projects, and be more than freshmeat and do more than sourceforge search.

A lot of thought has been put into swik. Search for a project. If it does not exist, swik’s robots will go out to the usual places and discover information about it. On its own this makes using swik useful. But if others come along and add extra information then it becomes quite valuable. I encourage you to get behind swik and make it work.

Update:On 7 July Google announced the Google Toolbar for the Firefox browser on Windows, Mac OS X and Linux. This move should be applauded. Now for Google video.

For some time I have been watching Google with growing concern. When they came on the scene, I was a very early user. I thought it was cool that they used Linux and thought that perhaps they would be an open source supporter. With the release of Google Video I think there is now enough evidence to conclude they are hostile to open source.

Google’s (lack) of Open Source contributions

http://code.google.com/projects.html is a list of open source projects done by Google. 9 projects, none of which are well known to me. I personally maintain three projects and contribute to several others. My employer contributes to 150. I think Google’s project contributions are nothing more than a mild bit of PR designed to attract open source developers. But their efforts fall flat.

Google’s Use of Open Source

Google is perhaps the world’s largest user of Linux. They probably get more benefit out of open source than most companies. They use Python. They use Java. Java is not open source, but you do not need to pay for it.

Windows only here

I had a colleague who used IE when the rest of used Firefox. We asked why and he pointed out that one of his tools of trade was the Google toolbar. http://toolbar.google.com/googlebar.html. We tried an open source plugin which worked fine for me. That particular version was a frustrating install so he canned Firefox. He is still on IE.

Next we have Desktop Search. Windows only.

Then, we have hello, also Windows only.

Next, Google Earth, Windows only

Picasa, Windows only

And of course, finally Video, Windows only. What really hurts about this last one is that Google are using the open source VideoLan player. (http://www.videolan.org/vlc/) I have used this player for years on my Linux and Mac machines. VLC has binaries for every operating system out there. Google is Windows only.

The death of open standards

When Netscape came out I was a Windows only user. What struck me at the time was the support they offered for other OSes. In my case this made me think that perhaps the Internet era would be one of open standards. It largely has been. I have personally embraced open standards computing. For me Netscape was the example.

Google had, and maybe still as the opportunity, to continue that fine tradition. Many of the people that work at Google have personally made serious contributions to open source and open standards. However, by making a growing number of tools Windows only, Google is perpetuating proprietary standards and locking out alternative operating systems such as Linux and Mac OS X. If this continues it will become unviable to use these alternatives. Google, the world’s largest open source user, will become the largest open source killer. They will have succeeded where Microsoft failed.

Google is no longer cool, they are evil

Google has up to now been considered cool. Not anymore by me. And not anymore I suspect by a growing number of open source developers, Linux and Mac users. The founders love to go on about their ethics and their ‘Do not evil’ mantra. A bit like the Hippocratic oath of doctors: ‘Do no harm’. Well, unless they start supporting open standards they are doing evil.

What Google needs to do

Either:

1. Release clients for Windows, Linux and Mac OS X for each product, or
2. Work with the open source community to enable interested developers to do same

What you can do

There is a web site called Linux Appeal, where you can petition Google to support Linux for Google Video. See http://www.linuxappeal.net/petition.php?function=petition_read&petition=35

I greatly enjoyed reading Thomas L Friedman’s “The World is flat”.

In the context of international trade and also competition, the term “level playing field” is often bandied around. A “level playing field” is one where each combatant has the same rules and no specific discrimination against them.

What happens if a lot of countries implement level playing fields? The whole world becomes a level playing field – the world becomes flat.

What happens after that, according to classical economic theory, is that the law of competitive advantage applies. If one country has a natural advantage over another in the production of a particular good or service, then they will be able to charge a lower price and production will move to that country.

The thesis of Friedman’s book is that the theoretical level playing fields and competitive advantage, have become reality. He argues persuasively, citing statistics and anecdotal material from interviews with key players up to the end of 2004.

The IT industry, in particular is exemplified as a flattened industry. He portrays the Americans losing jobs to Indians, in danger of losing their jobs to the 300 million Chinese learning English. Wherever smart software engineers are found, the almost zero cost of the Internet has enabled them to compete on the playing field.

The zero Internet cost alludes to some of the key enablers of the flat world. OpenSource is cited as one. I like the way he describes OpenSource as Self-Organizing Collaborative Communities. I am beginning to think that collaborating on software that is free in a GNU sense, is almost an emergent property of the Internet.

For anyone who has lived through the IT revolution of the last 20 years, Friedman appears a little naive at times. He seems to have fallen into the clutches of Craig Mundie, CTO of Micro$oft. He rightly considers the PC a flattener. He thinks Windows was way more significant than I would ascribe. My view is, that far from being a flattener, Windows stymied innovation, and held up the emergence of the world we now have by a decade.

In the end, you have to love one of the flatteners. The date 9 September 1995. Don’t remember? What started the .com boom? Still don’t remember? Read the book.

Back in March I started sing the Subversion revision control system in anger. The experience was not totally positive, thus the title of this post. I just tried to move a project to Subversion again this week. I reverted back to CVS after a day.

Main Features

Compared to CVS, my usual RCS, Subversion offers some major new features:

Atomic Commits

Commits either all succeed or all fail. This lets you avoid the scramble, trying to avoid breaking the build, when some of the files succeed and some fail.

Every Commit is a Tag

In Subversion, each time you commit, a transaction is performed against the repository and each file and directory gets a new version number.

Binary Files are First Class

Subversion knows about binary files. It can also effectively diff them. No need for -kb when you add them.

Problems

In practically using Subversion I saw many problems that are absent from CVS. I should have wrote this article earlier, because I cannot now recall the exact causes of each. While some of the points raised may be perceived as griping, I am used to CVS which is extremely reliable and mature.

Apache SSL

We got a couple of runaway processes using https to access the repository. Not sure exactly where the problem lay. We dropped back to http, which works well.

In my recent Subversion use, we had about 100MB of binaries. The server was again set up with https, this time with certificate based security. It seemed slow to import them when I set up the module. One of my colleagues waited over an hour for a checkout. After it completed we checked the server for any problems. There were no runaway processes. For some reason it was slow.

What Security? There is no security

After a successful login Subversion stores you password in plain text in the ~/.subversion/auth/ directory. Given that a lot of people (probably stupidly) tend to use the same password, or a small number of passwords, this is very dangerous. If you know someone is using Subversion, you know where there password is. I am not sure why Subersion was designed like this. The defence that the directory is accessible only by the owner, being in their home directory is extremely lame. There is now a switch to turn this off, but the default remains to store in plaintext.

Updates

One developer had a problem where he updated, got a screen full of information about updates but it had not completed. Not sure what the cause of this was. By contrast CVS will always update.

Checksum Error

Under some circumstances, you will get a checksum error. At this point you cannot do anything but delete the file. The circumstances are: abrupt termination of the Subversion client, or a buggy subversion client. IntelliJ Irida, still in development, does this. svn cleanup does not fix the problem.

Subversion gets confused

On about three occasions, Subversion got what I can only call confused. It refused to commit or revert. These situations invariably involved deleting the directory tree in the working copy and doing an update. svn cleanup does not seem to help. Nor does svn revert.

Server Corruption

One file got corrupted on the Subversion server and had to be recreated.

IntelliJ Support

IntelliJ will fully support Subversion in version 5, codenamed Irida. It is useable now for diffs and commit browsing. Do not try to perform any adds or commits however or you will get the dreaded checksum error. Update: The new IntelliJ Irida version of IDEA supports Subversion. Given that they are getting close to release, I decided to check it out. Unfortunately a refactor involving moving files to a new package totally screwed up. Back in CVS it worked fine.

Recommendation

When I originally wrote this post, the number and type of problems we experienced suggest to me that Subversion is not ready for primetime just yet. Three months later, and with my reliance on IntelliJ IDEA, my opinion remains the same.
UpdateI tested IntelliJ 5 last week. It;s Subversion support is now very good. Note that they do not suport the file: protocol for local repositories as yet. Other than that no problems. I think Subversion is now ready for a look.

On Security

Brian Behlendorf thinks that computer systems are going to be
understood in far more biological terms in the future. We talk about
viruses and anti-viruses. How about vaccines? He points out that many
viruses and worms are not harmful but they could be. In this way they
are like vaccines, where an infectious agent spreads and causes an
immune response. Perhaps the best way to provide security is to send
out harmless but annoying viruses to keep the computing immune system
well-primed.

Linus thinks if you are paranoid about security, as one foreign defense
force person in the audience asking the question was, you would feel
safer examining the code for trojans. While a lot of work, in open
source it is always possible, while with proprietary systems, it is not
always. Also, in terms of stupidity in coding he points out that a lot
of corners are cut where the developer knows no one can see the code.
In open source “if you see someone has a comment that it does  not
work in a particular case, you ask yourself What drugs is he on?”.

continue reading…

The first question on my mind about MSN’s new search engine is whether it provides an unbiased view of the Internet. Judge for yourself

“java sucks”

Google: 410,000
MSN: 846,373

“.net sucks”

Google: 3,000,000
MSN: 245,041

I am attending the OSDL Linux Enterprise Summit in Burlingame, just south of San Francisco. Interestingly a good proportion of the attendees are corporte lawyers. The focus right now is on getting the law right to avoid future difficulties. Following are some highlights from day one.

Open Source Licensing, Larry Rosen

1. Used to be mainly about copyright, but is now needing to deal with patents as well.
2. The differences of Composite versus Derivative works and the exact legal meaning of many others terms are being defined. Their is a lack of case law (a good thing!) which makes it a challenge.
3. Larry has a book called Open Source Licensing. He is hoping Justice Scalea of the Supreme Court reads it when considering open source.

Reviewing Use of OSS in the Enterprise, Karen Copenhaven>

1. Most organisations have a contingent liability due to breach of open source licenses.
2. Session mainly on the need for and conduct of a Legal Review process
   .e.g. Product contains OSS with breach who in turn distributes that to another customer, thus propogating the breach

3. Potentially a big threat to OSS use, because it may be easier to just say no to OSS. In reality most organisations are spinning up their lawyers instead.
4. Commercial software companies are starting to offer indemnities. 5 years ago no one did. HP is an example of providing indemnities for Linux.
5. A common approach is to get the lawyers to review a project and put it on an approved list. It is not sufficient to have a list of approved licenses. The pedigree of the source must be examined.
6. Black Duck makes tools to automate open source review.

Andrew Morton

I ran into Andrew who I have met before. We talked about adoption of the 2.6 kernel. He is the maintainer of 2.6. He moved over to the US from Wollongong, Australia about 4 years ago and is doing some work for the OSDL for his employer Digio. His quote, which is the quote of the day – “Open source is inevitable”.

Novell

Plenty of Novell people were on hand today. The head of server marketing thinks that 20% worldwide desktop use is achievable in the next 5 years. Currently it is at 3.5% but just a few years ago it was barely measurable.

What values does an organisation require to enable the use of open source software? A utopian view of the world. A board of ex-communists? I have been thinking about this recently in the context of the company I work for. We have great open source skills. We are looking for the subtle cultural things that will complete our open source orientation.

Though it does not mention open source, I think a key enabling value is Standards over Products.

When you have a standard, such as a published protocol or specification, the way is open to have multiple implementations. Often you will have
commercial implementations (products) and open source implementations. When there is no standard, it is often very difficult to do an open source
implementation. By supporting standards over products, it enables a competitive software selection environment. The implementation chosen
can then rest on the quality and features of each.

An example is Rules Engines. In August 2004, JSR-94 was finalised. There are several implementations available. ILOG JRules is a high quality commercial product. drools and JESS are open source implementations. Lets say you propose an architecture which includes a JSR-94 compliant rules engine. You then enable the open source implementations to compete with the commercial products.

The value provides a level playing field and engenders fierce competition between all implementations, open source or commercial products. The decision to use open source or not, is then taken on merit.