May 07, 2008

cvs unknown host error

I am getting a but rusty on cvs. I went to run cvs up on a project and got "unknown host error". A google search did not yield anything useful. Playing around with it, my problem was that the host was not added to ssh's known hosts. I sshed to the host and accepted it into known hosts. Then cvs worked fine.

Posted by gluck at 09:14 AM | Comments (0)

May 08, 2007

Jonathan Schwartz on open source

Jonathan Schwartz on open source

(Comments from the NetBeans session)

We have been held captive by the Java developers. The future is about reaching out to people without deep expertise.

The number of people passionate about open source licenses far outweighs those who have actually read them. Everyone is a lawyer now.

If I post a hit video to YouTube, someone else is monetising my free content. Something is wrong with that (Me: There is definitely a theme developing with all of these hints about compensating those who create software or content for free. So Sun is not only open sourcing all of their stuff, but paying others for their open source stuff. Wow).

Posted by gluck at 05:41 AM | Comments (0)

Community One 2007 General Session: Tim O'Reilly, Ian Murdock and others

What is Web 3.0 likely to be?
(my question)

Tim O'Reilly "Meaningless. Most transformative when we stop typing. 411 services Google, gestural interfaces. Each cell phone with an
intertial sensor. Instrumentation of people. Insurance based on GPS. Computers learn from us by what we do in our interactions
in the daily world."

Tim Bray "Essentially unpredictable. e.g. Twitter. Don't know but really exciting."

Other Statments

Ian Murdock "A layer of abstraction is great until you really need to know what is going on underneath it"

Tim O'Reilly "Proprietary databases are the competitive advantage of the future. Critical mass of buyers and sellers made ebay a lock in. No one can touch it. All of the mapping
sites still use NavTeq."

Tim O'Reilly "Google Competing to give you the result in line, rather than passing you off to another site. Google are starting to compete with their own users."

Rich Green "Value of a platform is the ratio of the lines of code you write compared to the lines of code in the platform. Web 2.0 has that. What is the Web 2.75 Petstore?"

Tim O'Reilly "Web 1.5 was the Dot.com crash."

Rich Green "Bill Joy. Where ever you work, most of the smart people are elsewhere".

Tim O'Reilly "The architecture of participation".

Rich Green "Scoop up the open source work of others and monetise it. Sun are looking at paying people."

Tim O'Reilly "Horatio Alger said "Go west young man. You have to find something new."

Tim O'Reilly "We are coming to the end of cheap outsourcing. Indian outsourcers seeing only 3-4 years of having a wage advantage."

Tim O'Reilly "Users do their thing out of their own self-interest. "

Tim Bray "You can make money from the long tail but may be not in the long tail." and Tim O'Reilly "Each long tail is attached to a very big dog"

Posted by gluck at 04:59 AM | Comments (0)

February 27, 2007

Kerberos is cool

The last few weeks I have been working on a single sign on/Kerberos project.

For anyone for whom either of those term is new, here is some food for thought. Windoze, Linux, Mac OS X (10.4), Firefox, IE, Apache, ssh... has, in the past 10 years been Kerberised. Rather than Microsoft's Embrace Extend Annihilate being the death knell for Kerberos, their endorsement legitimised it.

Add in SPNEGO, another M$ innocation, which added browsers into single sign on, and here we are. Much of everything we need is Kerberised. An overnight success which took a decade. It is a very good time to adopt Kerberos.

Over the next little while I intend creating a Kerberos/Glassfish HOWTO, showing how to add a Glassfish security realm to a Kerberos realm. If you look in your JAVA_HOME/bin directory, you will see kinit and klist, Yes Java was Kerberised in 1.4.2. There is a Kerberos Login Config: com.sun.security.auth.module.Krb5LoginModule

Posted by gluck at 06:59 PM | Comments (0)

February 12, 2007

Wotif.com live on Glassfish

Wotif.com, my employer, went live on Glassfish in mid January. I was the driving force behind the move. You can read about it on the Glassfish site here: http://blogs.sun.com/theaquarium/entry/how_to_run_a_top .

There is a questionnaire linked from the entry that is the raw question and answer source for the entry. You can see it here: http://blogs.sun.com/stories/resource/wotif/wotif_responses.html

Though not one of the main reasons for the change, we find ourselves on the JEE5 platform, and liking it. XDoclet boiler plate generation and EJB in-container testing are things of the past. It took a few days to move our code from EJB2 to EJB3. The end result is that we find ourselves early adopters of JEE5. The opportunitites this presents and some neat ways of capitalising on them are being documented on Robert Watkin's blog. See http://twasink.net/blog/ .

Posted by gluck at 11:59 AM | Comments (1)

February 04, 2007

Online Site Trends

Luke Welling of Hitwise showed up some trends using the Hitwise database. It is mostly site visits. Interesting to compare this with Google Trends and Tiobe.

Mailing lists seem to be getting replaced by feeds.

Ubuntu matching Debian in May 06. Redhat is still getting more visits than all of the others combined.

PostGres is gaining on MySQL and Oracle is in slow decline.

PHP continues its dominance over Perl and other dynamic languages.

Digg.com eclipsed slashdot.org in 2005.

Wikipedia is huge.

DEL.ICIO.US went crazy last year.

feedster.com dropped off after site difficulties and is now coming up.

Posted by gluck at 10:44 AM | Comments (0)

Kiwi Foo - First two days

Distributed Stuff

One was on distributed caching and file systems. We talked about memcached, ehcache and a new file system from Live Journal - Mogile FS. It is user space, so not really a true file system. It comes with a mod_mogile for Apache. Must take a look at that. It looks like use of memcached is settling on page caching. Serialization issues make it a bit expensive for object caching.

Emerging Internet Security Standards

This was a wide ranging discussion on whether any standards are emerging in Internet security after 10 years of nothing more than server SSL certs being taken up.

In Instant Messaging it looks like OTR, which stands for Off the Record, is emerging. Adium for Mac, which had a version 1 release today, has it built right in. Many others have plugins. You click a lock and it asks the other party to go off the record. If they have the software they can elect to. If not, the handshake does not complete and it stays in plain text.

OpenID is the other big one. It is a single sign on system for the web. Technorati and Wikipedia are moving to it. Firefox 3 will support it (Ben Goodger confirmed it is on the wish list). Firefox 3 is also down to support SAML, the Oasis standard. Sun's Liberty Alliance specifies SAML. So hopefully something is going to happen here.

As to email, it doesn't look like anything is going to happen. We speculated that it will take someone like Google Mail to introduce a standard. Maybe S/MIME, maybe something else. Interestingly lots of ISPs are supporting using SSL for MTA. I tested mine and it does not but apparently lots of Australian ones do. One tick for confidentiality.

Ruby Roundup

There wasn't much on Ruby in the sessions. But I got around the rubyists. I met the guy who decided not to accept a patch for Oracle prepared statements. "Our user community does not really use Oracle and prepared statements can be harmful to PostGres performance and are neutral for MySQL". One guy is an ex-Rubyist. His complaint was the community. Naive implementations solving problems that were elegantly solved 10 years ago, with the implementers thinking how cool they are. In the next breath I heard how banks and big enterprises are no longer interested in Java and are all doing Ruby. I look forward to them changing over to MySQL because the Ruby community does not care about Oracle.

There seems to be high respect for the JRuby project. It looks like it will imminently be able to run Rails. There are a few people showing how to deploy it on Glassfish. The Netbeans guys are making much of adding not just Ruby but Perl, Python and PHP language support. And IntelliJ IDEA released just a few days ago their Ruby plugin through their plugin manager. I played with it again (the last time was a few months back). It is ok. It has RDoc with CTRL-Q, syntax highlighting and minimal keyword auto completion. It needs to be much better to get me enthused. Allan Odgaard, the TextMate author, pointed out that Ruby autocompletion has just been added as a plugin to TextMate. (His focus is on supporting 100 languages, so he himself does not do too much that is language specific - he likes the Emacs analogy).

Tiobe announced that Ruby was the language of 2006, because it had the most growth. The latest Tiobe shows a leveling off of Ruby. Not sure if this is temporary or whether it will level off below Python.

Overall the super buzz around Ruby seems to be dying down.

Perl

Allison, a Parrot VM architect was here. I asked her about whether an open source JVM might get used by Perl and others. She thinks maybe, but the independent efforts, which are designed for dynamic languages will continue.

Segway

We all had a go on a Segway that the LiveJournal guy brought with him. It has a black key and a red key. Red is for fast. We all graduated from black to red and a had a great time. You delegate balancing to the machine. The rest is easy.

Google

The google guys were conspicuous by their t-shirts and tight-lippedness. Google's share price has taken a bit of a tumble, which was the main discussion. Chris De Bono, helpfully "does not comment on future share prices". This is very similar to his comment from OSCON two years ago in answer to questions about better Linux support with he "does not comment on future product announcements". With conversation that exciting I moved on. Interestingly, given that there now a 1/googol chance of any new hires making anything out of Google employee optios, all the Googlers are emitting a new company line that Google has found new creative ways to entice developers to join. I am up to three recruitment approaches from Google now in the past two years and am probably now on the do not call list.

I did manage to meet Lars Rasmussen who was on the team that did Google Maps. Due to its Cuba policy the USA has lost Lars and his Cuban wife to a country (Australia) that has normalised relations with Cuba. A good win for Australia.

Java

Lots of Java people haunting the corridors. Thankfully they were all IntelliJ users, so we could escape the oppression of the Eclipse nazis for a short time. I met the guy who did the TestNG plugin. JUnit 4 is at the end of the day not that exciting. I might take a look at TestNG. I spread the word about Glassfish which I am now using at work in production. I think it will be big. It seems clear that Glassfish is the platform to be on for new experiments. Some of the non Java guys, who do not exactly keep up with Sun news, were interested to hear that Sun had done an open source Java app server. And the open sourcing of Java is big. Glassfish is coming on to the Linux distros (Ubuntu already announced) so they will see it.

Linux

It was agreed that Linux newbies prefer Ubuntu these days. A grizzled Debian veteran still prefers the original formula. I showed off my Macbook running FC6 on VMWare Fusion in answer to questions about whether I had used Parallels.

Politics

Lots of it at the conference. We even had the NZ Communications Minister with us. He seemed very well informed on IP issues. NZ is doing a new Copyright Act and trying to avoid the disgraceful Australian precedent done as part of the Australia - USA bilateral trade deal where we let the US export their IP laws to us in exchange for a little bit of market access for our farm products.

He suggested that patent infringement lacked mens rea. I suggested that it is now hard to write a line of code without infringing some patent. So we all have mens rea, but we have no idea what patent we are infringing.

Corporates still seem best served by building up patent war chests. A bit hard for those of who do open source.

Posted by gluck at 10:26 AM | Comments (0)

October 29, 2006

The effects of Sun's Open Source Strategy

I was at Enterprise Java Australia's annual get together a few weeks ago.

The highlight for me was listening to Simon Phipps lucid telling of the open source strategy at Sun. It felt to me that the presentation we got was not too dissimilar to the one Phipps used at Sun.

It starts with Bill Joy's observation that "You can't hire all the smart people". The solution to this very real problem is that open source gives you access to them. You don't need to hire the smart people if you can access their output and you are not trying to compete on that.

Given that version numbers are now popular Phipp's has his own: Software Market 3.0. (Didn't anyone ever tell these guys you never use a .0 version of anything?). In this market you pay for services and software at the time of production, not before.

The mantra for all of this:

1. Collaborate over what does not differentiate. Do this using open source. Encourage open standards, because these mean substitutability.
   
2. Compete by innovating on top of the commodity base.
3. Contribute!

So how is this being applied?

Sun Application Server 9 is open source => Glassfish. Sun's Message Queue 4 is open source => Open Message Queue. The JDK is going to start to be, in November, according to Phipps.

And what are the effects?

Application Servers

For many years I have been using Orion as the standard server I install on new products. It was cheapware and I could get it in without a lot of fuss. Last year I switched to JBoss. It was OSS, and worked well in the applications I used it in.

Right now I am moving from Orion to Sun Application Server. The reasons are:

1. JBoss's Marc Fleury just gets up the nose of people. Some people hate JBoss and will not use it.
   
2. I don't want an expensive app server, because I will spend all of my time justifying new licenses rather than developing. Getting it approved for some applications and not others just leaves a mess.
3. Sun App 9 is just that: version 8. The biggest problem I have had with it is the quick start feature where services are started lazily, which is new in 9 - it is a great idea but buggy. Turn it off and things work great.
4. Glassfish is open source. You can step through the App Server code. If you get a bug you can often see the fix for it and add that to your bug report. The worst case is that you can patch your own version temporarily.
5. Glassfish is one of the first JEE 5 implementations. I will move to a POJO architecture by moving to EJB3. But according to Ben Alex, both EJB3 and Spring are POJO architectures. Josh Mackenzie of POJO fame, Paul Hammant and the light weight crowd have won. But it does not mean you need to use Spring to achieve it. It looks like migrating to EJB3 is going to be fast and painless. And no XML. Nada.
6. I can buy production support and training. Prod support is about AUD1500 per year per server and the training is about AUD800. Cheap. But revenue Sun was not getting from me before.

For Sun rather than getting a large licensing revenue from a small number of customers, they potentially get a small amount from a large number. They will be getting money from me they never got before.

In I, Analyst the blog of Richard Monson-Haefal Richard points out that Sun's market share grew by 6 percentage points over 2005 to 19.7% as of December 2005. See http://rmh.blogs.com/weblog/2006/05/bz_research_on_.html . He credits Glassfish as the likely reason. It will be interesting to see how things look this December, as Glassfish was only released in May.

Open Source Projects
We are currently using ActiveMQ 3 series. We have hit some bugs with it that seem to be longstanding. The 3 series is now being somewhat maintained by the Gluecode guys. With Glassfish we are using the built-in Sun Message Queue, aka Open Message Queue. Once again this is a version 4 product. It is being maintained by Sun Engineers. All of the enterprise features, including clustering, are there. Does this mean we will upgrade to ActiveMQ 4, or does it mean we will move to Open Message Queue. Time will tell.

I am a little worried, as an open source maintainer myself, that Sun's dumping of commercial quality software onto the open source world may kill many open source projects. But perhaps we will all finally have achieved our aim if we our own projects become obsolete.

Language Implementers
During an extended language discussion with Dave Thomas (not the prag one) a few months ago he would keep dismissing languages as toys. Finally it came down to a language is a toy if it does not have a decent virtual machine/runtime interpreter. The only two that are decent: JVM and CLR. I think Dave might have a point here.

Some stuff going on:

• Perl is creating a Parrot virtual machine

• The Ruby world has one right now that is 15 times slower than C. (see earlier blog) There are lots of VM projects. See http://wiki.rubygarden.org/ruby/page/show/VirtualMachineOptions. Ruby2 is planned to use Rite. See http://wiki.rubygarden.org/Ruby/page/show/Rite
• Microsoft needed a multi-language VM from the start. Projects to run Python (Iron Python), Ruby (Garden's Point Ruby.Net - see http://www.plas.fit.qut.edu.au/rubynet/) are well under along.

Clearly every language needs a great and performant VM that runs on all architectures and OSs. Why should each language implement its own? The answer is trust. Would you trust Microsoft? Or Sun? But if Sun open sources the JDK, that solves the trust issue.

Sun are adding invokeDynamic and some other goodies to the next JDK to better support dynamic languages. And they are open sourcing it. If this is done right, the other language implementers may drop VM work and just provide a JDK implementation. This strategy could see the JDK (renamed to some cool name of course like my suggestion of Hydra Virtual Machine) move into a central position in the open source world that it has never occupied before.

Couple this with Sun's new USD99 per incident developer support offering, and once again, Sun may see revenue they have never seen before. Once again of the type small amount * large number.

The law of large numbers

In summary, I think the totality of what Sun is doing amounts to a revolution. It may touch the lives of many of us. If it does the law of large numbers may see Sun's coffers brimming over with software and services revenue. I am not sure if this is Jonathan Schwartz's doing, but someone at Sun seems finally to have come up with a great software strategy.

Posted by gluck at 12:34 PM | Comments (4)

July 27, 2006

OSCON2006: Tim O'Reilly Keynote - Open Source Trends

New trends:

Ruby books are now outselling Python and Perl books. But JavaScript books have increased the most and are outselling all of the other dynamic languages. Why? Tim thinks it is driven by interest in Ajax, which is the hottest thing right now. Time to get over my JavaScript hatred. IntelliJ helps with that a lot. (Also interesting that Rhino is bundled in JDK 1.6)

The other thing new on the horizon is Django. Tim pointed out that both Rails and django grew out of closed source projects: Rails from 37 Signals, and django from Lawrence Journal World. django is Python's answer to Rails, so it will be an interesting one to watch.

Virtualization is a big new trend.

Another one is that being on someone's platform increasingly means you will also be hosted on their infrastructure.

Open Data. Owning your own data and being able to take it with you.

Firefox is the equivalent in the browser world. (Tim did not say this but someone said something yesterday about Eclipse being the new Emacs).

Asterisk and open VoIP is a big deal.

Ubuntu is on the rise. It has huge interest relative to RedHat.

Posted by gluck at 03:06 AM | Comments (0)

July 26, 2006

Report from OSCON2006: The Ruby Conspiracy

(Update: Wow I got a record number of comments to this blog. Answers to some common themes at the end of the post)

Who are those who are benefiting from Ruby on Rails? Answer: O'Reilly Publishing, the authors Bruce Tate and Dave Thomas and a handful of consultants.

At last year's conference, Tim O'Reilly had carefully analysed his book sales and was desperate to identify the next big thing. Same for the pragmatic programmers and for consulting companies wishing to push the next big thing. C# had been and is a disappointment. Despite a huge push it refuses to move up the Tiobe programming index.

I get the feeling that everyone needs a next big thing, and if there is not one, they create it. So what has happened to Java, after the spate of Beyond Java books? Answer: according to tiobe it has risen higher. So what is declining? VB.net and Perl. Not Java.

So am I ignorant of Ruby on Rails? We have two production applications running on Ruby. And how is it. Well, despite being perhaps no more than 5% of the functionality of our applications, Ruby on Rails is the number one consumer of Oracle CPU and logical gets. Why? Rails does not support prepared statements, so Oracle has to reparse every time. This is something that Java has had for years and years. And ActiveRecord seems not to have learnt Hibernate's lession; that OR tools suck for performance and need caching tricks to make them work well. Also, our Rails apps running in (now unmaintained) fast-cgi regularly go awry and fork more processes. Each one creates a new connection to Oracle. So, the opposite of connection pooling; connection denial of service. And does Ruby support Unicode. Not really. And is Rails threadsafe? No. So, is it Enterprise Ready. Absolutely, according to those with a clear vested interest in supporting the next big thing.

Are these problems solvable? Yes, in a fashion. For example, I have been told that lighthttpd and Mongrel should be used rather than fast-cgi. And MySQL should be used rather than Oracle.

And does it matter that Ruby is 15 times slower than Java. Of course not. How could it! Just buy more hardware. And more hosting costs. And more System Administrator salaries.

After all, the productivity benefits of Ruby are so much greater than Java you will save all of the money in development. Or do you. Our experience was that Ruby on Rails took longer than Java would have. And what about maintenance. Well we just refactor as things change. Or do we? There are no Ruby tools that support refactoring. And nor are they are expected due to the difficulties of implementing refactoring tools for Dynamic Languages, or so I am advised.

And what about support. Well there is the Ruby mailing list. Which is quite active. That should be good enough for anyone.

And what about Python? Python is arguably a more mature dynamic language with a much larger developer community and number of libraries. Why does it suck? Underscores and "self", according to one of the leading Ruby advocated. Wow, that sounds really bad. Doesn't Martin Fowler's Refactoring book recommend _ for fields? Must be deprecated. Oh and what about Django? Now this really sets the Rails people off. Why? Because it is Rails like. A rifpoff they say. Yes, but where do the Rails ideas come from. From my point of view it exactly what I have been doing in Java for years and years. So if Ruby can rip off from Java (most welcome BTW) why cannot Django ripoff Ruby on Rails? Answer, because the vested interests have decided there is money to be made from promoting Ruby, not Python.

In short, anyone who questions the benefits of Ruby on Rails is not with the program. You know, once upon a time, being open source meant being better because of improvements spurred by constructive criticism. Its about time the Ruby on Rails community accepted some.

Anyone for Haskell? Or J2EE 5?

(Answers to common comments:

1. Where are the line breaks?
Ah, sorry for that. This was a late night post, after a drinking session. Forgot to check "Convert Line Breaks" in the Text Formatting option. Also, I have been using Writely lately to blog, until it broke :)

2. Thanks for saying what everyone is afraid to?
No problem.

3. Why are you so ignorant?
I am capable of learning. Really. Enlighten me.

4. Why are there no references to back up your claims?
Some of the claims are explored more fully in other blog posts. The one about Ruby performance is based on freely available benchmarks. How important is it? The database is normally the slowest part, but it depends on your app.

5. Who was the nasty Railroader who claims django is a rip off?
I personally did not realise that django predated Rails. The Railroader was very tetchy about any success django might be having. He described django as "Rails-like". The rip off part I think was was a false connection I made. I do however think that the django crowd have realised the importance of hype. So perhaps to that extent they have learnt something from the Rails crowd.

6. What about Haskell?
Sadly no one that posted was that interested in Haskell. Our speaker yesterday said it takes a year to get into it, so maybe that is why.

Posted by gluck at 06:23 PM | Comments (66)

February 28, 2006

Going Web 2.0

I do not normally take to buzz words, hype and shiny new toys. These things usually annoy me.

I do however think there is something to Web 2.0. I don't want to go into what is meant by that here. See Tim O'Reilly's landmark essay for a definition. If you look at the technologies that go into it, most of them are late 90's. While Tim provides a broad explanation of what Web 2.0 is, for my purposes I am interested in the following technologies:

• CSS, semantically valid XHTML markup, and Microformats
• Unobtrusive rich application techniques (such as Ajax)
• XUL
• SVG
• Flash Remoting
• Syndication of data in RSS/Atom
• Aggregation of RSS/Atom data
• Clean and meaningful URLs
• Weblog publishing
• REST or XML Webservice APIs
• Some social networking aspects

Some of the browser standards introduced in the late 90s are:

• W3C Document Object Model (DOM) Level-1 Specification (October 1998)
  

• W3C RDF Recommendation (February 22, 1999)

• W3C CSS1 Recommendation (December 1996)

• W3C HTML 4.0 Specification (April 24, 1998)

So why are we proclaiming Web 2.0 in 2005/6? The answer I think is two fold:

First, I feared but expected that with IE dominance, Microsoft would go on to define a Web 2.0 of its own; one that work with Windows. While they have played a major part in Web Services it is the community itself that created the new platform - just like Web 1.0. The Web 2.0 stack prefers open standards and standards based browsers. Firefox and Safari both led the charge with built-in RSS readers and concern for web standards. Microsoft's dismantling of their IE team led to a period of stability where it was possible to build frameworks by working around limitations. Microsoft, very strangely in my experience, is paying some attention to web standards. A review of the IE blogs shows that there will be some movement toward greater CSS compliance, together with RSS support. IE7 is pretty much a catchup to Firefox/Safari for Microsoft.

Second, there is the network effect. Metcalfe's law states that the total value of a good or service that possesses a network effect is roughly proportional to the square of the number of customers already owning that good or using that service. A mobile phone example of the power of this law can be found in SMS. All GSM phones have SMS. While the 3G bandwidth speculators bid up spectrum licenses to dizzy heights the Generation Xers and Ys stuck to texting. SMS was created in GSM Doc 28/85 rev2, June 1985. SMS was ubiquitous precisely because the standard was so old. New standards such as MMS are not supported on all phones. People stick to what they know will work.

So, standards are introduced in the late 90s; time ticks by; an installed base grows and becomes ubiquitous; some smart developers build to it. Voila! Web 2.0. Tim's article partly defines Web 2.0 through exemplars. I think one of the best ways of web2ifying your mind is to configure netvibes as your home page. Netvibes is a great example of Web 2.0 in itself and lets you add portlets? widgets? Web 2.0 bits? from plenty of others.

I think what comes next is competitive advantage by being Web 2.0 compliant. A bit like when the web itself was new, and you were either on it or not. Back then there was a competitive advantage in going online. I think there is a window of opportunity right now to gain competitive advantage from Web 2.0.

(Attribution: Many of the links for this post were shamelessly lifted from Wikipedia)

Posted by gluck at 12:11 AM | Comments (1)

February 26, 2006

A patent on digital computing

Last year I attended a meeting of the ACM at HP in San Jose. The topic was the early history of computing. During the session the presenter touched on a patent court case for a patent on digital computing that had occurred in 1973. I was not exactly up with the latest tech news back then and missed the story. That you have not heard of the case if because the patent was not upheld. Had it been, and had the patent applicant done the usual evergreening, computer might still refer to a job role, rather than a machine.

I am probably not anti patents; just anti the present US patent system which is being foisted on the world through their bilateral trade deals. Yes, unfortunately Australia now grants software patents. Hopefully the Europeans will bring some sense to this by continuing to refuse to go the US way. My opinion is that these temporary government granted monopolies should be granted in far fewer cases than they now are. The original idea when they were first introduced in England hundreds of years ago was to foster innovation, not to prevent it.

This very fascinating story is related at http://www.scl.ameslab.gov/ABC/Trial.html.

As an aside, the court case also found that John Vincent Atanasoff and Clifford Berry had constructed the first electronic digital computer at Iowa State College in the 1939 - 1942 period, not the Mauchly and Eckert who had been taking credit for three decades. The Wikipedia entry gives a balanced account. Encarta does too, even mentioning that Atanasoff was not given credit until the court case in 1973.

Posted by gluck at 11:28 PM | Comments (0)

February 08, 2006

Whats up with SourceForge?

Something seems to be up with SourceForge.

I have been using it heavily this week. CVS has been down three times for extended periods. The Admin web site has been down 4 times. This is seriously interrupting me!

There are two possibilities:

1. This is just a run of bad luck
2. Sourceforge is underfunded

To explore the second option lets look at the Form 10 filed by VA Software on 12 December 2005.

Sales of Sourceforge software declined from 1.9m to 1.4m. R&D on Sourceforge dropped from .9m to .8m.
Only 136 customers have licensed SourceForge.

Sales from advertising have gone up 36% over the same period.

Overall VA Software had a net loss of 1.2m for the 30 September 2005 quarter.

What is unclear is how much of the advertising revenue comes from sourceforge.net and how much comes from slashdot.org. Based on my own viewing habits I would guestimate that less than 10% comes from sourceforge.net. Running slashdot is much simpler than running sourceforge. Slashdot is a blog. Sourceforge is an entire software development hosting environment complete with compile farms.

So I think the second possibility, underfunding, is the likely reason for the outages. It may also be the cause of the sluggish introduction of subversion, which is about to come out of beta and be generally available, and the long period where project statistics were not available.

It may be there that there is simply not enough money in it.

Having said that, the importance of sourceforge to the open source community cannot be overstated. If they decide to offload it I hope it finds another benevolent home. Sun? O'Reilly? The UN?

Posted by gluck at 03:41 PM | Comments (1)

February 06, 2006

Release early, release often.

Linus Torvalds said right from the beginning "Release early, release often". This seems to be a critical requirement for open source projects to attract collaborators. There is a whole chapter on this in The Cathedral & The Bazaar, the classic open source text.

Having released ehcache-1.2beta3 with the distributed stuff in, some old collaborators and new ones have come forward. Making the beta release is a symbol that the new version will be released. Collaborators know their efforts will not be wasted.

Apart from releases here is what has been going on with lines of code:


Fisheye is cool. See the whole ehcache fisheye here. The steeper the line the more you have been doing. The flat lines represent periods of inactivity.

I am now driving ehcache-1.2 to final release. If anyone else is interested in helping out contact me at gluck _ A T _ gregluck.com.

Posted by gluck at 09:56 PM | Comments (0)

January 18, 2006

Relative Browser Market Share Trends

It is hard to work out exactly which browser market share statistic to pay attention to. The numbers vary depending on whether people mainly access from work and by region around the world, as well as the type of site. Be that as it may I think the trend line is interesting and perhaps more relevant.

Below is a chart for Firefox/Mozilla and miscellaneous browsers compared with IE for the past three and a half years as measured by the famous w3schools.com browser market share survey.



The absolute percentage for Firefox/Mozilla et al for January 2006 is 33.60%. This is one of the highest percentages reported. w3schools is generally a leading indicator, because the web cognoscenti and web designers and authors frequent it. More interesting than that is the trend.

Having been involved with many project decisions about which browsers to support, I always felt that 10% non IE was necessary to ensure sites were built to standards and interoperability. That 10% was in danger in 2001/2002. The only thing that saved one high profile site I worked on was the concern about journalist reviews, who were a strong Mac community. Mozilla had not been doing much to capture new users, though I used it right through from version 0.90. Firefox came along just in time really.

Where it will end? I think most users will always want to go with just what their OS gives them. I tend to. On Linux it is Firefox and on Mac OS X it is Safari, which I mainly use. I also have IE on CrossOver Office running on Linux and Firefox and Mozilla on Mac. According to w3schools, operating system visits for Mac/Linux went from 4% to 6.8% over the past three years. This represents a hard base of non IE users. Add to that what can be captured from Windows users. Much depends on whether manufacturers bundle it. This is starting to happen. In October HP announced that users would be able to choose Netscape at install time on and in December Dell announed bundling of Firefox.

No matter where it ends there is now enough non IE marketshare to assure web standards and to avoid Microsoft domination of standards. RSS is an example of something that came out of the community and is now a standard browser feature. Another is tabbed browsing and yet another is pop-up blocking.

The real worry is the low non-Windows market share. There are sites like WebEx and Google that offer degraded or no services to non Windows users. (Try using Google Video as a Linux user for example.) Hopefully the new Macs will help build that share up.

Posted by gluck at 02:44 AM | Comments (0)

January 17, 2006

The Sun and Moon: Thailand's Suriyana and Chantra Open Source Windows Distributions

Pantip Plaza in Bangkok is the mecca for software pirates. It is 6 stories of booths, probably around 500 500 in total, selling up to the minute versions of software for 100 Baht (USD2.50) per CD or DVD. International travellers from all over the world flock there in droves.

I picked up one piece of software at Pantip when I was there in late December and it was not pirated. There was a crowd around one booth in particular near the doors on the ground floor. There was a yellow banner as backdrop, as shown below.




Yes, the buzz was about a Thai open source distribution for Windows: Chantra. The version I picked up was 1.0, which has now been followed up by version 1.1. Thai's but not internationals were providing their email addresses and names and picking up a free CD and brochure.

Chantra, which is Thai for Moon was released in November 2005. It includes, inter alia, the following:

1. OpenOffice 2.0 Thai version
2. Dia
3. FreeMind
4. Firefox
5. Thunderbird
6. BitTorrent
7. Gaim
8. Gimp
9. Inkscape
10. VLC
11. Audacity
12. 7-Zip
13. ClamWin
14. Nvu
15. FileZilla
16. PuTTY
17. Notepad2
18. AppServ
19. Subversion
20. Tux Racer

In the 20 minutes I was there about 30 people walked away with CDs. Chantra is distributed by an organisation called SIPA, part of the Thai government. They operate an open source portal at thaiopensource.org, which was showing 411,637 hits after a month of operation.

SIPA are also planning to release the Suriyanâ server software distribution. Suriyanâ is Thai for Sun. So we have the Sun and the Moon. Nice.

It is very encouraging to see this type of thing going on. I suspect it is a reaction to pressure on Thailand to cut down on piracy. Hopefully, efforts like these will mean that piracy will be substituted for open source use, which will help to drive the open source movement.

Posted by gluck at 09:03 PM | Comments (0)

December 01, 2005

Seeing through Google's nontributions

I have been aggravated and annoyed for some time by Google pretending that their use of open source software entitle them to say they contribute. My personal open source contributions rival their entire corporate contribution. Google have some great open source developers working for them - on closed source software. But Google do not contribute significantly to open source. Indeed by ignoring Linux in many of their intiatives they threaten it.

So it was with some pleasure that I happened along a blog entry by Navaneeth Krishnan on his attendance at FOSS in India. It is reproduced below.

http://weblogs.java.net/blog/navaneeth/archive/2005/11/fossin_intel_go_1.html

2:30 pm
"Google and Open Source" by Zaheda Bhorat begins. The whole Google thing looks like a recruitment and PR drive. They have a "We are hiring" caption right in their presentation slide. Google's contribution to open source is that they run on Linux. I wonder if that's Google's contribution to Linux or Linux's contribution to Google. And of course it seems they have open source examples of how to use the Google API.

The next 30 minutes is about the Google Summer of Code. Zaheda discusses how the whole stuff works and how successful the program was. She then gets a student on stage who shares his experiences being a participant in the Summer of Code. I hear him say "Thank you google" at least thrice.

Posted by gluck at 10:55 PM | Comments (2)

November 29, 2005

Writely - A Web Document Editor that supports ODF

This is a test of writely, a web document editor. I edited this documetn on Firefox on Linux.

A WYSIWIG Blogging Front End

It supports publishing to most types of blog, including Movable Type, which is why you are reading this on my blog.
I published it there.

Open Document Format

It also supports saving documents using Open Document Format, an open standards-based interchange format being backed
by Sun, IBM, Adobe, Google, Corel and others. This format was initially supported in Open Office, but being an open standard,
it is now supported in many others, including writely.

Posted by gluck at 12:46 PM | Comments (0)

August 01, 2005

O'Reilly Open Source Conference Coverage

OSCON 2005 kicks off tomorrow 1 August 2005.

I am here all week and am planning to update this blog entry with news and other tidbits.

Wednesday - Keynotes and Sessions

Tim O'Reilly

Tim has some interesting visualisation tools for book sales. Of interest to me personally was that C# declined last year and Java gained. The ratio of Java books to C# books sold is about 10:1. On job advertisements, Java is 16% to C#'s 4%. The irony in all this is that I now have two C# books, both O'Reilly: one called C# in a Nutshell, and a Mono Workbench book handed out in yesterday's session.

Perl continues its long decline. Python continues to slowly build. Ruby has appeared just in the last 6 months and is growing rapidly.

Kim Polese

Kim was the original project manager for Java, while at Sun, and has been famous ever since. She is now at SpikeSource and was talking about the corporate use of runtime Java classes. She was surprised that a small number of components (20-30) are used by most companies. I was pleased to see one of my own contributions (ehcache) appear in her slides. Having surveyed this myself, I am not. SourceLabs, another open source services company has recognised the same thing. Both are providing testing and certification of predefined and custom stacks.

Andrew Morton

Andew, a fellow Aussie, is now at OSGI. He is the maintainer of the 2.6 kernel. He took a swipe at Sun for persisting with Solaris beyond its used by date.

Craig Mclanahan - Comparison of Web Frameworks/Matt Raible Spring

Struts, WebWork, Tapestry, Spring and JSF. Craig who knows a little bit about this topic, being behind Struts and JSF. A show of hands indicated most people were (still) using Struts. Spring is the current hot framework. A lot of people think its XML is too voluminous. Spring is a good example of the new frameworks that allow choice. They are platforms rather than monolithic applications. A promising new web framework which uses Java 5 annotations rather than XML is beehive currently in the Apache incubator.

It is a little clear what will happen with JSF. It has been coming for a long time, but only one person in the audience was using it.

SiteMesh

Jo Walnes, a colleage of mine, presented SiteMesh. SiteMesh is a layout manager which aims to separate "functional" from "presentation" html. It uses a filter to mesh the two together.

Tuesday Tutorials - Ajax and Mono

Ajax

This was a very enoyable tour through the fascinating world of AJAX. The letters stand for Asynchronous, JavaScript and XML. For those of you who are not familiar with AJAX, check out Google maps. Asynchroous requests are made outside the main GUI thread you interact with. This approach is how most desktop software works. The user always retains the locus of control. So your browser pages act more like a desktop app.

Of course doing this requires mindbending JavaScript. I personally hate JavaScript for lack of decent tools and consequent difficulty in developing it. DojoToolkit can do the hard bits for you. IntelliJ now supports the editing of JavaScript which also lowers the pain point. I am very keen to do my first AJAX app.

Mono

Mono is an open source implementation of C# and the CLR. I have never been able to install Monodevelop on Redhata or Fedora distros. Mono comes from Ximian, now part of Novell. Ximian has always had its own Gnome distribution, and naturally builds Mono on that. Happily you can now download an installer which works. I played with Monodevelop yesterday. It is no IntelliJ but is useable.

Mono plays very well with Linux. It is ridiculously easy to build GUI apps using Glade and Mono. You design in Glade, save it as XML and bind to it at runtime.

iKVM is shortly to be added to the standard Mono distro. It tranlates byte code to intermediate code and vice versa. So you can use Java libraries from Mono and any .Net assemblies from Java. Apparently JBoss, Weblogic and Eclipse all work fine running in the Mono CLR. Under the covers it uses GCJ and Classpath, so the limitations of iKVM are the same.

Monday 1 August - Tutorials Day 1

I went to two tutorials - Ruby with Dave Thomas and an Eclipse Session

Ruby

This session was well attended by first timers. Dave Thomas whizzed through the language features and then onto some applications of Ruby.

Who needs IDEs?

I spent some time during the session checking out IDEs for Ruby. I played with Arachno Ruby, FreeRide (some guys who write it are there), the Eclipse plugin and some others that needed the Fox GUI toolkit, which I could not install. They all seem a bit raw to me. The Eclipse plugin is promising but gives errors.

Dave Thomas uses a text editor. He is disparaging of the whole idea of an IDE. Indeed he was disparaging of the need of any feature not currently available in Ruby. Guido Van Rossum, the Python guy, has a similar attitude. Having bought Dave's book, I also discovered irb, the interactive ruby shell.

Ruby on Linux

Ruby on Linux seems to be in an interesting state. The yum install for Fedora Core 4 does not work properly. I eventually downloaded and compiled the source. I then needed to reinstall RubyGems to get it all working. With that done it was easy to go to RubyForge and get some other stuff like ActiveRecord. Dave uses a Mac.

The Dynamic Language Wars

During the session Dave made a few criticisms of Python and Perl. At a session given by Guido at PARC in January he criticised Ruby. It seems to me that the dynamic languages are competing for hearts and minds.

Ruby seems to be targeting itself as a general purpose OO language. Rails is seen as a breakthrough in popularising it. Any missing features are glossed over. e.g. In answer to the lack of OS threads in the current Ruby Dave's answer was "it only matters if you on a multi cpu machine..." to "..use multi process which is better anyway" to finally "it is coming in a future version". ActiveRecord, which Dave calls a Table abstraction had limitations but "there cannot be more than two days work to make it do what Hibernate does". Gavin King will have to slap himself for the three years it has taken him to do the original.

So will I use it?

In the end I was sufficiently impressed with Ruby to buy the book and make a committment to learning it. Hopefully I will be able to write a few useful scripts here and there.

Introduction to Eclipse

Bill Dudnin walked us through a tutorial on Eclipse. It sort of became an intro on Java/Advertorial for his company so I did not stay right to the end. (I am learning Ruby instead). However in the break Bill gave me some good answers to some problems I have been having as I have started using the Eclipse Platform.

OSGI

On Friday afternoon I was working on adding some libraries to an exisitng plugin I set up earlier in the week. All the quick attempts to get it in the classpath failed, including adding it to jre/lib/ext. Very strange. Anyway the explanation is that Eclipse implements the ideas of osgi.com, including the seemingly strange class loader behaviour.

The Usability problem

I find the Eclipse IDE very unintuitive and frustrating to use. I have seen others have the same problem. Bill assures me that is normal. Suggestion to IBM - do some usability testing, or maybe just try IDEA occasionally.

Mac OS X

The new app we are doing is in Java 5. The eclipse launchers for Windows and Linux seem fine with that. Not the Mac launcher. It wants 1.4.2. I tried hacking its package contents the other night with no success. Bill does not know about 1.5 on Eclipse and neither do I (yet).

More generally Bill said that there is quite a lot of stuff that does not work that well in the Eclipse Platform for the Mac.

The Ant afterthought

The Eclipse project does not use Ant at all, apart from one developer, who does the Ant integration. It shows. Bill suggested that anyone who was Ant centric might consider using Netbeans.

What documentation

So where do you go for documentation on all of these vexing issues? Bill reassured me that there is pretty much no decent documentation out there. In fairness, I found the RCP tutorial part 1 actually worked this time, whereas it was a real struggle the last time I played with it in January. Parts 2 and 3 remain a year out of date and looked too unlikely to work for me to go there.

Summary

I see usability and documentation as the two biggest issues with Eclipse and the Eclipse platform. My current project will require me to tame the beast. If I can learn to use Lotus Notes, I can deal with any lack of usability.

Sunday 31 July

OSCON seems to have serendipitously coincided with a beer festival on down at the river here in Portland. I am awaiting the arrival of Paul Hammant and Jo Walnes, at which time I will have a few pots. (Queensland speak). I spent the afternoon tacking down and squashing an elusive timing bug in ehcache-constructs AsynchronousCommandExecutor , an asychronous, fault tolerant, VM restart tolerant implementation of the Command pattern.

It is a hot steamy day here in Portland. Every street corner in the downtown is occupied by suspicious looking characters who approach you seemingly to panhandle, only at the last instant they don't. It was been explained to me by the local Peet's Coffee that these are drug dealers. Apparently they are mostly harmless. However because of the threat they will shoot up in the toilets, in the last decase all of the coffee shops closed their restrooms.

Tomorrow I am hoping for some other valuable insights into Ruby (Dave Thomas) and Eclipse (Bill Dudney). I have been working on a cross-plaform Eclipse Platform 3.1 app the last few weeks. I find it greatly improved over 3.0, however the plugin classloader appears to be incomprehensible. Sadly, the Mac launcher is 1.4.2 only, and I have not yet figured out how to launch JDK1.5 apps with it, although they work fine on Linux and that other very non open source OS.

Update 7:02pmPaul Hammant is here. I can taste the beer.

Posted by gluck at 12:41 PM | Comments (0)

July 12, 2005

The changing face of open source distribution

In the past, open source Java projects were largely independently distributed, often with nothing more than a sourceforge site. In the past few years branded collections of projects have emerged. The best known is the Jakarta collection at Apache. Some others are http://opensymphony.com, http://codehaus.org, and http://opensource.thoughtworks.com . The new thing emerging is commercial support and packaging into distributions "stacks" of the most popular projects.

JBoss, SourceLabs, Spikesource, Gluecode (now IBM) and others are making money out of of supporting Java open source. What you see in each case is that they only support a narrow selection of open source projects that make it into in their distributions. That the distributions are "stacks" further clarifies that there is a narrow selection at each level - often only one. They tend to go with the sweet spot: a configuration where most of the users are. The effect of this selection in turn reinforces the popularity of the selected projects.

Most users of Linux distributions or JBoss distributions stick to the projects included, because the work is done and it is easy. In the old days people would put their own together. I think we are seeing a similar change occurring in Java tools and libraries. An example of this in the Java space is http://jpackage.org/. I had an email just this weekend from someone asking about updating the ehcache packages there, because his company only used libraries that were in jpackage. I cooperated with him. As the method of distributing open source Java changes, individual projects need to support the new distribution approaches or suffer a decline in use.

A lesson in what can happen once you get dropped by a distribution is the XFree86 project. It was the standard Linux X server, a windowing system, used on millions of machines worldwide. There was a disagreement over licensing last year. Within a few months Red Hat had dropped them in favour of the X.org project. I do not know of anyone who has bothered downloading and installing XFree86 who is using an X.org system.

For those of us that write open source software, the main satisfaction is in having others use your code. The more people that use your code the greater the satisfaction. As a non-aligned open source contributor, I think there is a need to be aware of these changes in distribution to remain widely used.

Posted by gluck at 10:47 AM | Comments (0)

July 08, 2005

Have you swikked yet?

For the past 5 months I have been talking to SourceLabs about open source. They are a promising startup offering support for commonly used open source. The research I have read consistently shows lack of support as the largest inhibitor of open source adoption.

To make themselves known to the open source community, SourceLabs decided to give something back. That something is swik. Swik is a little hard to describe, thus the title of this post - have you swikked yet? It is a combination of del.icio.us, freshmeat, and an open source themed wikipedia. Its aim is to be a repository of knowledge about open source projects. Like wikipedia, anyone can edit anything. Like freshmeat, it has information on open source projects. Like del.icio.us it is a social network. The hope is it will provide some much needed meta information about projects, and be more than freshmeat and do more than sourceforge search.

A lot of thought has been put into swik. Search for a project. If it does not exist, swik's robots will go out to the usual places and discover information about it. On its own this makes using swik useful. But if others come along and add extra information then it becomes quite valuable. I encourage you to get behind swik and make it work.

Posted by gluck at 10:42 AM | Comments (0)

June 30, 2005

Google is hostile to open source, open standards and no longer cool

Update:On 7 July Google announced the Google Toolbar for the Firefox browser on Windows, Mac OS X and Linux. This move should be applauded. Now for Google video.

For some time I have been watching Google with growing concern. When they came on the scene, I was a very early user. I thought it was cool that they used Linux and thought that perhaps they would be an open source supporter. With the release of Google Video I think there is now enough evidence to conclude they are hostile to open source.

Google's (lack) of Open Source contributions

http://code.google.com/projects.html is a list of open source projects done by Google. 9 projects, none of which are well known to me. I personally maintain three projects and contribute to several others. My employer contributes to 150. I think Google's project contributions are nothing more than a mild bit of PR designed to attract open source developers. But their efforts fall flat.

Google's Use of Open Source

Google is perhaps the world's largest user of Linux. They probably get more benefit out of open source than most companies. They use Python. They use Java. Java is not open source, but you do not need to pay for it.

Windows only here

I had a colleague who used IE when the rest of used Firefox. We asked why and he pointed out that one of his tools of trade was the Google toolbar. http://toolbar.google.com/googlebar.html. We tried an open source plugin which worked fine for me. That particular version was a frustrating install so he canned Firefox. He is still on IE.

Next we have Desktop Search. Windows only.

Then, we have hello, also Windows only.

Next, Google Earth, Windows only

Picasa, Windows only

And of course, finally Video, Windows only. What really hurts about this last one is that Google are using the open source VideoLan player. (http://www.videolan.org/vlc/) I have used this player for years on my Linux and Mac machines. VLC has binaries for every operating system out there. Google is Windows only.

The death of open standards

When Netscape came out I was a Windows only user. What struck me at the time was the support they offered for other OSes. In my case this made me think that perhaps the Internet era would be one of open standards. It largely has been. I have personally embraced open standards computing. For me Netscape was the example.

Google had, and maybe still as the opportunity, to continue that fine tradition. Many of the people that work at Google have personally made serious contributions to open source and open standards. However, by making a growing number of tools Windows only, Google is perpetuating proprietary standards and locking out alternative operating systems such as Linux and Mac OS X. If this continues it will become unviable to use these alternatives. Google, the world's largest open source user, will become the largest open source killer. They will have succeeded where Microsoft failed.

Google is no longer cool, they are evil

Google has up to now been considered cool. Not anymore by me. And not anymore I suspect by a growing number of open source developers, Linux and Mac users. The founders love to go on about their ethics and their 'Do not evil' mantra. A bit like the Hippocratic oath of doctors: 'Do no harm'. Well, unless they start supporting open standards they are doing evil.

What Google needs to do

Either:

1. Release clients for Windows, Linux and Mac OS X for each product, or
2. Work with the open source community to enable interested developers to do same

What you can do

There is a web site called Linux Appeal, where you can petition Google to support Linux for Google Video. See http://www.linuxappeal.net/petition.php?function=petition_read&petition=35

Posted by gluck at 01:05 PM | Comments (7)

May 17, 2005

The World is Flat: A Book Review

I greatly enjoyed reading Thomas L Friedman's "The World is flat".

In the context of international trade and also competition, the term "level playing field" is often bandied around. A "level playing field" is one where each combatant has the same rules and no specific discrimination against them.

What happens if a lot of countries implement level playing fields? The whole world becomes a level playing field - the world becomes flat.

What happens after that, according to classical economic theory, is that the law of competitive advantage applies. If one country has a natural advantage over another in the production of a particular good or service, then they will be able to charge a lower price and production will move to that country.

The thesis of Friedman's book is that the theoretical level playing fields and competitive advantage, have become reality. He argues persuasively, citing statistics and anecdotal material from interviews with key players up to the end of 2004.

The IT industry, in particular is exemplified as a flattened industry. He portrays the Americans losing jobs to Indians, in danger of losing their jobs to the 300 million Chinese learning English. Wherever smart software engineers are found, the almost zero cost of the Internet has enabled them to compete on the playing field.

The zero Internet cost alludes to some of the key enablers of the flat world. OpenSource is cited as one. I like the way he describes OpenSource as Self-Organizing Collaborative Communities. I am beginning to think that collaborating on software that is free in a GNU sense, is almost an emergent property of the Internet.

For anyone who has lived through the IT revolution of the last 20 years, Friedman appears a little naive at times. He seems to have fallen into the clutches of Craig Mundie, CTO of Micro$oft. He rightly considers the PC a flattener. He thinks Windows was way more significant than I would ascribe. My view is, that far from being a flattener, Windows stymied innovation, and held up the emergence of the world we now have by a decade.

In the end, you have to love one of the flatteners. The date 9 September 1995. Don't remember? What started the .com boom? Still don't remember? Read the book.

Posted by gluck at 12:25 PM | Comments (0)

March 28, 2005

Subversive thoughts on Subversion

Back in March I started sing the Subversion revision control system in anger. The experience was not totally positive, thus the title of this post. I just tried to move a project to Subversion again this week. I reverted back to CVS after a day.

Main Features

Compared to CVS, my usual RCS, Subversion offers some major new features:

Atomic Commits

Commits either all succeed or all fail. This lets you avoid the scramble, trying to avoid breaking the build, when some of the files succeed and some fail.

Every Commit is a Tag

In Subversion, each time you commit, a transaction is performed against the repository and each file and directory gets a new version number.

Binary Files are First Class

Subversion knows about binary files. It can also effectively diff them. No need for -kb when you add them.

Problems

In practically using Subversion I saw many problems that are absent from CVS. I should have wrote this article earlier, because I cannot now recall the exact causes of each. While some of the points raised may be perceived as griping, I am used to CVS which is extremely reliable and mature.

Apache SSL

We got a couple of runaway processes using https to access the repository. Not sure exactly where the problem lay. We dropped back to http, which works well.

In my recent Subversion use, we had about 100MB of binaries. The server was again set up with https, this time with certificate based security. It seemed slow to import them when I set up the module. One of my colleagues waited over an hour for a checkout. After it completed we checked the server for any problems. There were no runaway processes. For some reason it was slow.

What Security? There is no security

After a successful login Subversion stores you password in plain text in the ~/.subversion/auth/ directory. Given that a lot of people (probably stupidly) tend to use the same password, or a small number of passwords, this is very dangerous. If you know someone is using Subversion, you know where there password is. I am not sure why Subersion was designed like this. The defence that the directory is accessible only by the owner, being in their home directory is extremely lame. There is now a switch to turn this off, but the default remains to store in plaintext.

Updates

One developer had a problem where he updated, got a screen full of information about updates but it had not completed. Not sure what the cause of this was. By contrast CVS will always update.

Checksum Error

Under some circumstances, you will get a checksum error. At this point you cannot do anything but delete the file. The circumstances are: abrupt termination of the Subversion client, or a buggy subversion client. IntelliJ Irida, still in development, does this. svn cleanup does not fix the problem.

Subversion gets confused

On about three occasions, Subversion got what I can only call confused. It refused to commit or revert. These situations invariably involved deleting the directory tree in the working copy and doing an update. svn cleanup does not seem to help. Nor does svn revert.

Server Corruption

One file got corrupted on the Subversion server and had to be recreated.

IntelliJ Support

IntelliJ will fully support Subversion in version 5, codenamed Irida. It is useable now for diffs and commit browsing. Do not try to perform any adds or commits however or you will get the dreaded checksum error. Update: The new IntelliJ Irida version of IDEA supports Subversion. Given that they are getting close to release, I decided to check it out. Unfortunately a refactor involving moving files to a new package totally screwed up. Back in CVS it worked fine.

Recommendation

When I originally wrote this post, the number and type of problems we experienced suggest to me that Subversion is not ready for primetime just yet. Three months later, and with my reliance on IntelliJ IDEA, my opinion remains the same. UpdateI tested IntelliJ 5 last week. It;s Subversion support is now very good. Note that they do not suport the file: protocol for local repositories as yet. Other than that no problems. I think Subversion is now ready for a look.

Posted by gluck at 04:10 PM | Comments (7)

February 02, 2005

OSDL Conference Key Thoughts: Linus Torvalds, Andrew Morton, Mitch Kapor, Brian Behlendorf

On Security

Brian Behlendorf thinks that computer systems are going to be understood in far more biological terms in the future. We talk about viruses and anti-viruses. How about vaccines? He points out that many viruses and worms are not harmful but they could be. In this way they are like vaccines, where an infectious agent spreads and causes an immune response. Perhaps the best way to provide security is to send out harmless but annoying viruses to keep the computing immune system well-primed.

Linus thinks if you are paranoid about security, as one foreign defense force person in the audience asking the question was, you would feel safer examining the code for trojans. While a lot of work, in open source it is always possible, while with proprietary systems, it is not always. Also, in terms of stupidity in coding he points out that a lot of corners are cut where the developer knows no one can see the code. In open source "if you see someone has a comment that it does  not work in a particular case, you ask yourself What drugs is he on?".

On the Communities Surrounding Open Source

Brian says Apache does not have leads. Decisions are made by concensus. One nice thing is the graceful gradient of contribution: user -> expert user -> bug submission -> patches -> committer -> maintainer. This makes it easy to move up.

Mitch wonders about the right level of control. Too open and you get a babble; anarchy. Too closed and you turn contributions away. The Firefox leads think they have been too tight. For the last two years they have not admitted anyone to their group. A key is to decouple systems. This allows parallel development. The project lead should have good judgement more than anything else. What to leave out and what to leave alone.

Andew Morton manages the 2.6 kernel through 50 maintainers of each subsystem. Repository management becomes a key skill.

Open Standards and Open Source

Brian believes that they are two sides of the same coin. He points out that Apache was formed to implement a server for HTTP.

Linus thinks that some companies pay lip service to standards but deliberately seek to add proprietary features when implementing standards. These companies are not motivated to improve the standards. He thinks all open standards should demand an open source reference implementation. Standards are not always set by standards bodies. Open source projects can become defacto standards. These are accepted by the public as standards, because no one can take the implementation away, so it is always safe to build on.

On Software Patents

Linus has thought this has been on the open source radar for the past 5 years. It is bad for open source. It is just as bad for proprietary software. He despairs at the special interests in Europe who keep trying to get software patents in. A light at the end of the tunnel is the patent grants given by IBM, Sun and others. He thinks many other companies will follow suit, but not Microsoft.

Mitch Kapor wrote a paper in 1990 entitled "Why patents are bad for software". He goes way back. The big problem is that tens of thousands of bad patents have been issued. 15 years ago the Patent Office changed policy. Until then they checked for prior art and the other rules. Then they decided to grant anything and let it be fought out in the courts. This could create a future Patenet Bhopal, with toxic stockpiles of patents waiting to explode. He thinks that Microsoft's last stand will be to use their toxic stockpile as a Patent WMD, a weapon of mass destruction. He says we need patent reform. Which will happen first? He does not know.

5 Years From Now

Linus does not predict the future. He says those who look into the distance at Utopia stumble on the rock in front of them. He looks at the rocks. He wants to fix things today, and then fix things tomorrow and so on. He says he is an anti-visionary.
He has no visions.

Andrew sees open source as inevitable.

Mitch sees irreistable force in open source meeting immovable objects. He thinks interesting things will  happen. He cites Wikipedia as an example of what can happen with decentralised self assembling systems. An invaluable resource created far more cheaply than their closed source counterparts.

Brian sees open source as an emergent property of the Internet itself. The software lives in an ecosystem where many parallel experiments with different forms take place concurrently.  This causes super-fast evolution and survival of the fittest. He likens closed source to the failed centrally planned economies of the communist countries. The market won. Microsoft has criticised the lack of a central authority to take responsibility. Brian asks "Does Microsoft see out lack of a single point of failure as a problem?"

The Nature and Culture of Open Source Developers

Linus thinks that cooperation is not motivational. Its more fun to compete. Lets not cooperate too much. Bad coders do not get their contributions accepted. Good programmers do. Good programmers are almost never politicians. It is insufficient to want a career path to do open source. You need another motivation. You do it because you would do it anyway.

Andrew thinks that politics is dealt with by developers getting subverted. They end up more loyal to the open source project than to their companies.

Mitch notes that most open source developers are in the US and Europe. (He seemed to miss the fact that Andrew Morton, an Australian was sitting next to  him and that per capita, Australia contributes the most to  open source.) Developers from other countries use open source but do not contribute as much. Maybe language? Maybe culture? Brian avoids developers who want to make a pot of gold. He thinks developers over 30 are likely to recognise that they need enough for a home and to care for their children. From their it is about feeling good about what you do, about contributing, seeing your code live and not be yanked out a week before release by a "deranged Venture Capitalist".

Brian thinks open source developers have to be good communicators to succeed. They need to be able to debate and defend themeselves. They also need good judgement. An open source programmer will usually be a good hire.

Personal Observations on the Panel

The panel was an amazing assembly of open source talent. Looking at them I came up with some descriptions:

Linus Torvalds - Glowing with goodness, happiness and calmness. He projects it like the Dalai Lama.
Andew Morton - The dour realist. Linus' right hand man as he calls him.
Mitch Kapor - The wise old owl. He is really from another time, which gives him perspective on the current time. Though look out for Chandler, a PIM. He is cooking up a killer app. Also check out the new calendaring standard CalDav.
Brian Behlendorf - He reminds me of a friend of mine. He seems to be the archetypical Californian (not sure if he is), from pony tail  to Zen views. The biological perspective he has is very appealing.

Posted by gluck at 04:54 PM | Comments (0)

MSN Test Drive: "java sucks" vs ".net sucks"

The first question on my mind about MSN's new search engine is whether it provides an unbiased view of the Internet. Judge for yourself

"java sucks"

Google: 410,000
MSN: 846,373

".net sucks"

Google: 3,000,000
MSN: 245,041

Posted by gluck at 02:13 PM | Comments (3)

February 01, 2005

Report from Day One of the OSDL Linux Enterprise Summit

I am attending the OSDL Linux Enterprise Summit in Burlingame, just south of San Francisco. Interestingly a good proportion of the attendees are corporte lawyers. The focus right now is on getting the law right to avoid future difficulties. Following are some highlights from day one.

Open Source Licensing, Larry Rosen

1. Used to be mainly about copyright, but is now needing to deal with patents as well.
2. The differences of Composite versus Derivative works and the exact legal meaning of many others terms are being defined. Their is a lack of case law (a good thing!) which makes it a challenge.
3. Larry has a book called Open Source Licensing. He is hoping Justice Scalea of the Supreme Court reads it when considering open source.

Reviewing Use of OSS in the Enterprise, Karen Copenhaven>

1. Most organisations have a contingent liability due to breach of open source licenses.
2. Session mainly on the need for and conduct of a Legal Review process .e.g. Product contains OSS with breach who in turn distributes that to another customer, thus propogating the breach
3. Potentially a big threat to OSS use, because it may be easier to just say no to OSS. In reality most organisations are spinning up their lawyers instead.
4. Commercial software companies are starting to offer indemnities. 5 years ago no one did. HP is an example of providing indemnities for Linux.
5. A common approach is to get the lawyers to review a project and put it on an approved list. It is not sufficient to have a list of approved licenses. The pedigree of the source must be examined.
6. Black Duck makes tools to automate open source review.

Andrew Morton

I ran into Andrew who I have met before. We talked about adoption of the 2.6 kernel. He is the maintainer of 2.6. He moved over to the US from Wollongong, Australia about 4 years ago and is doing some work for the OSDL for his employer Digio. His quote, which is the quote of the day - "Open source is inevitable".

Novell

Plenty of Novell people were on hand today. The head of server marketing thinks that 20% worldwide desktop use is achievable in the next 5 years. Currently it is at 3.5% but just a few years ago it was barely measurable.

Posted by gluck at 02:40 PM | Comments (0)

January 22, 2005

An open source key value: Standards over Products

What values does an organisation require to enable the use of open source software? A utopian view of the world. A board of ex-communists? I have been thinking about this recently in the context of the company I work for. We have great open source skills. We are looking for the subtle cultural things that will complete our open source orientation.

Though it does not mention open source, I think a key enabling value is Standards over Products.

When you have a standard, such as a published protocol or specification, the way is open to have multiple implementations. Often you will have commercial implementations (products) and open source implementations. When there is no standard, it is often very difficult to do an open source implementation. By supporting standards over products, it enables a competitive software selection environment. The implementation chosen can then rest on the quality and features of each.

An example is Rules Engines. In August 2004, JSR-94 was finalised. There are several implementations available. ILOG JRules is a high quality commercial product. drools and JESS are open source implementations. Lets say you propose an architecture which includes a JSR-94 compliant rules engine. You then enable the open source implementations to compete with the commercial products.

The value provides a level playing field and engenders fierce competition between all implementations, open source or commercial products. The decision to use open source or not, is then taken on merit.

Posted by gluck at 02:14 AM | Comments (3)