Performance Benchmark Results: JVM vs CLR

30 January, 2005 Greg Luck 8 comments

I was talking with someone the other day about Java vs .NET performance. I have never thought that much about it. He said that .NET was fast because it had a Just In Time compiler (“JIT”). I choked on my coffee. Java’s JVM has had a JIT for about 6 years now. Not only that, but I had never heard that .net was faster.
So, I decided to go and run some benchmarks for myself. Rather than dirty my machine by installing Windows, I fired up VMWare and installed it on a disposable file. I then started the arduous and frustrating experience of installing .net onto my VMWare pack of XP. Reading the EULA, I discovered that Microsoft had already foreseen the threat of benchmarks and banned disclosure of any benchmark information without written permission from them.
So, If I were to tell you that Java JVM based on JDK1.5 was 42% faster than .net’s CLR, based on a benchmark I had, that would be breaching the EULA. In reality I cannot even get started because of the mind boggling activation process required. Currently my 60 day trial activation key has been used too many times and my Visual Studio is disabled. I am enjoying the experience already!
There are some braver souls than me around. See http://www.shudo.net/jit/perf/. There benchmarks are current. They were last updated in December. They show that on SciMark, Java scored 324 vs .NET of 240. On Linpack, Java scored 413 vs .net’s 282. Adding the scores together gives 737 for Java and 522 for .net.
On their site there is a lot of detail and a lot of different versions. Check it out.
So, in complete agreement with the EULA, I can tell you that Java’s VM is 42% faster than .net’s CLR.
And the next time someone claims that .net is faster because it “has a JIT compiler” point them at the site.

Java none

Comments

Written by Look again about 8 years ago.

You need to look again. The number your cite aren’t quite right, and your math is wrong — 240 + 282 != 695. It’s 522, making JDK 1.5 41% faster than .net on the combine mflops.
Of course, you can’t really add mflops like you did, but the numbers do indicate that for math, the JDK is about 40% faster.
Written by Robert Watkins about 8 years ago.

Actually, the real reason why the CLR is “faster” than the JVM (for certain tasks, of course) is that Microsoft push a lot more tasks down to the native infrastructure that Windows provides. In other words, it does a lot less inside the CLR than Java does inside the JVM.
While Microsoft focus on a single platform for the CLR, this is a logical choice for performance improvements.
Written by Sam Pullara about 8 years ago.

Java: 324+413 = 737
.NET: 240+282= 522
Written by Dafydd Rees about 8 years ago.

This is meaningless. Are they saying that the CLR is faster executing the same sequence of instructions on the same operating system as the JVM? Surely they don’t have the same instruction set so it’s impossible to compare “speed” directly. It’s just like comparing the “speed” of an Intel chipset with, say a G5. Clock speed alone means nothing – depends on what instructions are executing during each “tick” and efficiently they were compiled from the source code. If we could comare differently specified VMs for speed, I wonder if Cincom Smalltalk VM would beat both the CLR and the JVM. They’ve been doing this a lot longer than Sun or Microsoft.
Written by Me about 8 years ago.

You used VMWare to run the benchmarks? There goes your credibility! Did you run both the Java and C# code on the same stack?
Written by Greg Luck about 8 years ago.

Note that I did not run any benchmarks. That is a breach of Microsoft’s EULA. I am reporting performance from another website.
Written by Steven Shaw about 8 years ago.

Perhaps the .NET proponent was just ignorant of the state of the art on the Java platform. But it’s possible that he was having trouble communication about how on the CLR, the JIT is always used to produce native code. MSIL is never interpretted by the CLR whereas on the JVM, bytecode is often interpreted until hotspot identifies a method as requiring jitting.
Also worth keeping in mind that the Microsoft JVM was the first to include a JIT, IIRC.
Written by Terry about 8 years ago.

Actually the .net results for the SciMark 2.0 Bench mark are out dated, and use a 1.0 version of .net.
Out of curiosity, I ran the results on an XP box, using my 1.5 Java JRE, and .net 1.1, and .net 2.0 beta two, and here are the composite scores.
Java 1.5: 230
.Net 2.0 C#: 356
.Net 2.0 C: 616
.Net 1.1 C#: 340
.Net 1.1 C: 575
Note for the Java time, I just ran the bench mark off the web, so it may have incurred some
browser over head.
Personally, math oriented tests such as these can be misleading, as the two libraries can have such a wide variety of implementation, accuracy ect. Its not always a valid comparison. I’d love to see the see the SPEC JVM98 benchmark ported to C#.
I’ll admit to being a .net fan, as well as Java, but I find .net to be generally 5-15% more performant then Java, which makes sense to me considering the CLR is designed specially for the windows platform, where as Java needs to provide cross platform support. I’m not sure what Sam’s point is. It seems valid to ask, you have a box, what runs similar tasks faster on it, the clr, or the jvm? I would be curious to see if java would have better performance ruining on the same physical box with Linux, instead of xp.

Greg Luck's Blog

Performance Benchmark Results: JVM vs CLR

Comments

Recent Posts

Browse by Tags

Categories

Archives

Meta