Iterate Performance

      * With iterator:
      * 1.6 with 100,000 store size: puts take 45ms. keySet 7ms
      * 1.6 with 1000,000 store size: puts take 381ms. keySet 7ms
      * 1,000,000 - using FastRandom (j.u.Random was dog slow)
      * INFO: Average Get Time for 2065131 observations: 0.013553619 ms
      * INFO: Average Put Time for 46404 obervations: 0.1605034 ms
      * INFO: Average Remove Time for 20515 obervations: 0.1515964 ms
      * INFO: Average Remove All Time for 0 observations: NaN ms
      * INFO: Average keySet Time for 198 observations: 0.0 ms
      * 9999 - using iterator
      * INFO: Average Get Time for 4305030 observations: 0.006000423 ms
      * INFO: Average Put Time for 3216 obervations: 0.92008704 ms
      * INFO: Average Remove Time for 5294 obervations: 0.048545524 ms
      * INFO: Average Remove All Time for 0 observations: NaN ms
      * INFO: Average keySet Time for 147342 observations: 0.5606073 ms
      * 10001 - using FastRandom
      * INFO: Average Get Time for 4815249 observations: 0.005541354 ms
      * INFO: Average Put Time for 5186 obervations: 0.49826455 ms
      * INFO: Average Remove Time for 129163 obervations: 0.015120429 ms
      * INFO: Average Remove All Time for 0 observations: NaN ms
      * INFO: Average keySet Time for 177342 observations: 0.500733 ms
      * 4999 - using iterator
      * INFO: Average Get Time for 4317409 observations: 0.0061599445 ms
      * INFO: Average Put Time for 2708 obervations: 1.0768094 ms
      * INFO: Average Remove Time for 17664 obervations: 0.11713089 ms
      * INFO: Average Remove All Time for 0 observations: NaN ms
      * INFO: Average keySet Time for 321180 observations: 0.26723954 ms
      * 5001 - using FastRandom
      * INFO: Average Get Time for 3203904 observations: 0.0053447294 ms
      * INFO: Average Put Time for 152905 obervations: 0.056616854 ms
      * INFO: Average Remove Time for 737289 obervations: 0.008854059 ms
      * INFO: Average Remove All Time for 0 observations: NaN ms
      * INFO: Average keySet Time for 272898 observations: 0.3118601 ms

As a result I have used an alternative to iteration using an algorithm called FastRandom. The result is 0.16 ms, 2,300 times faster!

For very small maps, ConcurrentHashMap iteration is quite quick. From experimental testing in Ehcache 1.6 we use iteration up to 5000 entries and FastRandom for sizes above that.

size() performance

Recommendations

To give ConcurrentHashMap the best chance of optimisation remember to set the size and expected concurrency when you create it. In ehcache we set the size to the exact size configured for the cache, and we set concurrency to 100 threads. ]]> 2009-06-19T00:34:42Z 2009-06-17T03:02:02Z tag:gregluck.com,2009:/blog/1.172 2009-06-17T03:02:02Z

80MB heap limit

Crashed sites do not recover immediately or start a new instance.

while true; do wget "http://gregrluckapphelloworld.appspot.com/"; sleep 30;  done;

Static content is not distributed through the Google CDN

HTTP/1.0 200 OK
  Date: Tue, 16 Jun 2009 09:47:01 GMT
  Expires: Tue, 16 Jun 2009 09:57:01 GMT
  Cache-Control: public, max-age=600
  Content-Type: image/jpeg
  Server: Google Frontend
  Content-Length: 237952
  Connection: Keep-Alive

Conclusion

Why use Ehcache with Google App Engine?

Recipes

Setting up Ehcache as a local cache in front of memcacheg

Using memcacheg in place of a DiskStore

Distributed Caching

Dynamic Web Content Caching

Getting Started

In a file named ExampleScalaGet.scala:

   import java.net.URL
   import scala.io.Source.fromInputStream

   object ExampleScalaGet extends Application {
     val url = new URL("http://localhost:8080/ehcache/rest/sampleCache2/2")
     fromInputStream(url.openStream).getLines.foreach(print)
   }

run it with:

scala -e ExampleScalaGet

The program outputs:

 <?xml version="1.0"?>
 <oldjoke>
 <burns>Say <quote>goodnight</quote>,
 Gracie.</burns>
 <allen><quote>Goodnight, 
 Gracie.</quote></allen>
 <applause/>

If you are on a Mac the easiest way to get scale installed is "sudo port install scala".

The Scala program ways in at 6 lines. Now, let's compare it's tersity with the other languages, all doing a GET and printing the result (See http://ehcache.sourceforge.net/documentation/cache_server.html for the examples). Whitespace lines are not included.

Scala 6 lines, using java.net.URL
PHP 6 lines, using curl lib
Python 3 lines, using urllib2
Ruby 6 lines, using open-uri
Java 19 lines, using java.net.HttpUrlConnection, which does a few things before it gives the InputStream as used in the Scala example.

Thanks to Brad for rounding out the examples.

Scala looks very interesting. Comparing it with the others:

Brad has also blogged about his Scala example here: http://bradclow.blogspot.com/2009/01/scala-example-for-accessing-ehcache.html

You are most welcome to join what has become an informative forum thread here: https://sourceforge.net/forum/forum.php?thread_id=2546225&forum_id=322278

So far we have posts from myself, Jim Webber, Brett Dargan and others interested in creating or finding a REST convention for referring to all and specifying means of multi-get, multi-put and multi-delete.

Kohsuke Kowaguchi joined the project and copied his code in and released it. His focus was V3 Embedded. It supported one goal: run.

There was disagreement as to the features and the code to use. Dave's plugin was to support a wide range of goals supporting integration of V2 and above into the build process.

Now to use the convenience name you normally add a pluginGroup:

The end result is that we have two plugins called maven-glassfish-plugin which are different, but because they use the special naming of maven-name-plugin, both are invoked with mvn glassfish:goal, causing a namespace conflict.

Now when I do mvn glassfish:run I get:

 mvn glassfish:run
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'glassfish'.
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Required goal not found: glassfish:run in org.glassfish.maven.plugin:maven-glassfish-plugin:2.1
[INFO] ------------------------------------------------------------------------

mvn org.glassfish:maven-glassfish-plugin:run

It would be nice for one of Kohsuke, Dave or Byron to sort this out.

My suggestion is for Kohsuke to rename his to maven-glassfish-embedded-plugin.

Ancestry

Mitochondrial & Y DNA

Compared With

Susceptibilities

Well, Brett Dargan, a colleague, and I had our own version of this over the past two days. We came down to Sydney to run an Architecture Away Day. All went well. We were sitting at the Swiss Grand Resort and Spa looking out over Bondi beach, knocking down beers with the attendees after the event about 5:30pm when the waiter (server for my US readers) pointed out a big storm brewing to the South. I felt the first unease.

Then a few minutes later I received an ominous SMS from Virgin Blue to tell me that my 7pm flight had been delayed. I rang up Customer Service, but of course several hundred other people were trying to do the same thing, so I gave up after spending 6 minutes in the queue. More beers. Brett then rang up 15 minutes later and got through. We were now on a 9pm flight. So, off to the Cricketer's Arms Hotel in Surrey with Nugget for Tapas and more beers.

Having a good time at the Cricketers Arms. Catching up with some former colleagues and enjoying some great Tapas: Chorizo, Lamb Kofta and Grilled Haloumi, washed down with Pure Blonde low carb beer. Mike Edwards, a fellow drinker mentioned "Great lightning show out there". Sure enough the storm that had been brewing was now coming close and looking bad. So I checked http://bom.gov.au and saw that there was a severe weather warning for Sydney over the next few hours, with heavy rain and damaging winds.

At this point I was beginning to think that it could be a bad night. By now it was time to go to the airport, so we said our goodbyes and went outside - right into a massive rain downpour. We managed to flag a taxi who, when we said the airport for our destination, said no, then drove 20 metres down the road, and then waved to us to get in. The street was a river by now, with 5 cm of water over the asphalt. We both got soaked to our ankles. From there we had an uneventful ride to the airport.

All went well until we got on our plane. We sat and waited, and waited, and waited. Then the steward, who was already looking well harried, made an announcement that a Flight Instrument was broken, and engineers were trying to fix it. This went on for half an hour, after which were deplaned. More waiting at the gate and then it was announced they had a plane for us at the opposite end of the airport at the JetStar terminal. We all trudged through an airport whose shops were closing up down there and eventually got on another plane. For non regular Sydney visitors, note that Sydney has a flight curfew of 11pm. Anyway the we got on, the baggage got loaded and were taxied out and waited to access the runway. And waited. And waited.

By now it was 10:30pm. Then the harried steward came on the Intercom and said that we were waiting for some other flights to land (I guess they take precedence when you get close to the curfew) before we could take off. He mentioned that we would be cutting the curfew close. After that we had regular updates every 5 minutes, as we moved inexorably towards the curfew time.

Now it was 11:02 pm. It was not looking good. Then the captain came on with encouraging news: we were now after curfew, but we had a flight clearance and could take off on the East-West runway out over the ocean. We were waiting for a few other planes.

At this point we allowed to move around the cabin. The harried steward kindly explained that the snafu had started at 3:30pm while the weather was still fine. He blamed short staffed Sydney air traffic controllers.

A few minutes later he came back on to say the cross-winds were too high, and we, along with several other 737s were waiting for them to abate. The larger planes were getting away.

You can guess what happened next. The captain announced the winds were not abating and we were going back to the gate, just as soon as a gate was available, because of course most of the airport and Virgin Blue staff had gone home. 10 minutes later we deplaned into a enclosed gangway. One problem: the door to the airport was locked. We sat in there like rats in a trap for a quarter hour before we were finally let out.

Back at the gate we sat around and waited, and waited, while Virgin Blue figure out what we all going to do. In the end they announced we would need to sleep at the airport, because "the delay was weather related". I guess it was apart from the broken plane which caused our particular problem, which was in their control, and the airport issues which were not.

At this point Brett and I ran for a taxi to try and get a local hotel. Three hotels later I realised that other flights had already been bounced by the curfew ahead of us. We ended up back in the city at the Grace Hotel. The Night Auditor was checking us, and a horde of other stranded travelers in. I imagine this scene was being played out all across the city.

So, as I write this entry, I am sitting at Gate 39 at Sydney airport for a 12 noon flight. It was just announced over the intercom that our flight would be delayed due to our cabin crew not having yet arrived...

Exit Reality is Windows only. If you are on a Mac, http://q.exitreality.com/relaunch.html will tell you

"Mac users are able to experience full hardware accelerated 3D by running Windows with either:
(using OpenGL) or BootCamp (using Direct3D)"

In other words, Mac users should run Windows!

Insultingly, they then go on to say on their "Platform page"

"ExitReality 3D platform has been built in open standards and complies with International Standards Organisation regulations (ISO) and W3C. It is a free 3D platform that anyone can use to set up their own virtual world on their website.

The ExitReality 3D platform has been developed for all internet users. Any web page can be viewed and interacted with in 3D, and anyone can create their own virtual world using ExitReality."

Any platform means Windows and anyone means a Windows user. I can only say they are in a Different Reality.

I haven't been this annoyed for years. Almost enough to lodge a False and Misleading Advertising Complaint with the ACCC.

package samples;

import java.io.InputStream;
import java.io.OutputStream;
import java.net.HttpURLConnection;
import java.net.URL;

/**
 * A simple example Java client which uses the built-in java.net.URLConnection.
 *
 * @author BryantR
 * @author Greg Luck
 */
public class ExampleJavaClient {

    private static String TABLE_COLUMN_BASE =
            "http://localhost:8080/ehcache/rest/tableColumn";
    private static String TABLE_COLUMN_ELEMENT =
            "http://localhost:8080/ehcache/rest/tableColumn/1";

    /**
     * Creates a new instance of EHCacheREST
     */
    public ExampleJavaClient() {
    }

    public static void main(String[] args) {
        URL url;
        HttpURLConnection connection = null;
        InputStream is = null;
        OutputStream os = null;
        int result = 0;
        try {
            //create cache
            URL u = new URL(TABLE_COLUMN_BASE);
            HttpURLConnection urlConnection = (HttpURLConnection) u.openConnection();
            urlConnection.setRequestMethod("PUT");

            int status = urlConnection.getResponseCode();
            System.out.println("Status: " + status);
            urlConnection.disconnect();

            //get cache
            url = new URL(TABLE_COLUMN_BASE);
            connection = (HttpURLConnection) url.openConnection();
            connection.setRequestMethod("GET");
            connection.connect();
            is = connection.getInputStream();
            byte[] response1 = new byte[4096];
            result = is.read(response1);
            while (result != -1) {
                System.out.write(response1, 0, result);
                result = is.read(response1);
            }
            if (is != null) try {
                is.close();
            } catch (Exception ignore) {
            }
            System.out.println("reading cache: " + connection.getResponseCode()
                    + " " + connection.getResponseMessage());
            if (connection != null) connection.disconnect();

            //create entry
            url = new URL(TABLE_COLUMN_ELEMENT);
            connection = (HttpURLConnection) url.openConnection();
            connection.setRequestProperty("Content-Type", "text/plain");
            connection.setDoOutput(true);
            connection.setRequestMethod("PUT");
            connection.connect();
            String sampleData = "ehcache is way cool!!!";
            byte[] sampleBytes = sampleData.getBytes();
            os = connection.getOutputStream();
            os.write(sampleBytes, 0, sampleBytes.length);
            os.flush();
            System.out.println("result=" + result);
            System.out.println("creating entry: " + connection.getResponseCode()
                    + " " + connection.getResponseMessage());
            if (connection != null) connection.disconnect();

            //get entry
            url = new URL(TABLE_COLUMN_ELEMENT);
            connection = (HttpURLConnection) url.openConnection();
            connection.setRequestMethod("GET");
            connection.connect();
            is = connection.getInputStream();
            byte[] response2 = new byte[4096];
            result = is.read(response2);
            while (result != -1) {
                System.out.write(response2, 0, result);
                result = is.read(response2);
            }
            if (is != null) try {
                is.close();
            } catch (Exception ignore) {
            }
            System.out.println("reading entry: " + connection.getResponseCode()
                    + " " + connection.getResponseMessage());
            if (connection != null) connection.disconnect();
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            if (os != null) try {
                os.close();
            } catch (Exception ignore) {
            }
            if (is != null) try {
                is.close();
            } catch (Exception ignore) {
            }
            if (connection != null) connection.disconnect();
        }
    }
}