Gentlemen, you can't fight in here! This is the War Room!

The image tile servlet is more or less complete, and I can now create tiles from images of any size (or so I believe) without using lots of memory. As I mentioned in my previous post, the largest image I'm currently testing with is 8192*9976*8 bits. I'll dig up some even larger ones soon and do some further testing.

The servlet now works as follows: The Google Maps image viewer figures out which tiles it needs based on the state of the view window, and makes HTTP GET-requests, like so: imageTileServlet?obsId=6&x=0&y=2&z=5

For each request, the image tile servlet should return a 256x256 pixel JPEG image. It looks up the obs with the given id, and opens the file given by its "complex" value. The servlet then reads metadata from the file (dimensions and bit depth). Based on this and the GET parameters, the servlet can calculate the relevant part of the image and set the correct image read parameters, like so:

ImageReadParam param = reader.getDefaultReadParam();
param.setSourceRegion(new Rectangle(xpos, ypos, tileSize, tileSize);
param.setSourceSubsampling(subsampleLevel, subsampleLevel, 0, 0);

SourceRegion is the part of the image we are interested in decoding. SourceSubsampling allows the reader to skip pixels while reading, if set to 2 the reader will read every other row and column.

The servlet will then resize the image using a Graphics2D object, encode to JPEG and write the resulting stream to the HttpServletResponse object. The user's web browser will in most cases cache the images, so each tile shouldn't need to be requested more than once.

So far the source image must be a JPEG file, but adding PNG, GIF and BMP is a simple matter, as Java's Image I/O API has built-in support for these formats. I also plan to add DICOM image decoding at a later time.

Right now the tile servlet is a little sluggish. I have my test deployment of OpenMRS running on a Pentium III laptop with Ubuntu server. Loading an image tile from my largest JPEG takes several hundred milliseconds. Smaller images are read significantly faster.

I have created a portlet that contains the Google Maps-based image viewer. I'm currently ironing out a couple of JavaScript and CSS issues, which I expect to resolve soon. As the svn server is supposedly up and running, I'll probably make my first svn commit when this is completed, though I may do a little cleaning up and refactoring first.

I also believe it is time to update the wiki page for my project, I'll probably get that done within a couple of days. Next up is support for annotations.

 

Now, with all that out of the way, let's talk about something really interesting: Amusing facts from WWII.

During the war, the British developed a new class of bomb to use against bunkers and submarine pens, the so-called earthquake bomb. The design was built in two sizes: The 5400 kg 'Tallboy' and the 10000 kg 'Grand Slam'. The Grand Slam was the world's largest bomb at the time. These weapons were dropped from specially modified Lancaster bombers and reached supersonic speeds before hitting the ground. The bomb casings allowed the weapons to penetrate several meters of soil and concrete before detonating their massive load of torpex high explosive.

At RAF Scampton, a Royal Air Force base north of the English town of Lincoln, a Lancaster bomber and a Grand Slam bomb were set up as "gate guards" after the war. In 1958, the road going past was going to be extended and the gate guards had to be moved. They rolled out a crane and tried to lift the bomb, which for some reason wouldn't budge.

The RAF crew initially suspected that the bomb casing was filled with concrete, or had filled up with water over the years, but then someone got a horrible suspicion. "No... It couldn't be... could it?" This area had been open to the public, and a lot of people had climbed up on the bomb and taken pictures sitting astride.

So, they called up a technician who carefully scraped off the paint and opened a service hatch. Right enough, the Grand Slam bomb was still filled with explosives from 1944!

A bigger crane was called in and the bomb was carefully moved under police escort to the coastal experimental range at Shoeburyness. There it was rigged for demolition, and the resulting explosion proved without a doubt to anyone within a 16 km radius that the bomb was still very much alive.

Carefully does it!

The subsequent investigations failed to determine how a live earthquake bomb could have been set up as a gate guard. They crunched some numbers, and estimated the damage the bomb could have done if detonated at its location outside the RAF base. Besides taking out the base itself, the bomb might have flattened most of the northern part of Lincoln, including the cathedral dating back to 1250.

In a world gone mad, only a lunatic is truly insane.

I've been busy with a few things during the previous couple of weeks, mainly moving to a new flat in Trondheim. I've gotten in a few hours of work here and there though, and I'm currently back to working full time.

My main task at the moment is implementing a servlet that loads an image file from the 'complex obs' system, carves it into tiles and resizes it. My first version works very well with smaller images, but fails miserably on large ones because Tomcat runs out of memory. For example, one of the images I use for testing is an X-ray image, a 8192x9976 pixel grayscale jpeg. 8192*9976*8 bits = 78 MB of memory required to store the whole decoded image. Multiply with 3 for a 24-bit colour image.

This can be reduced significantly though, using built-in functionality in Java's Image I/O API. Setting the right Image Read Parameters allows me to only decode the portion of the image that I actually need, as well as applying subsampling to the image decoder. Seeing as each http request returns an image tile of only 256x256 pixels, there is huge room for improvement.

I enjoy listening to BBC World Service radio, and todays episode of 'Digital Planet' features a brief discussion of OpenMRS and the recent Durban conference. An mp3 of the show can be downloaded from the show's home page. (WMA/RealMedia stream, or mp3 download at the podcast link)

In other news, my mentor Mike has spoken with Summer of Code organiser Leslie Hawthorn at the aforementioned conference, and she says we are good to go ahead with using Google Maps API. Hurrah!

Terms of (non)service

The previous week I ran into an unexpected problem: It turns out that the Google Maps API Terms of Service may prevent me from using the API in my summer of code project. This despite of the fact that I only use the map controls, and not the map data. The problematic clause is the following:

The API may be used only for services that are generally accessible to consumers without charge. Accordingly, You may not use the API for any service that requires a subscription or other restricted access, or for which a fee is charged.

An OpenMRS deployer must necessarily restrict access to the patient data.

Fortunately, I can see light at the end of the tunnel. (Though it might be from an oncoming train) I have been spending some time looking into alternatives in case I have to drop Google Maps completely. OpenLayers, for example, looks promising. It doesn't seem as polished as Google Maps, but it should work.

Because of all this I have put the image viewer itself on hold, and have instead focused on the back-end parts of the project. I've been looking at bmckown's complex obs system, and am now running the complex obs build of OpenMRS locally. I've written a simple module that tries to interact with the complex obs system, and I've almost completed a preliminary implementation of an image tile carver, (resizing images and carving them into 256x256 pixel squares upon request) implemented as a ComplexObsHandler.

First test

The current school semester is over, and I've finally started properly on my GSoC project. I've been exploring the Google Maps API, its possibilities and limitations. Yesterday I made a functioning mockup showing how an image viewer might work. Right now only zoom and pan works, but I'm currently reading up on Google Maps' markers and info windows that presumably may be used to implement image annotation.

Note: The mockup is hosted on my own webserver in my flat. I won't be posting any links to it here, because I have limited bandwidth.

OLPC XO-1

A buddy of mine bought an OLPC through the "give one get one" program. Unfortunately, it appeared to have died a few weeks ago. Lucky for me though... I offered to try to fix it. Lo and behold: After a few minutes of work it was alive and well, and my friend wants to lend me the XO during the summer. The XO has some cool features, like it's low weight, very low power consumption and epaper-like daylight readable display. I'll be coding out in the sun for sure!

It's official: I'm participating in Summer of Code

Yesterday I was notified that my project application got accepted into Google Summer of  Code. I can't wait to begin! I'll be working for OpenMRS, a web based medical record system primarily focused on developing countries. During the summer I will make a medical image viewer with several features for studying and annotating images, typically X-ray photos.

My original application proposed to implement the image viewer as a Java applet. The plans have since been revised, and the plan now calls for an AJAX web app instead. I will be using the Google Web Toolkit (GWT) and  the Google Maps API during development, something I'm really looking forward to.

The change from Java applet to AJAX will lead to much better performance in low-bandwith scenarios. It will also move much of the image processing workload from the client side to the server, enabling better performance on low-end hardware but probably stressing the server somewhat more.

I will keep this blog updated with my progress during the summer, posting at least once a week.