Liftweb 1.1 and Google AppEngine

With only minor modifications, I have repeated Joe Kutner's installation of Liftweb on AppEngine. I'm running OS/X, so I started by using macports to install maven2. I saw that Java App-Engine was just updated to 1.3.1, so that was installed and I adjusted my APPENGINE_HOME and M2_HOME in .profile. The latest liftweb version is 1.1-M8, so I'll try that one.

Now that I know I have updated libraries, I generated a liftweb project with:

mvn archetype:generate -U -DarchetypeGroupId=net.liftweb

-DarchetypeArtifactId=lift-archetype-blank -DarchetypeVersion=1.1-M8

-DremoteRepositories=http://scala-tools.org/repo-releases

-DgroupId=com.folkertsfotografie. rlftest -DartifactId= rlftest

Rather than using -Dversion=1.0-SNAPSHOT, I entered a version of 1.1-M8 when prompted in maven. As usual, then next commands were cd rlftestl and a quick mvn jetty:run to see that http://localhost:8080/ was working. As an eclipse user, I added mvn eclipse:eclipse. So far, so good. The simple helloword application came up normally under the jetty installed with liftweb.

To integrate with AppEngine, I just added the boilerplate appengine-web.xml in my WEB-INF directory. The contents are:

<?xml version="1.0" encoding="utf-8"?>

<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">

<application>rlftest>

<version>3</version>

<system-properties>

<property name="in.gae.j" value="true" />

</system-properties>

<sessions-enabled>true</sessions-enabled>

<static-files>

<exclude path="/**" />

</static-files>

</appengine-web-app>

With that done, I built the war file with mvn package. To run that war file under AppEngine, just type

$APPENGINE_HOME/bin/dev_appserver.sh target/rlftest-1.1-M8.

That also worked, I could see http://localhost:8080/. Finally, I published this to appsite with $APPENGINE_HOME/bin/appcfg.sh update target/rlftest-1.1-M8.

With that done, I browsed my AppEngine control panel and tested that the hello world application. It worked without a hitch. Now, I can get back to learning liftweb and Scala. It isn't at all clear to me how liftweb and datastore will work together, but I'll just have to burn that bridge when I get to it. So thanks Joe, you have been a great help getting me started with Lift.

Google Analytics: integration with web apps.

Google Analytics is a powerful, free tool to gather information on events in the click streams of your website's visitors. I'm trying to develop a web application for photographers, starting with my wife's site, folkertsfotografie.com. I'm tracking progress on fotositeproject and the new webapp is running on GAE, Google App Engine. This is a rather Google-centric project. I'm using Picasa to hold image galleries. The RSS from Blogger, Picasa and flickr are being integrated using Google's Feed API via the jQuery library jGFeed. So when we wanted to add analytics, Google Analytics was on the top our our list.

I had hoped to develop with Grails, but it has proven to be too slow. This is a known issue, and both Google and SpringSource are working on speed ups. I switch to Gaelyk, a lighter Groovy framework for GAE, which is resulting in reasonable performance. I looked at GAE sites using other Java frameworks, and was surprised how much faster they are. However, GAE also has similar issues with Rails on jRuby, presumably for the same architectural reasons. GAE performance with Groovy is still an issue, but one that I trust will have a happy ending in the next few months. GAE does a lot of validating of included libraries on startup, which is probably a really good thing for security. I wonder if they can have some standard pre-verified jar files for groovy, jruby, grails, sitemesh and so on. When I want to use one of these jars, I agree to use the 'Google compatible version'. When I load my app, they check the MD5 hash to make sure that I am still using an unadulterated copy & they can just copy their jar image into my application's memory without the expensive validation. Can't we pay this price once when we load the app, not every time we start it up? There is still activity in the Grails community to develop GAE integration, so I can't believe that this issue will remain for long.

---

Returning to my original topic: Google Analytics requires adding some code snippets to the bottom of each tracked 'page' or in the event handlers for tracked events. In a MVC design, the code snippets must resign in the views. However, the URLs are tied to controllers, so the keys for each URL need to be managed by the controller. I had assumed that a quick search would find others who had discussed this topic. Surprisingly, this is not the case. I have written a page that will read a Picasa RSS feed and then show the album in Galleriffic, a terrific jQuery image gallery. I want to have URLs like /gallery/show/sarah2009, so I don't want to place 'the key' in a file like /album/show.gsp, since the controller don't even expose that URL to the user. I certainly want to track patrick2009 with a different code that campingTripSpring2007, even if they both use the same view. So for each controller, I need a map of the view, the model, Google Analytics keys to use. This isn't quite model data, since the model should only have 'domain' data, which in this cased is information about the albums, media feeds, images, models and clients. It seems that only the controller is in a position to address this.

REST and application evolution

REST turns out to be the key to allowing software to be upgraded without breaking applications. The idea is pretty simple,you agree on the small number of REST verbs. You then have interfaces that are specified by URL and a version number (perhaps part of URL). You then allow clients to specify their version number in a request.

This is discussed in a couple of InfoQ talks. Alex Antonov gave Case Study: RESTful Web Services at Orbitz. He shows how Protocol Buffers allow for efficient (he claims 7 times faster than SOAP) and flexible message passing using HTTP. Protocol Buffers is an open source HTTP messaging protocol developed by Google. The exchanged data uses protobuf.

Steve Vinoski gives a great talk on RPC and its fundamental flaws. He was an expert in CORBA in the 90's and he has recently moved to REST and Erlang. He is quite convinced that functional programming is the future and that imperative languages, such as Java and C# are based upon a the wrong model of distributed programming.

These talks dovetail nicely with Kirk Wylie's earlier InfoQ talk, Restful approaches to financial systems.

Google Sites - easy project web sites

I'm working with my wife to develop a web site for her business, FolkertsFotografie. Currently, it is a Smug Mug site. She isn't a fan of selling images via shopping carts, so SmugMug is not a special advantage. I'm wanting to learn about Google AppEngine, so we set up a 'project' to make this happen.

This is where Google Sites fits in. I wanted to organize all of the brainstorming, designing and so forth. Google Sites makes it easy. See for yourself at Fotositeproject. There is easy integration with a host of Google tools, such as Google Docs, Calendar, Groups, Analytics and Webmaster Tools. Plus, I can rest knowing that I have offsite backup of everything and I do not have to worry about keeping the servers running 24/7.

Google Analytics & Webmaster Tools gives me as much monitoring data as my employer gets from IT. But unlike me, Google provides IT services for free.

Grails: By the numbers

Everybody likes to talk about highly productive environments, but numbers are notoriously hard to come by. This makes alterlab's ALTERthoughts Blog, entitled Grails vs. Rails especially interesting. Using Java J2EE as a base, both Ruby on Rails and Grails were found to be twice as productive. Grails beat Rails, but by a margin so slim it seems a dead heat.

Erlang, Mochiweb and Webmachine

Webmachine is amazing. Why doesn't .Net or Java have something this elegant?

Erlang Factory 2009 Palo Alto - Justin Sheehy - Webmachine @ Yahoo! Video

Have a look. I was impressed by the clean design, but I was blown away by the debugger described in the last ten minutes. This is an open source project in Erlang, the high concurrency functional language created by Joe Armstrong at Ericsson. As noted in the talk, Webmachine is a high quality component to include in a web framework, like Mochiweb. Mochiweb itself is interesting: it is an Erlang framework for high concurrency web applications. When serving static content, it has been compared with yaws and nginx in a deathmatch on Joes Blog! Richard Jones is contemplating a Million User Comet App. Erlang is turning into a pretty cool language for web apps with high concurrency.

I wonder if there is any attempt to write an equivalent in Scala, a high concurrency functional language for the JVM? I have got to get up to speed with Scala and Lift. There is a lot going on here...

Involve them in the same conspiracy

As previously noted, I am listening to the lectures from Berkeley's CS 61A. In the fifth lecture, the class is listening to a video of Alan Kay. He ends the talk, "If you want others to go along with you, you have to involve them in the same conspiracy." Oddly, this made me think about spreadmarts. Wayne Eckerson at TDWI create this term. He described a spreadmart as:

renegade spreadsheets and desktop databases that are wreaking havoc on organizations. Since then, many people have adopted the term because it highlights a painful, yet largely ignored, problem that plagues organizations today.

Spreadmarts contain vital pieces of corporate data that are needed to run the business. But since spreadmarts are created by individuals at different times using different data sources and rules for defining metrics, they create a fractured view of the enterprise. Without a single version of corporate data and centrally defined metrics, employees can’t share a common understanding the business. With spreadmarts, each worker marches to the “beat of their own drummer” instead of marching together toward a common goal. In short, spreadmarts undermine corporate productivity and sabotage business alignment.

I think Mr. Eckerson needs to consider Alan Kay's words. Mr. Eckerson has just cast the creators of spreadmarts as corporate saboteurs that are destroying the One True Way™ that he and and upper management wish to bestow upon their ungrateful underlings. This is a very top down approach, that probably isn't going to work well. He does present a 'co-opt' solution where you agree to using CVS output so the spreadsheet jockeys can still use their beloved Excel. Bu with title like Reeling in Spreadmarts, In Search of a Single Version of Truth: Strategies for Consolidating Analytic Silos and Taming Spreadsheet Jockeys, I hear contempt for the existing business processes and the folks on the front line that do the business' work.

Consider how differently Kirk Wylie treats the traders in financial firms in RESTful Approaches to Financial Services. They are exactly the 'spreadsheet jockeys' derided by Eckerson. Wylie describes centralized systems as über-systems, clearly a derogatory term. Mr. Wylie also recognizes the proper desire for traders to keep control of their data processing. He notes that developers are only acquiring the needed business understanding to become productive after three to four months - he is looking for bottom up solutions that support the 'front line', not centralized solutions to give upper management a dashboard of global KPIs, key performance indicators. He accepts that "Any system that doesn't consider the traders pathological dependency on Excel is doomed to failure." In the rest of the talk, he describes how RESTful solutions allow him to roll out shared data across a variety of format, including Excel, to end users on demand.

Like Alan Kay, Kirk Wylie is involved in the same conspiracy as his users. I'm betting he has more success than Mr. Eckerson is rolling out working systems.