A Grails Porcelain Ajax Framework - Part I: Component Refresh

The Foundation

The foundation of a good porcelain Ajax framework is component refresh, i.e., refreshing an area of the page with new/updated information. Component refresh is equivalent to reloading the page, but provides a better user experience since it happens asynchronously, and reduces the amount of work the server has to do.  Because we are requesting a block of HTML from the server, we don't have to write any additional JS to update the page. jQuery.load does all the heavy lifting. With component refresh it becomes trivial to Ajax enable pagination, sorting, filtering, lazy loading, tabs and many other actions for a better user experience and better performance. In this post, we'll learn how to create a porcelain Ajax framework with component refresh using Grails.

The App

Porcelain frameworks are usually built around an application. For our purposes, lets say we have a simple app for creating, listing, viewing and editing products. The products have a name, price, and description. Additionally, each product can be viewed in four different colors. In our app, the only difference between colors will be the image that is displayed for the product, but the different colors could easily be different tabs of product information, or different configurations of the product.

All our work will be on the product list, which will look like this:

our made up products list

The Project

We start with a standard Grails project and add a Product domain class and generate the controller and views. How to do all that with Grails is out of scope for this discussion, but it's pretty easy.

By default, Grails lists products in a table. Typically products are listed as "cards" to give them more room, so we'll enhance the markup slightly:



At this point we haven't written a line of Ajax code, we're just creating a simple app that will work even with JS disabled. In anticipation of adding Ajax functionality though, we should break the list page up into the components that we plan to refresh. To do this, we'll make use of the g:include tag, which allows us to inline another controller action in our page. We'll create two new controller actions and adjust our templates accordingly:



Notice the original list code has moved into renderList and we return productIds instead of the actual product object.

And we replace the list in list.gsp with g:include controller="product" action="renderList". renderList.gsp is just:



and renderProduct.gsp is the product div above.

For the colors pages, we're going to reuse renderProduct:



and that's it. We now have all our functionality setup and it all works even with JavaScript disabled. Time for some progressive enhancement.

The Framework

The core of our component refresh implementation is going to be g:include. Since g:include is plumbing, we need to ensure that our porcelain covers it very thinly to make future extension easy. One option would be to  use Groovy meta-programming magic to intercept calls to the built in Grails taglib where g:include is. This  has the advantage of being transparent to the user. We don't have to worry about using the right tag, and if we already have a lot of existing code using g:include, this would be the way to go. However, it has the disadvantage of being complicated. If we're starting from scratch, it's much simpler to define our own tag. So enter the FrameworkTaglib:



Instead of g:include, we will use ajax:include. The new tag does two thing. First, it wraps the content of the component in a div with the class "ajax-component" so that out framework JavaScript will know that it's a component. Secondly, it sets the id of that div to the controller action and optional id for the include. That will allow us to call the correct action from our Ajax requests.

The second part of our framework is framework.js:



refresh is also a very thin wrapper, this time around jQuery.load. It has the nice feature of allowing us to use any element inside a component as a representative of the component. Also, we don't have to pass a URL, and can pass a data string or object, just like jQuery.load.

With these two components, our framework is ready! Our first task will be to Ajax enable the color links so we can view the different colors without leaving the list:



With our framework code in place, it couldn't be simpler. We strip the query string off of the link and pass it as data to refresh and we're done. The color links now refresh the individual products in place, for a greatly improved user experience. Also, notice we use jQuery.delegate so that when the component is refreshed we don't have to reattach event handlers.

So that was easy, but what about pagination? Surely it will be a bit more work to Ajax enable pagination.



Or maybe it's the exact same code. It looks like progressively enhancing links in this way might be something to add to framework.js. Notice that we can refresh the entire list as a component, but also have nested components that can be refreshed individually.

In very little code, we've built a framework that has allowed us to progressively Ajax enhance both pagination and content toggle in about seven lines of JS. But lets take an objective look at what this framework does well and what might be problematic.

You can download the complete project to this point from GitHub.

The Good

There is a lot to like about the framework as it stands.

• Very simple usage.
• We take advantage of progressive enhancement. Normal links become refreshes very easily.
• The whole framework is less than 50 lines of code at this point.
• Components can be nested inside other components.

The Bad

Using this code as is on a serious project is probably not the best idea for a few reasons:

• All components are wrapped as divs. That means refreshing something like a table row is impossible.
• In framework.js we assume that the default URL mapping of "/$controller/$action?/$id?" is used, which may not be the case, in which case refresh will fail.
• We have to pass all the parameters to refresh. For just progressively enhancing links, this isn't such a big deal since the link should have all the parameters, but as we get into more advanced features, it will become a pain point.
• Several features of g:include will not work. In particular, the model and params attributes will not be present in a refresh. For simple cases, that's not a problem, but a seriously application will quickly feel the pain of such limitations.
• Two nitpicks
• We use the namespace "ajax" for the taglib, which is not likely to be unique. Another poorly written plugin could cause problems.
• We use the id attribute to store the controller action information. A class might be more appropriate, if a little harder to parse.

The Conclusion

The core idea of a good component refresh is being able to pair together the view (GSP) and the code that generates the model (the controller action). This pairing allows us to treat each component as it's own entity, making refresh and other advanced features possible. Grails and jQuery provided us with some very nice pipes to start with, so creating the component refresh porcelain was perhaps easier than it would normally be, but I think the simplicity helps the underlying concepts to show more easily.

Next time we will build multi-component refresh and try to fix some of the problems with our framework. It's not good from much beyond basic enhancement at this point, but we're off to a good start.

Remember, you can download the complete project to this point from GitHub.

JSON is Overrated

JSON is great. It's an extremely concise, machine parsable yet still human readable format. It's the best thing to happen to data interchange since XML, and it makes XML looks like the big piece of dog crap that it is. (Seriously, angle brackets? And redundant closing tags? Who thought that was human readable?)

However, some of you may be surprised to know, JSON is not the only format that can be returned in an Ajax request. Don't get me wrong, if I make a request to the Twitter API or some other webservice, I'm not going to be happy if it returns XML. That stuff is a pain. But there are other uses for Ajax that just consuming raw data. Did you know you can return HTML?

For example, search results since the beginning of time have been paginated. The nice thing to do nowadays is to make an Ajax request when the user requests the next page so we can avoid reloading the header, footer, sidebar and whatever else in your page isn't a search result. Now you almost certainly rendered the first page of results on the server, using whatever templating language it is you use. Something like:



So when it comes time to make an Ajax call and load that next page of results, naturally you will write another controller method to render the results as JSON and then a bunch of hideous JavaScript to concatenate HTML together and replicate the logic you already have on the server. Because that's a lot easier than:


([/sarcasm] for the facetity impaired; it's a terrible affliction.)

When you're creating any sort of Ajax endpoint, just ask yourself one question, "Is anyone other than a script on this site going to consume this?" The answer is probably no, so I'm going to ask you a question in return, "What is wrong with you? Why would you want to generate HTML on the client by concatenating strings together instead of on the server with a nice templating language?"

If you're reading this blog, you probably already know that HTML concatenation is a code smell. You quickly realize that replicating all that rendering logic in JavaScript is going to be a mess. I bet you also know someone who's code stinks, so please find them and give them a gentle talking to about how their shiny new JSON hammer is not the best tool for screwing with Ajax.

Quit being lazy. lern2javascript noob

[Note: This is a blog post from a year ago, but I'm reviving it because I'm seeing developers and companies lose work because they don't have JavaScript experience. If you work on web applications, you need to learn JavaScript. In addition to this post, which targets Java developers, I've added links to good getting started resources at the end.]

Prologue: Why you need to learn JavaScript

For the foreseeable future (probably the next 5-10 years), the majority of software development will be done on web applications. Many of these will be ports of existing desktop applications or designed to replace such applications. Developing desktop-like applications on the web is going to require that Ajax thing you've been hearing about, something called DHTML, DOM, etc. That means you'll be using JavaScript.

We Java developers have been conditioned to flee at the first mention of JavaScript which has meant that it has long been the job of the 'creatives' to do the JavaScript work. That's not going to cut it anymore. While the work UX experts do is crucial to developing any application, they have different training, priorities and experiences. RIAs need solid technical developers who understand best practices, patterns, unit testing, etc. and can apply them to JavaScript.

One way Java developers try to avoid having to deal with JavaScript is through frameworks like JSF or GWT. There are certainly advantages to using a framework, but they are no substitute for being comfortable with JavaScript. Developing RIAs through a framework without knowing JavaScript is like using Hibernate without a solid understanding of SQL and relational databases. You may get by, but it wont be easy and it wont be pretty.

Hopefully you're now convinced that JavaScript is in your future. I want you to know that you don't have to be dragged into it kicking and screaming. It will be fun. You know those closure things that they're talking about putting in Java 7? JavaScript has always had them. Ever get tired of writing the same Java code over and over? JavaScript code is far shorter and less repetitive.

Differences: The Good, The Bad, and The Ugly

I'm going to focus on the immediately important differences for now. We'll pick up the more subtle ones as we go.

• JavaScript, like Java, has a C like syntax so you should feel pretty much at home. It has fewer keywords than Java, but adds a few new ones. The most important one is function, which we'll talk about more in the next chapter, but there's also var(declares a variable), with (to generally be avoided), and in (multiple uses).


• JavaScript is loosely (weak) typed. For a UI, that's a good thing, because most of the data you're working with is string data, so you don't have to declare String everywhere and if you need a string to occasionally be a number, it happens automatically. Every object has a boolean value of true except 0,the empty string and false. An undefined or null variable also evaluates to false, so you can check to see if an object has the method/property that you want like so:
  

  if(!foo.bar) {
   throw "foo must have bar";
   // you can throw anything to raise an error
  }
  

  Of course, you can do whatever you like if the object you were passed doesn't have a particular method.


• There are only Numbers in JavaScript. There is no distinction between int,float, double and long. Math.round, Math.floor and Math.ceil are there if you need them. The maximum value is 1.7976931348623157e+308 so don't worry too much about overflowing. You shouldn't be doing any heavy computation on the client anyways. If you try to do arithmetic with something that isn't a number, the result will be NaN (not a number). NaN and anything is NaN. That includes undefined variables:

  var foo = 1;
  foo + bar; // == NaN, since bar is undefined

• There is no char type. You can use either single or double quotes to delimit a string. 'f' is always the string "f". charCodeAt(index) on strings will give you the character code as a number.
  


• In addition to functions, objects and arrays are first class citizens (technically functions and arrays are also objects). You can define array literals using the [] syntax: var myArray = [1,2,3];. For objects you use {} and provide a list of name:value pairs: var myObj = { foo: "bar", baz: [1,2,3] }. Both array and object literals can span multiple lines. It's generally good form to break lines after a comma (if you must break lines):
  

  var foo = {
   bar: "baz", qux: 123,
   quxx: { a: 1, b: 2, c: 3}
  };

• Perhaps the most important JavaScript data type is the function. We wont get into it now, but you need to know that you can treat functions like any other variable. e.g.,
  

  var foo = function(a,b) {
   return a + b;
  };
  
  foo(1,2); // == 3

• When a function is called as a property of an object, it gets access to this. e.g.,

  var foo = new Object();
  foo.bar = function(a) {
   return a + this.b;
  };
  foo.b = 2;
  foo.bar(1); // == 3
  foo.bar(2); // == 4



• Note that while you can get put a reference to foo's bar function in a variable, it will no longer have access to foo via this.

  var bar = foo.bar;
  bar(2); // == NaN (not a number)

  Since this.b is undefined, 2 + undefined is NaN. You might be wondering why this.bar doesn't result in an error. Well, it's because this is always defined. Since JavaScript executes in the context of the browser window, the default this is the window. This is where all global variables live. Incidentally, you can reference the window explicitly with the window object.
  


• There is a difference in scoping in JavaScript as well. You're used to having block local variables. Well, in JavaScript variables are either global or function local. Blocks share their enclosing scope, so
  

  var foo;
  if(true) {
   foo = 1;
  }

  is the same as

  if(true) {
   var foo = 1;
  }

  foo will still be visible after the else block. If you fail to define a var inside a function, a global variable is created (or if it exists it is overwritten). e.g.,

  
  function() {
   foo = 1; // sets window.foo = 1
  }

  So, think carefully about scope when you declare and use variables.
  


• JavaScript is interpreted, which means that you don't even know 100% if a line of your script is valid until it actually runs. That makes unit testing crucial. Fortunately tools like JUnit, Javadoc, etc. all have JavaScript equivalents that will seem very familiar. There are also a host of development tools including Eclipse plugins, Aptana, IntelliJ support, etc. to help you along.
  


• Lastly, you should know that JavaScript inheritance is prototypal. It is fully possible to achieve class based inheritance, which we will do with the help of a library called Prototype, but the prototypal nature of JavaScript gives us some interesting options which we will explore later.

Your Environment

We're Java developers, so we know the power of a good IDE. Eclipse, NetBeans and IntelliJ all have decent JavaScript support. There are a variety of plug-ins for Eclipse (I use JSEclipse), but I'm going to tell you right now that none of them will give you the code completion that you're used to. It's simply not possible with a dynamic, weakly typed language. You'll have to learn APIs the old fashioned way, by reading the docs and using them.

Fortunately, JavaScript gives you an amazing debugging and exploratory tool that you have never had and likely will never have with Java. Its name is Firebug. Go ahead and install it. Done? You need to restart Firefox. OK. Good. Now see in the bottom right corner? Click it. This is Firebug. It does a lot of things, but for now we're just going to mess around with the 'command line'. Do the following:

1. Click the in the bottom right corner of Firebug.
2. Paste this into the command line and press Ctrl+Enter or click 'Run':
   

   $('demoSteps').childElements().each(function(item,i) {
    item.insert("("+i+")");
   });
   

OK, it's not the most impressive example, but you did just execute code without having to reload the page or anything. Here we're making use of a library called Prototype. Bookmark the API immediately. What the above code does is call a function named '$' (yes, that's a valid variable name in JavaScript) with the argument 'demoSteps'. When $ is given a string, it looks for the element in the HTML with id="demoSteps". We get the child elements and iterate over them, calling the anonymous function we defined there on each element, along with it's index. We then append the index in parentheses to the body of each element.

The equivalent JavaScript without Prototype is a lot longer. You could start learning JavaScript without any libraries, but then you'd quickly get caught up in fixing bugs and writing the same code over an over. Fixing bugs doesn't give you a good appreciation for a language, nor does clunky repetitive code. In any serious web project you will use Prototype or a similar library, so it doesn't hurt to know it. After you're comfortable, we will delve deeper into the JavaScript language and you'll learn what Prototype hides from you.

Before we go, drag this onto your bookmarks toolbar: Insert Prototype. Now you can go to any web page, click that bookmarklet, pop open the console and start using Prototype to play. For homework, explore the Prototype API, especially the utility functions and the Enumerable methods. Try to write as many snippets using the API as you can.

Resources

• http://javascript.crockford.com/ Read these essays. They're all pretty short, but they are essential. Also, get a copy of Crockford's book.
  
• http://jquery.com/ jQuery is very popular. It has tons of plug-ins, with new ones popping up everyday. You will need it.
  

jQuery Greasemonkey Template & Netflix Queue Shuffler

jQuery is quickly becoming my weapon of choice to do DOM manipulation/enhancement. I'm still a huge Prototype fan for general code, but jQuery really does the DOM well.

So, when I set out to write a Greasemonkey script to randomize the order of the movies in my Netflix queue, I needed to find a way to include jQuery. The end result is fairly brief:

// ==UserScript==
// @name           
// @namespace      
// @description    
// @include        
// ==/UserScript==

/**
 * @param {Function} func the function you want to execute 
 * with jQuery. The only argument passed will be jQuery, so
 * your function should look something like this:<pre>
 * withJQuery( function($) {
 *   $(".foo .bar")...
 * } );
 * </pre>
 */
function withJQuery(func) {
  if(!unsafeWindow.jQuery) {
    var s = document.createElement('script');
    s.src = 'http://code.jquery.com/jquery-1.1.2.js';

    s.addEventListener('load',function() {
      // lots of people like to have their own $ function
      unsafeWindow.jQuery.noConflict(); 
      func(unsafeWindow.jQuery);
    },false);

    document.getElementsByTagName('head')[0].appendChild(s);
  } else {
    func(unsafeWindow.jQuery);
  }
}

// your code goes here, use withJQuery when you need it

The trick was figuring out that in a Greasemonkey script, you can't do element.oneventname = handler (you have to use addEventListener) and that once jQuery loaded, it would be in unsafeWindow, not window.

If you came here for the queue shuffle script, sorry to have bored you. I'll do another 'JavaScript for Java Devs' chapter soon.