Wednesday

RegExs for Common JavaScript Bugs

I use these 2 quite a bit:

:\s*[\[{]\s*[\]}]
This catches empty object and array literals in another object literal. Chances are you put this in a class/prototype definition like so:

Class.create({
myHash: {},
myList: []
});

That's roughly equivalent to declaring 'myHash' and 'myList' as static in Java. Unless you really want every instance of that class to share the same hash/array, that initialization needs to happen in the constructor:

Class.create({
initialize: function() {
this.myHash = {};
this.myList = [];
}
});

,\s*[\]}]
Catches [1,2,] and {1,2,}. The trailing comma causes IE to fail silently, which is always fun to debug. Since this is always wrong, you can change that to ,(\s*[\]}]) and do a find and replace with $1 as the replacement.

Post another regex findable bug if you know one.

Monday

A Java Developer's Guide to JavaScript - Functional Programming and JavaScript Mechanics

Homework Check

Last time we introduced the Prototype library. Hopefully by now you've at least looked over the Prototype API and are semi-comfortable with Firebug. If not, stop and take a few minutes to look around and write a short script or two.

Functional Programming

Functional programming is one of those things we don't really do in Java, since Java lacks first class functions. Essentially, when you can pass functions around like any other variable, you often end up sticking them together ("composing" them) to create new functions. The result of one function gets fed into another, etc. Programming effectively in JavaScript requires recognizing that many patterns that are explicit in Java are implicit in a functional language like JavaScript.

There are several 'classic' functions that are usually the basis of any good functional programming library:

map/collect
A map function takes a value and returns a new value. It's complement is reduce/inject. Here's an example using Prototype's collect function:

var foo = [1,2,3,4,5];
var bar = foo.collect(function(a) {
return a * 2;
}); // bar = [2,4,6,8,10]

reduce/inject
Takes two values (at least one is usually a collection, but they both can be) and combines them into one value. When called inject, the first value is usually the 'accumulator' and the second is a singular value. With Prototype, you'll use map/collect and inject on collections, but that's not the only way it is used. Map-Reduce is what powers Google searches. Here's an example:

var foo = [5,4,3,2,1];
var bar = foo.inject(1,function(a,b) {
return a * b;
}); // bar == 120

A little explanation for this one. The first call, the parameters are (1,5), the next call is (5,4), then next is (20,3), etc. The above is how inject works in Prototype, other versions of reduce can pass two outputs of reduce instead of the previous output and the next input. e.g., it might do (1,5), then (4,3), (2,1), resulting in 5,12,and 2, then apply the reduce function on the results and so on until we have a single value. To see how map/reduce powers Google, read Can Your Programming Language Do This?
select/findAll
Used to select/find a subset of a collection. e.g.,
[1,2,3,4,5].select(function(a) {
return a % 2 == 0;
}); // = [2,4]
all and any
Returns true if any or all of the elements in a collection match a given predicate. e.g.,
[1,2,3,4,5].any(function(a) {
return a == 4;
}); // == true

With Prototype, if you do not pass a predicate, then the value is evaluated as a boolean. You could use this to detect if a collection contains all (or any) non-zero, non-false and non-empty string values.
There are other common functions, but these cover the conceptual core. Most other functions will build off of these in some way. As you get to know the common functions, you'll discover uses for them that are extremely powerful.

Manipulating Functions in JavaScript

Before we talk about design patterns, we need to quick take a look under the JavaScript hood and see how we can manipulate the scope and arguments of functions. Remember that when we call a function as a member of an object, the this keyword lets us access variables in the scope of that object. But if you have only a reference to a function and you try to call it, this will refer to the global (window) scope. To allow us to set the scope of a function, we can use call. Here's an example:
var foo = { a: 1};
function aPlus(b) {
this.a = this.a + b;
return this.a;
}
aPlus.call(foo,5); // == 6 && foo.a == 6
This would be almost equivalent to
foo.aPlus = aPlus;
foo.aPlus(5);
However, the second example will overwrite aPlus on foo. If aplus happens to be an existing variable or method, we've now overwritten it.

call
can be used whenever you know the exact number of arguments you'll be passing to a function, but what if you don't? You might say, "When would that happen?" What if you have a function that takes a function as an argument, and returns another function? This is something we do a lot in functional programming, so we need a way to keep our function producing functions DRY.

Before we discuss how you would call a function without knowing the number of arguments, we need to talk about a special variable: arguments. Inside any function, the arguments variable is automatically defined. arguments is an array-like structure that has all the arguments passed to the function. In JavaScript, the parameters you declare fo a function are just convenient labels for indecies into arguments. The JavaScript interpreter doesn't care if you call the function foo(a,b) with 1,2, or 7 arguments. a = arguments[0] and b = arguments[1], and if arguments[1] is undefined, so is b. Like an array, arguments has a length, so you can iterate over it. e.g.,
function sum() {
var sum = 0;
for(var i = 0; i < arguments.length; i++) {
sum += arguments[i];
}
return sum;
}
Suppose we also have this function:
function double(a) {
return a * 2;
}
and we want to create a function that takes the sum of 2 or more numbers and returns twice the result. To reuse our existing functions, we need some way of creating a function that passes all of it's arguments to sum regardless of the number of arguments passed to it. apply to the rescue! Like call, apply lets you call a function and set the value of this, but instead of taking the arguments from the 2nd argument on, it takes an array (or array-like object) as the second parameter. e.g.,
function doubleSum() {
return double( sum.apply(this,arguments) );
}
doubleSum(1,1,1,1,1); // == 10
doubleSum(1,2,3); // == 12
doubleSum(); // == 0
We haven't made use of Prototype here yet because we haven't needed it. However, you may have noticed that we keep referring to arguments as an "array-like" object. That's because it's not a real Array and doesn't have any of the instance methods you can expect an array to have. Prototype provides a utility function for just this purpose, $A. $A(arguments) returns a true array with all of the arguments in it, so you can call all the standard Array methods and Prototype methods on it. We can rewrite sum like so:
function sum() {
return $A(arguments).inject(0,function(a,b) {
return a + b;
});
}
Using apply we can even write call:
function myCall(func,scope) {
var args = $A(arguments).slice(2); // creates a copy of arguments from index 2 on
return func.apply(scope,args);

};
That covers call and apply. Next time we'll see how we can use what we now know about functional programming and JavaScript mechanics to implement the design patterns we know and love from Java.

Techdirt: The Formula



If you read Techdirt very often, you know that they are the first to catch it when a corporation (or government, to be redundant) is trying to hang on to a business model that is losing relevance. While I often find Techdirt insightful, they too often use the formula "Why doesn't $company stop wasting resources doing $X and do $Y instead?" Where $X is something like filing a lawsuit, which often involves a lawyer on retainer or a few thousand in legal fees (i.e., a drop in the bucket of a corporate budget), while $Y is something like investing millions and billions in building new cable/telephone infrastructure. The implication is that $X is somehow keeping them from doing $Y, and that the two are comparable strategies, which is rarely the case. It would be like me saying: "Instead of whining about the flaws in corporate business strategies, Techdirt should fund startups to compete using their superior ideas."

That said, I like Techdirt most of the time and can generally count on them to be one of the first to break significant tech news, they could just use some fresh material.

Saturday

jQuery Greasemonkey Template & Netflix Queue Shuffler

jQuery is quickly becoming my weapon of choice to do DOM manipulation/enhancement. I'm still a huge Prototype fan for general code, but jQuery really does the DOM well.

So, when I set out to write a Greasemonkey script to randomize the order of the movies in my Netflix queue, I needed to find a way to include jQuery. The end result is fairly brief:

// ==UserScript==
// @name
// @namespace
// @description
// @include
// ==/UserScript==

/**
* @param {Function} func the function you want to execute
* with jQuery. The only argument passed will be jQuery, so
* your function should look something like this:<pre>
* withJQuery( function($) {
* $(".foo .bar")...
* } );
* </pre>
*/
function withJQuery(func) {
if(!unsafeWindow.jQuery) {
var s = document.createElement('script');
s.src = 'http://code.jquery.com/jquery-1.1.2.js';

s.addEventListener('load',function() {
// lots of people like to have their own $ function
unsafeWindow.jQuery.noConflict();
func(unsafeWindow.jQuery);
},false);

document.getElementsByTagName('head')[0].appendChild(s);
} else {
func(unsafeWindow.jQuery);
}
}

// your code goes here, use withJQuery when you need it


The trick was figuring out that in a Greasemonkey script, you can't do element.oneventname = handler (you have to use addEventListener) and that once jQuery loaded, it would be in unsafeWindow, not window.

If you came here for the queue shuffle script, sorry to have bored you. I'll do another 'JavaScript for Java Devs' chapter soon.

Monday

Grails, DWR, and the dynamic web

This post used to be titled "Using DWR with Grails", because the Grails DWR plugin lacked support for custom converters and creators. I wrote this patch (EDIT: Nabble screws up the patch formatting so that it doesn't take. You can download the patch here) adding this functionality and I was going to blog about how to apply and use it. However, before I posted this, I did a bit of searching, and apparently my patch got applied the same day I submitted it. It would have been nice if someone had told me they took my patch, but presumably this means that the next release will have the functionality. If you can't wait, I presume you can build from the trunk, or use my forum post. Enough of that.

I'm not big on making predictions, but this one seems pretty obvious to me. The reason there are so many Java web frameworks is that they all suck at being web frameworks. There is no clear winner in the Java space. We started with Servlets, which were terrible for web development because you couldn't tell what the heck the final HTML would look like. Then we had JSP, which is better, but once you start trying to apply DRY, get into a templating system and tag libraries, you're suddenly just as removed from the final result as you were with Servlets. In both cases, if I want to know what is being produced, I have to fire up my JEE container and do a request. Not to mention that creating templates and tags requires some lovely combination of XML and Java code. I don't know about you, but that feels off to me.

Why? XML doesn't give me much flexibility. DRY gets violated all the time because there is no mechanism in the design of XML for reasonable reuse. Java on the other hand lets me refactor and reuse to my heart's content, but as we've already established, it's crap for actually producing HTML.

And what about Ajax? JSP is fine if your site uses tables for layout and all links result in a full page refresh or occasional popup window, but if you want to build a responsive, dynamic site (using say, Ext) you'll end up having rendering logic in two places, WhateverCrapFramework and JavaScript. The best possible outcome is that you hardly use JSP (or whatever your 'controller' is) and rely on your Ajax toolkits.

Again why? Because static typing everything is not appropriate for user interfaces. A user only has one input type available to them: text. They can't enter FooBarDtos directly, they don't even know what one is. You have to take some text and convert and validate it. Granted, they can also click things, but HTTP seals the deal for the web because all data submitted must be in text form. Your types cannot help you until you have already converted and validated the data, at which point it's ready to pass on to your service layer. So why the heck do you need types everywhere in your controller? You don't, but Java forces them on you anyways.

I think Java is great for web applications by the way. Spring+Hibernate remains my first choice for implementing a solid service and data access layer (I like Guice too). Static typing is great for helping ensure that your services are solid, and your data remains valid.

Back to Ajax. I made a big investment in JSF in graduate school. I nearly killed myself and my wife building a framework for creating Ajax enabled components for JSF. It took me a few months after finishing to realize that JSF still doesn't make Java web development any less like wearing plate mail while figure skating.

When you get right down to it, the best language for developing a web UI is the language that was always meant for the web: JavaScript. DWR completely bridges the gap between Java and JavaScript. It converts all that pesky string data into nice typed data for your Java service layer and then turns around and converts those types that the user doesn't know about into pleasant little strings for web browsers to gobble up. You can expose your Spring service beans directly, secure them with Acegi (you better be doing that anyway, if you're relying on your controller to secure access, you're using a sieve as an umbrella), and call them from JavaScript without even having to know how HTTP requests work, much less Ajax.

In conclusion, we need real developers on the client side. Good designers/UX folk don't usually make the best technical developers, and the dynamic web (especially for enterprise) requires a solid, coherent client side codebase, and things like DWR make that easy. If you're a Java web application developer that's still afraid of JavaScript, you'd better learn to face your fears.

The IE script load event

This is just a short rant. I promise I'll post more about Grails latter.

I really hate IE. Take this code to create a script tag and fire a callback once it's loaded:

var script = document.createElement("script");
script.setAttribute("type","text/javascript");
script.setAttribute("src", fileName);
Event.observe( script, "load", function() {...});

Works great in Firefox. But in IE6? Never fires. You have to do this instead:

Event.observe( script, "readystatechange", function() {
if(script.readyState == "complete") {
...do stuff...
}
} );

It's not that there is extra functionality. It's that a decision was made to provide a complicated interface that can handle like two extra edge cases, but the much simpler, obvious and consistent 'onload' handler gets left out.

Friday

Custom DWR Converters

DWR is great. If you're full bore into Ajax development, it eliminates the need for a awkward controller like JSF or Spring MVC. You can write static DHTML that talks directly to your service layer, which keeps the UI code clear and simple. Secure your services with something like Acegi, and you're almost there.

I say almost, because the way DWR marshals your beans is either all or nothing. You can tell DWR, convert only these classes, and only these properties and that's it. What if you have different user roles that have different read permissions? e.g., detailed user info should be accessible to the admin and no one else. What if you want summary information when an object is nested inside another one? e.g., my Label BO has a name, description and a couple of flags associated with it. When I retrieve a labeled thing, say a message, I really only care about the label name so I can render a list of labels on the message. But when I retrieve a Label for editing, I need all the information about the label. Let's see if we can fix this.

Because DWR is open source, I can browse around and find the source for BeanConverter. It has a lot of hooks (including isAllowedByIncludeExcludeRules, which would be helpful for role based filtering), but what we really need here is a hook to control what converter is used. Because we have the source code, why don't we add one? I'll just create a subclass of BeanConverter and override convertOutbound like so:

public OutboundVariable convertOutbound(Object data, OutboundContext outctx)
throws MarshallException {
Map ovs = new TreeMap();

ObjectOutboundVariable ov = new ObjectOutboundVariable(outctx);
outctx.put(data, ov);

try {
for (Entry entry : (Set>) getPropertyMapFromObject(
data, true, false).entrySet()) {
ovs.put((String) entry.getKey(), getPropertyValue(
(Property) entry.getValue(), data, outctx));
}
} catch (MarshallException ex) {
throw ex;
} catch (Exception ex) {
throw new MarshallException(data.getClass(), ex);
}

ov.init(ovs, getJavascript());

return ov;
}

The code is pretty much the same as in the superclass, but with 1 key difference. My version calls getPropertyValue. In the template class, I keep the default behavior, loading the converter from the converter manager, but we'll create a subclass in a moment that takes a different approach.

First, let's look at what dwr.xml offers us in terms of configuration. It looks like you can set properties on converter instances using param elements. I prefer to use a concise format for configuration whenever possible, so our format will be something like "propertyName:includeExcludeProperty1,includeExcludeProperty2 otherProp:more,of,the,same". Supposing this was an include, then when we create the converter for the otherProp property, it will include the more,of,the and same properties and no others. Got it?

We need to provide parsing for our custom format in the converter:

public class SubBeanConverter extends TemplateBeanConverter {

protected Map subInclude = new HashMap();
protected Map subExclude = new HashMap();

public void setSubIncludes(String defs) {
this.subInclude = parseIncludeExclude(defs, true);
}

public void setSubExcludes(String defs) {
this.subExclude = parseIncludeExclude(defs, false);
}

private Map parseIncludeExclude(String defs,
boolean include) {
Map map = new HashMap();
for (String def : defs.split("\\s")) {
String[] split = def.split(":", 2);
BeanConverter bc = new BeanConverter();
bc.setConverterManager(getConverterManager());
if (include) {
bc.setInclude(split[1]);
} else {
bc.setExclude(split[1]);
}
map.put(split[0].trim(), bc);
}
return map;
}
...

TemplateBeanConverter is our original subclass of BeanConverter. Since we're using the same format for includes and excludes, the same method is used to parse them both. For each property we list, we create a BeanConverter instance that we will use in place of the usual converter.

The last step is to override getPropertyValue and use our special BeanConverter instances:

protected OutboundVariable getPropertyValue(Property property, Object data,
OutboundContext outctx) throws MarshallException {
Object value = property.getValue(data);
if (value != null) {
BeanConverter exConverter = subExclude.get(property.getName());
if (exConverter != null) {
return exConverter.convertOutbound(value, outctx);
}
BeanConverter inConverter = subInclude.get(property.getName());
if (inConverter != null) {
return inConverter.convertOutbound(value, outctx);
}
}
return super.getPropertyValue(property, data, outctx);
}

Easy right?

Next time I'll talk about my experience using Grails with DWR.

UPDATE: I mentioned role based filtering above, but I will leave that as an exercise to the reader. One hint: if you're using Acegi to control access to pages,
org.acegisecurity.context.SecurityContextHolder.getContext()
.getAuthentication().getAuthorities()
will get you the list of granted authorities (AKA roles, e.g. ROLE_ANONYMOUS) for the current user.

Thursday

More than you wanted to know about JavaScript object creation

I began this journey because I needed to create a new instance of a JavaScript class with a variable number of arguments. Not knowing the number of arguments, I couldn't just write new MyClass(arg[0],arg[1].. etc.), so I had to figure out how to replicate the behaviour of new in some other way.

You may have heard that JavaScript inheritance is prototypal. If you don't know what that means, thin of it as one object inheriting from another by making a copy of the original object. Everything in JavaScript is an object, including functions. You may think that JavaScript has classes. It doesn't. It really just has functions.

JavaScript is prototypal, but it wanted to appeal to programmers that were used to "classical" inheritance, so it included the 'new' keyword that we all love so much. That means you can do things like:

function MyClass(arg1,arg2) {
this.foo = arg1;
this.bar = arg2;
this.baz = 3;
}
var mine = new MyClass('foo','bar');
// mine.foo == 'foo'
So what's going on? MyClass is called, but there is this 'this' object that isn't declared anywhere. Where did it come from? new created it for us and put it into the scope of the MyClass function. If a function called with 'new' doesn't have a return statement, it returns 'this' by default. You could try to call MyClass by itself, but you would end up creating 3 global variables because this defaults to window. So how do we redefine this? There are two methods on all functions (remember, functions are objects, just like everything else) that let you control the value of this: call and apply.
var foo = { qux: '123' };
MyClass.call(foo,'foo','bar');
// foo.foo == 'foo' && foo.qux == '123'
// if you don't know the exact number of arguments you can use apply
var args = ['foo','bar'];
// apply takes an array for the arguments
MyClass.apply(foo,args);
You might now be thinking that MyClass.call({},'foo','bar') is the same as new MyClass('foo','bar'). Well, new does a bit more than that. Remember the prototype? Calling new more or less creates a copy of the class's prototype, which is then used as this when calling the function. e.g.

MyClass.prototype.myFunc = function(a,b) { return a+b; };
var mine = new MyClass(...);
// mine.myFunc == function(a,b) { return a+b; }


One brief digression from our quest: John Resig blogged about a way to create classes that are instantiated correctly even if you forget new. An interesting read, but not quite what we need since we may need to instantiate classes not under our control.

So if you create a new object, copy all the properties of MyClass.prototype, and then call MyClass.apply(...), is that the same as new MyClass(...)? Nope. new does some things behind the scenes that you cannot replicate without calling new. e.g.,

var mine = {};
for(var i in MyClass.prototype) {
mine[i] = MyClass.prototype[i];
}
MyClass.apply(mine,'foo','bar');
// mine instanceof MyClass == false

So, what we have is an instance that is almost, but not quite identical to an instance created with new. If you know that the object will only be duck typed, then this is good enough. However, if it's possible that it might end up in an instanceof or have some other low level type check, this isn't good enough. The final trick that you need is new:

function newInstance(type,args) {
var f = function() {};
f.prototype = type.prototype;
var o = new f;
return type.apply(o,args) || o;
}

By creating a no-arguments function with the same prototype, we can safely use new to get the prototype copying, and then call the constructor using apply. instanceof type (and other checks, like prototype.constructor) will return true, and the new instance will walk and talk like a duck of MyClass.

I'm a nerd. Or do I mean geek?... Hi, my name is Noah.

Aaaand it's Thanksgiving. I'm up late because I just got my slice and I can't stop tinkering.

I think my master's project is dead meat. Yes, JSF component creation is woefully painful and slow. Yes, a framework helps tremendously. Yes, my framework was good, and let you choose your Ajax implementation, and so on.

The problem with all that is that the JSF Ajax implementations suck. And the learning curve for JSF (much less components) sucks. It's just too quirky. Hopefully JSF 2.0 will make component creation easy, enable RESTful services, and will make it easier for newbies and old hands to just get things done. I'd like to hope all of that, but the only positive things I tend to hear about 2.0 are from Ed Burns, who seems like a very nice guy, but whose rhetoric reeks of corporate "gotta toe the line/I think my company is the best ever, is not a dinosaur who is slowly and inconsistently embracing open source, and is a bit too bureaucratic to be able to save Java." JavaScript is it, ya know.

Which brings us to my next topic. Kevin at work is the new Grails evangelist. I have to admit, Groovy is a cool language. If not for the fact that JavaScript is the NBL, I think Groovy would be the weapon of choice. Optional static types, intuitive C like syntax, closures everywhere, functional programming, etc. It's a readable LISP, IMHO. Since we'll probably be using Ext for my next project, (in wonderful freezing cold Pittsburgh, no less...) I thought that I'd take the opportunity to get familiar with it by working up a sample project. It's going quite well, so I must remember to get a few entries out of it (yes the separate tech blog is pretty much dead).

Groovy is a great complement to JavaScript/Ajax programming primarily because the thought processes are the same. There are some more syntactic niceties in Groovy, and there is sadly no Prototype for Groovy, but switching is completely effortless. And if I need to, I can always turn the static types back on and write some Java code. If you are a JEE developer and haven't looked at Grails yet, you need to.

More bulletins as events warrant. Cheers.

Wednesday

In the swing of things

I'm finally starting to feel settled in Austin. It helps to actually live here. Working downtown is pretty cool, and I like the bus. Despite the smells, it's usually a nice ride. I get a lot of reading done; too much in fact. I need a side project, so I'm trying to revive my master's project. We'll see how that goes.

November 10th (Saturday), we are having a BBQ from 3ish to whenever I turn into an angry drunk and kick everyone out. If you're reading this, you're invited. If you're not reading this, something is wrong, please stop before you destroy the universe. Thanks.

I need to read something intellectually stimulating, but I'd prefer fiction. Any ideas? Oh, and happy Halloween.

Friday

Living in Austin

There is so much to do here. In the last month I've learned more about bikes, built my own road bike, taken the dogs to two of the off leash dog parks, taken the bus downtown a few times (~30min in which I can read, write or just enjoy the scenery; in a car any of those will get you killed), and fixed 98% of the problems with the house, of which there were several dozen. Oh, and I've been very, very lazy, played video games a lot, and spent probably a week's worth of time on reddit. You may not believe it, but not having a job gets old after a while.

Last night we saw Quiet, Lovely for the first time in a few years, although we had to duck out early as H's stomach bug decided to grace us with one final encore. As an aside, The Mohawk seems like a nice place to get a drink, but until they finish their outside stage, it's a bit small for a decent show. Also, why the heck would you start a show at 10 PM on a Thursday night?!?!?! People who like music sometimes have jobs, do you really think you'll sell more beer by forcing them to leave before the show is over? Tonight, Meryll plays at Stubbs. Going by last night, we should show up at about 3 AM.

I fly to Boston on Sunday to start work. According to my parents, I've been to Boston, but didn't get to see much as I was only -2 months old. Think about it. I hope I like traveling as much as I think I do.

Monday

it's not so bad...

One of my committee members dropped out. Since my defense was supposed to be tomorrow, that means postponing it a few weeks while we find another person and give them time to get up to speed. Disappointing, but not that big a deal. We still move on Friday, I'll just have to make one last trip up to Hueco later this month.

Yes, my life will be marginally more stressful between now and then, but there's nothing to do about it, and it's not too bad.

Update: It's really not so bad at all. Another professor, hearing my plight, took pity on my poor soul and has undertaken the herculean effort of digesting my 68 page project document in less than two days. It looks like he'll actually make it too! My defense is postponed only until Wednesday. That doesn't mean it's a sure thing, but it gives me a good shot.

Sunday

blog, blag, blech

Grad school doesn't leave a lot of room for a social life. Certainly nothing to write about, unless you care about my frustrations with technology. That should change soon. That ends Tuesday, then we move to Austin, have a whole month off, and then start a job that sends me all over the country. My first day is in Boston. That's pretty cool.

Hopefully this will be a journal of cool stuff I do while traveling and when I'm home in Austin. I'm sure I'll also resume a bit of the philosophical bent this blog had long ago, but probably with less of the bad poetry/stream of consciousness/"look how emo I am" content. Stupid emo kids.

I'll be 25 this year. There's no avoiding it, I'm a grownup now. But, that doesn't mean we have to act like our parents.