noah's blog

Android - Reliable Alarms to Services

2010-09-27T09:07:00.005-05:00

I love programming for Android. The APIs just make sense. You can tell that fluent Java developers designed this thing. Despite the fact that everyone's favorite criticism is that Android is "fragmented" I haven't had any significant cross-hardware issues. Follow the guidelines (What?!? I have to follow rules if I want my app to work? So much for freedom...) and 95% of apps should just work on any platform without much trouble. I'd say you're even a step ahead of iOS development because the platform has extremely simple ways to support multiple screen sizes, so your app can easily look good on any device. That said, there are occasional bugs, and this post is about one of them.

This may not even be a bug, but rather a case of poor documentation. The problem arrises when you use the AlarmManager to schedule a repeating alarm that is supposed to wake the phone and start a service. e.g.,

PendingIntent pi = PendingIntent.getService(this, 0, intent, PendingIntent.FLAG_UPDATE_CURRENT);
alarmManager.setRepeating(AlarmManager.RTC_WAKEUP, timeToStart, 15 * 60 * 1000, pi);

It will probably work great during most of your testing as the phone will be awake. However, once you stick your phone in your pocket and forget about it for a while, you'll realize, "Hey, wasn't an alarm supposed to go off?" You'll pull your phone out, turn it on, and suddenly it will go off. The first few times you'll think it's odd, until you realize that the alarms aren't going off when the phone has been asleep for awhile, but they are delivered as soon as it wakes.

Hopefully you find this post or something like it and don't have to spend days Googling to figure out that only a BroadcastReceiver can reliably receive a wakeup alarm and then spend another day learning that the Context that onReceive gets ~~can't start services. The~~ ~~solution~~ ~~is pretty simple, but you'd never figure it out from the documentation.~~

EDIT: I may be totally wrong about the context being a problem as you can see in the comments. Try it without the using getContext first and if you don't have any problems getting the service to start, avoid the Cargo Cult code I'm using below. If you do have problems and the context trick helps, please let me know in the comments.

import android.content.BroadcastReceiver;
import android.content.Context;
import android.content.Intent;
import android.content.pm.PackageManager.NameNotFoundException;


public class AlarmProxy extends BroadcastReceiver {

    Context getContext(Context ctx) {
        try {
            return ctx.createPackageContext("com.yourpackage.ofyourapp", 0);
        } catch (NameNotFoundException ignore) {}
    }
    
    
    @Override
    public void onReceive(Context ctx, Intent intent) {
        Context context = getContext(ctx);
        Intent newIntent = new Intent(context, YourService.class);
        newIntent.setData(intent.getData());
        newIntent.putExtras(intent);
        context.startService(newIntent);
    }
    
}

The context your receiver gets is probably a ReceiverRestrictedContext ~~which isn't connected to your application and can't start services, so~~ you may need to use createPackageContext to get your app context and start your service. You'd then set alarms by sending them to your receiver:

Intent intent = new Intent(this,AlarmProxy.class);
// ...
PendingIntent pi = PendingIntent.getBroadcast(this, 0, intent, PendingIntent.FLAG_UPDATE_CURRENT);
alarmManager.setRepeating(AlarmManager.RTC_WAKEUP, timeToStart, 15 * 60 * 1000, pi);

Happy alarming!

EDIT: Don't forget to register the receiver in your Android manifest:

<receiver android:name=".AlarmProxy"> </receiver>

A Grails Porcelain Ajax Framework - Part I: Component Refresh

2010-07-26T10:00:00.003-05:00

The Foundation

The foundation of a good porcelain Ajax framework is component refresh, i.e., refreshing an area of the page with new/updated information. Component refresh is equivalent to reloading the page, but provides a better user experience since it happens asynchronously, and reduces the amount of work the server has to do. Because we are requesting a block of HTML from the server, we don't have to write any additional JS to update the page. jQuery.load does all the heavy lifting. With component refresh it becomes trivial to Ajax enable pagination, sorting, filtering, lazy loading, tabs and many other actions for a better user experience and better performance. In this post, we'll learn how to create a porcelain Ajax framework with component refresh using Grails.

The App

Porcelain frameworks are usually built around an application. For our purposes, lets say we have a simple app for creating, listing, viewing and editing products. The products have a name, price, and description. Additionally, each product can be viewed in four different colors. In our app, the only difference between colors will be the image that is displayed for the product, but the different colors could easily be different tabs of product information, or different configurations of the product.

All our work will be on the product list, which will look like this:

our made up products list

The Project

We start with a standard Grails project and add a Product domain class and generate the controller and views. How to do all that with Grails is out of scope for this discussion, but it's pretty easy.

By default, Grails lists products in a table. Typically products are listed as "cards" to give them more room, so we'll enhance the markup slightly:

At this point we haven't written a line of Ajax code, we're just creating a simple app that will work even with JS disabled. In anticipation of adding Ajax functionality though, we should break the list page up into the components that we plan to refresh. To do this, we'll make use of the g:include tag, which allows us to inline another controller action in our page. We'll create two new controller actions and adjust our templates accordingly:

Notice the original list code has moved into renderList and we return productIds instead of the actual product object.

And we replace the list in list.gsp with g:include controller="product" action="renderList". renderList.gsp is just:

and renderProduct.gsp is the product div above.

For the colors pages, we're going to reuse renderProduct:

and that's it. We now have all our functionality setup and it all works even with JavaScript disabled. Time for some progressive enhancement.

The Framework

The core of our component refresh implementation is going to be g:include. Since g:include is plumbing, we need to ensure that our porcelain covers it very thinly to make future extension easy. One option would be to use Groovy meta-programming magic to intercept calls to the built in Grails taglib where g:include is. This has the advantage of being transparent to the user. We don't have to worry about using the right tag, and if we already have a lot of existing code using g:include, this would be the way to go. However, it has the disadvantage of being complicated. If we're starting from scratch, it's much simpler to define our own tag. So enter the FrameworkTaglib:

Instead of g:include, we will use ajax:include. The new tag does two thing. First, it wraps the content of the component in a div with the class "ajax-component" so that out framework JavaScript will know that it's a component. Secondly, it sets the id of that div to the controller action and optional id for the include. That will allow us to call the correct action from our Ajax requests.

The second part of our framework is framework.js:

refresh is also a very thin wrapper, this time around jQuery.load. It has the nice feature of allowing us to use any element inside a component as a representative of the component. Also, we don't have to pass a URL, and can pass a data string or object, just like jQuery.load.

With these two components, our framework is ready! Our first task will be to Ajax enable the color links so we can view the different colors without leaving the list:

With our framework code in place, it couldn't be simpler. We strip the query string off of the link and pass it as data to refresh and we're done. The color links now refresh the individual products in place, for a greatly improved user experience. Also, notice we use jQuery.delegate so that when the component is refreshed we don't have to reattach event handlers.

So that was easy, but what about pagination? Surely it will be a bit more work to Ajax enable pagination.

Or maybe it's the exact same code. It looks like progressively enhancing links in this way might be something to add to framework.js. Notice that we can refresh the entire list as a component, but also have nested components that can be refreshed individually.

In very little code, we've built a framework that has allowed us to progressively Ajax enhance both pagination and content toggle in about seven lines of JS. But lets take an objective look at what this framework does well and what might be problematic.

You can download the complete project to this point from GitHub.

The Good

There is a lot to like about the framework as it stands.

Very simple usage.
We take advantage of progressive enhancement. Normal links become refreshes very easily.
The whole framework is less than 50 lines of code at this point.
Components can be nested inside other components.

The Bad

Using this code as is on a serious project is probably not the best idea for a few reasons:

All components are wrapped as divs. That means refreshing something like a table row is impossible.
In framework.js we assume that the default URL mapping of "/$controller/$action?/$id?" is used, which may not be the case, in which case refresh will fail.
We have to pass all the parameters to refresh. For just progressively enhancing links, this isn't such a big deal since the link should have all the parameters, but as we get into more advanced features, it will become a pain point.
Several features of g:include will not work. In particular, the model and params attributes will not be present in a refresh. For simple cases, that's not a problem, but a seriously application will quickly feel the pain of such limitations.
Two nitpicks

We use the namespace "ajax" for the taglib, which is not likely to be unique. Another poorly written plugin could cause problems.
We use the id attribute to store the controller action information. A class might be more appropriate, if a little harder to parse.

The Conclusion

The core idea of a good component refresh is being able to pair together the view (GSP) and the code that generates the model (the controller action). This pairing allows us to treat each component as it's own entity, making refresh and other advanced features possible. Grails and jQuery provided us with some very nice pipes to start with, so creating the component refresh porcelain was perhaps easier than it would normally be, but I think the simplicity helps the underlying concepts to show more easily.

Next time we will build multi-component refresh and try to fix some of the problems with our framework. It's not good from much beyond basic enhancement at this point, but we're off to a good start.

Remember, you can download the complete project to this point from GitHub.

How to Build an Ajax Framework

2010-07-22T15:45:00.001-05:00

Plumbing & Porcelain

I've built three or four Ajax "frameworks" in my career. I don't know if you would call them frameworks because there are really two kinds of things I mean when I say "framework". There is the extremely general purpose plumbing framework, which can do anything, but as a result leaves you to write a good bit of code and maybe pick and choose which parts you need. Some examples of plumbing type Ajax frameworks/components would include sever side frameworks like DWR, Ajax4JSF, etc. but also client side frameworks like dojo, Scriptaculous, jQuery, etc. Also most of the Rails type server frameworks have Ajax support built in: e.g., Grails and Django.

Then there are the porcelain frameworks. The problem with the plumbing frameworks is that they can do anything, so you're presented with a big toolbox full of neat things and a bunch of pipes, but you have to put it all together. A porcelain Ajax framework is usually built by an organization for their apps, or even for a single application. Sometimes they become an entire platform internal to the application and sometimes no thought is put into developing the porcelain and you end up with each developer doing Ajax in their own way.

Inner Platforms & Thoughtless Scaffolding

Some developers recoil at the thought of internal platforms. They tend to make a few use cases easy but then require horrible black voodoo workarounds when trying to do something the original architects didn't think of. Other developers are completely frustrated with the thoughtlessly thrown together and generally rickety Ajax platform they've been trying to maintain.

An inner platform sometimes arises when otherwise highly competent enterprise (read: Java or .Net) developers first encounter front-end programming and dynamic languages. They are used to structure and standards and ACID properties and so forth. This lack of static types and compilers instills a certain sense of fear and the only way to conquer that fear is to create "coding standards" and, then when those are not followed, to create an internal UI framework that forces everyone to code in a certain way, without closures, functional programming, overloaded operators, dynamic dispatch, and all those other ugly hacks that are bad for performance.

Other times, an inner platform is really just the result of well intentioned senior developers trying to make things easier for the more junior members of the team. Unfortunately, they often do so by hiding important information and you end up with a very leaky abstraction that is buggy and that the users (programmers other than the authors) are completely helpless to fix because they have no experience with the underlying framework. The points is that there are many ways to end up with an inner platform that does more harm than good.

Thoughtless scaffolding on the other hand is just the result of... thoughtlessness. Most often it's because a programmer without much experience and no real computer science training is writing all the code and whatever tool he's read about today is the perfect tool for the job. But it also happens when a new project is started and the technical leadership fails to identify the patterns that should be used. Maybe they start off with some vague notion like, "JSON is the answer," and become committed to a bad strategy early on and never have the time or the courage to go back and clean it up.

Both extremes have problems, but they also have advantages. For the tasks it was built for, the inner platform makes adding new Ajax features extremely simple. On the other hand, the thoughtless scaffolding never gets in the way. Because very little of it is connected, it's easy to rip part of it out and build something new entirely.

We want to find a middle ground between over engineered and thrown together, but where that middle ground lies can depend a lot on whether maintenance or new features are your priority. Instead we will focus on the core principles and features that any porcelain Ajax framework should encompass.

Core Principles

These are the core principles I have learned through trial an error over the past 5 years, building and using frameworks across the spectrum from somewhat thoughtless to a massive inner platform:

Remember it's a web app - It's not a desktop app. You're probably not build the next Gmail, so stick with what works. Use semantic markup, keep the pages simple and light, and use progressive enhancement so the app works without JS. For public facing apps this is a must obviously, but even for internal apps where you know that your users have a decent browser with JS enabled, you will stay out of trouble by keeping it simple.
Do everything you can on the server - Especially rendering HTML. JSON is overrated. jQuery's load method can do 95% of what you need in one line of JS. For the most part, JS should not be producing HTML. Things go much more smoothly when all HTML is produced server side because then the logic to generate the markup doesn't need to be duplicated. There are exceptions, like if you have a feature using 3rd party code, like a Google Map, that simply will not work without JavaScript. Then it might be necessary to generate some of the UI in JS. If you must generate HTML in JS, use a templating framework. If you are concatenating anything more than 100 characters of HTML together, you're making a mess of your code.
Keep the porcelain thin - Do not under any circumstances create more than one layer of abstraction between the code and the core web technologies. Most of the HTML output of your application should correspond to HTML in a template file. Taglibs are fine, but try to use your templating language even inside of them. Do not create a system where the developer doesn't have to know HTML. The same goes for JavaScript and CSS. Do not write code to generate CSS ever. You may use a framework that generates CSS to reduce repetition, but try to only use the big ones with a good community. Do not generate JavaScript code ever. Generate inline JSON instead and have your scripts consume it to determine behavior dynamically. Use stateful CSS rules.
Cover all the pipes - When you realize that to make it all work you need every load() call to add some parameter, or check some condition, you better hope you can go to one place in your code and make that change. Hopefully the plumbing provides good hooks, or there is some AOP/IoC magic that can cover it. Otherwise, you may have to create very very thin wrappers around plumbing functionality. Not in the hopes of portability (you will never port this app to a different framework, it will be rewritten), but so that you can funnel all calls to that feature through one place.

Core Features

So what features does a HTML based, server oriented porcelain framework provide?

Component Refresh - This is the core. There needs to be a simple way to designate an area of the page as a component and an easy way to make a call to refresh its HTML. This may be as simple as using jQuery's load with a selector to get the desired page fragment, although it's nice if the server can do the minimal amount of work necessary to render the HTML you need.
Multi-Component Refresh - Sometimes you need to refresh several parts of the page whose only common parent is the body element. You could just reload the whole page, but for an action that only affects part of the page that's both inefficient and bad UX. You could also use multiple requests, but if there are more than 2 components, then many browsers will block the 3rd request until one of the others return. There is probably also a server part of the framework that allows the components to share data instead of making redundant calls.
JSON - Not to be consumed by Ajax calls, but to inline as script tags in the page. Most frameworks can take the same data model you pass to your templates and convert it to JSON automagically for your JS to consume too. You probably will want to include it inside your components' HTML so you can update the JS state when a component is refreshed.

Next Time Let's Build One

Soon, I hope to have some example code for a simple Grails porcelain framework. Actual code can be a lot more informative and illuminating than pithy maxims.

JSON is Overrated

2010-07-14T21:48:00.000-05:00

JSON is great. It's an extremely concise, machine parsable yet still human readable format. It's the best thing to happen to data interchange since XML, and it makes XML looks like the big piece of dog crap that it is. (Seriously, angle brackets? And redundant closing tags? Who thought that was human readable?)

However, some of you may be surprised to know, JSON is not the only format that can be returned in an Ajax request. Don't get me wrong, if I make a request to the Twitter API or some other webservice, I'm not going to be happy if it returns XML. That stuff is a pain. But there are other uses for Ajax that just consuming raw data. Did you know you can return HTML?

For example, search results since the beginning of time have been paginated. The nice thing to do nowadays is to make an Ajax request when the user requests the next page so we can avoid reloading the header, footer, sidebar and whatever else in your page isn't a search result. Now you almost certainly rendered the first page of results on the server, using whatever templating language it is you use. Something like:

So when it comes time to make an Ajax call and load that next page of results, naturally you will write another controller method to render the results as JSON and then a bunch of hideous JavaScript to concatenate HTML together and replicate the logic you already have on the server. Because that's a lot easier than:

([/sarcasm] for the facetity impaired; it's a terrible affliction.)

When you're creating any sort of Ajax endpoint, just ask yourself one question, "Is anyone other than a script on this site going to consume this?" The answer is probably no, so I'm going to ask you a question in return, "What is wrong with you? Why would you want to generate HTML on the client by concatenating strings together instead of on the server with a nice templating language?"

If you're reading this blog, you probably already know that HTML concatenation is a code smell. You quickly realize that replicating all that rendering logic in JavaScript is going to be a mess. I bet you also know someone who's code stinks, so please find them and give them a gentle talking to about how their shiny new JSON hammer is not the best tool for screwing with Ajax.

OAuth and Android - Dropbox is wrong; never ask for passwords

2010-07-07T22:21:00.002-05:00

EDIT: Not having an iPhone and not having any dev experience with it (yet), I wasn't sure whether or not this was possible on an iPhone. However, I've had an iPhone dev assure me that it is and that several apps (TripIt being an example) do it. So there really is no excuse. If OAuth is good on the web, it's good on iPhone and Android too.

Dropbox recently announced their API, which is pretty exciting because it's a great service. It uses OAuth for authentication, but under the Authentication For Mobile Devices section, there is this statement:

While Dropbox uses OAuth for the base of its authentication strategy, OAuth doesn’t work very well for mobile devices. It’s much too difficult and error prone to have a mobile device switch context to a browser, do the OAuth handshake, and then magically return to the original application with the tokens in hand ready to use. It’s bad for user experience and it’s bad for developer sanity.

I emphatically disagree with this statement, at least as far as Android is concerned. I think OAuth vastly improves the user experience, even from a mobile app and this attitude that it's too hard or too much trouble is damaging to both the experience and security of users.

Lets take their statement piece by piece.

It’s much too difficult and error prone to have a mobile device switch context to a browser

This is trivial on Android. I'll use the Gowalla OAuth flow in my examples because I think it's an elegantly simple implementation of OAuth, but this works for any OAuth system:

That's it. The browser is launched and the user will see the Gowalla authorization page. If they're already logged in, they don't even have to type their username and password. Now there is an activity that is difficult and error prone. I hate entering login credentials on my phone, and I can copy and paste them from LastPass. Even with Swype, typing on a mobile device is neither easy or error free. Save your users the headache and use OAuth.

It’s much too difficult and error prone to ... do the OAuth handshake, and then magically return to the original application with the tokens in hand ready to use.

Again, this "magic" is almost trivial for an Android app. You simply specify that your activity can handle the custom data URI scheme you specified in your redirect link:

Once the user clicks "Allow", your activity will be launched and the Intent data will have a parameter telling you the access code needed to complete the OAuth handshake. This isn't any more difficult and error prone than implementing OAuth in a web application. But it's not that difficult, because there are plenty of great OAuth libraries out there to help you on the web server, and you can use them on your Android device too.

It’s bad for user experience

If this is true for OAuth on mobile devices, then it's even more true for the web and we should abandon OAuth completely. An app launching a browser is not an unfamiliar thing. Receive an email with a link on a desktop, you click the link and it opens your browser. This is normal, good UX. I click a link to comment on your site and you take me to Facebook? That's weird. As a user I'm not sure what is going on. Suddenly everything looks different... I thought I was commenting on your blog? Why are you on my Facebook?

The response to this (for web applications) is that users simply need to become familiar with the concept and then it wont seem weird at all, it will seem normal. That argument also works for mobile devices. In fact, if you are worried about the user not knowing what is going on, just tell them! "You will now be taken to Gowalla to authorize this app. You may be asked to log in. After you authorize myApp, you will be returned here."

OAuth is Good for Dev, and Good for UX

The idea that OAuth isn't suited for mobile isn't in any way grounded in the Android mobile experience, as far as I can tell. That might explain why Dropbox has a lot to say about it's Objective-C library, but for Java they only offer two words: "Coming soon."

Anything you can truthfully say about OAuth being bad UX or bad for dev in regards to mobile is equally if not more true for the web. But nobody is suggesting that OAuth be abandoned on the web and we go back to asking users for their passwords. So don't do it in your mobile apps either.

jquery.writeCapture.js - Now with 60% more magic!

2009-12-10T16:08:00.004-06:00

I've gone ahead and written a jQuery plugin for writeCapture.js. I've reduced the problem to making one extra call in the chain and then the voodoo black magic takes over and you can't go wrong. Here's a kitchen sink example:


$('#foo').writeCapture().find('.bar').html(dangerousHtml).end().
find('.baz').load('returnsDanger.php').after(moreDangerousHtml).end().
addClass('loading');

In this context, "dangerous" HTML is any HTML that does or could contain a script tag that uses document.write, aka, the evil method. The implementation details are wonderfully complicated, but I've made the usage so simple that any code monkey can just chain whatever they like off of a call to writeCapture() and everything will be perfectly "safe". e.g., document.write will not destroy the application and all the content will show up where it is supposed to. So enjoy.

EDIT: the comments here aren't a good place to ask for help. You can get help through GitHub.

GitHub, Taming document.write and node.js fun

2009-12-02T13:00:00.005-06:00

I have a few projects on GitHub now.

The serious project is writeCapture.js. It exists to solve a corner case of the "load some HTML and inject it into the document" pattern. I love this pattern because it's generally the simplest way to combine Ajax with progressive enhancement. The idea is that you load a HTML fragment and just stick it into the document, or replace the old fragment with the new fragment. It degrades easily, and keeps things simple. The only problem is when the HTML you're loading is outside of your control, or can't be changed for some reason. Maybe it's a service provided by a 3rd party. If the HTML contains script tags, and the script tags contain calls to document.write, you're going to have 1 of 2 bad outcomes:

If you just use innerHTML, the scripts probably wont run (at least, not in every browser), so whatever content document.write provides will be missing. That's probably bad.
If you use jQuery.html() or .replaceWith() or any jQuery manipulation method, the scripts will run, and all the content already in the document gets wiped out. Oops.

So this is the part where you realize that you need writeCapture.js. It's open source, well tested, and covers every possible convoluted way that document.write can be slipped into a script: inline, local script file, cross domain and any combination of the 3 (even the script that writes a script tag that includes a script that uses document.write case. Seriously.) You're welcome.

Lastly, I've been having fun with node.js. node can do a lot of things, but using it as a web server/application is the one that appeals to me most, so I wrote a simple web server that streams whatever files are in the current directory. Of course, you can make it do anything you want in response to a HTTP request (or use raw sockets, or whatever) and I'm sure there are a dozen (G)rails like frameworks being written for it as we speak. Enjoy.

How to refactor without reFing it up

2009-10-20T08:59:00.010-05:00

Sometimes technical debt is unavoidable. There are lots of good reason why the code you have today is increasing your maintenance costs and slowing feature development. Sometimes you have to accept a hack in order to ship (shipping is a feature). Or if your product has been around a while, it's probably evolved far beyond the original design, and it just needs to be restructured. Or maybe it's not a good reason but it happened anyways, e.g., you trusted a less than stellar developer, forgot to write unit tests, or adopted a framework that sounds great in theory but just doesn't make any sense in practice. In any case, taking an iteration or more (when you can afford it) to deal with technical debt is just par for the course.

The first mistake made by many developers when they start refactoring is to start by cleaning up code. That way is a rabbit hole that will take you through multiple iterations to increasingly madder tea-parties (read: standups) and end with your PM screaming "Off with his head!" because you've made the codebase unworkable for weeks, all in an effort to make things better. It starts so innocently, with a single class. Then you notice that the problem with that class is related to one of its dependencies, so you chase that dependency down and discover that the general architecture is too small (or too large) so you eat the mushroom or drink the drink and soon you're drowning in your own tears. The caterpillar may console you momentarily by giving you a hit off his hookah, but it's just going to get weirder. Better to stay out of the rabbit hole.

The key is limiting the scope and hammering it into everyone involved that "WE DO NOT TOUCH ANYTHING OUT OF SCOPE". I'll say that again, in all caps WE DO NOT TOUCH ANYTHING OUT OF SCOPE. "But, but, the service layer is still a mess, which is why w-" WE DO NOT TOUCH ANYTHING OUT OF SCOPE.

The first step in a successful refactoring is for a senior developer or lead to sit down and figure out the smallest possible refactoring that can be done. Then create specific instructions on how the refactoring should be done. Literally, instructions like "Move line X to class Y and put Z where X was." You don't want to leave it up to the developer to come up with their own way of doing things. Developers get bored easily, and a single developer can try six different ways of doing things before he decides on one he likes.

Once you have detailed instructions and guidelines, cut your developers loose. After the first commit, be prepared to devote at least 10 minutes of your next standup to explaining the instructions again, because it turns out half of them didn't read them. It's not that you have stupid or bad coders on your team, it's that they're smart and think they don't need instructions. It's more fun to get coding and figure it out as you go, right?

Because you made the original refactoring small, once the effort is underway, you need to start planning the next one. Small and iterative works for features and it works for bugs. It will work for refactoring just as well. Don't waterfall refactor.

It goes against every technical design instinct we have, but the better course of action will almost always be to incrementally improves components of a system rather than redesign it from the ground up. You also have to keep the app running. If HEAD is unusable for even a day because of your refactoring, you're trying to change too much at once. Sometimes that means writing some glue code that is ugly and inelegant so that the component you're refactoring can be pretty and elegant. Your inner architecture astronaut will be screaming as he burns up on rentry, but even NASA sometimes needs duct tape.

Here is an example to get you started: Take your JSP/ASP/GSP or PHP templates. Define model classes for them, based on the needs of the UI, not based on whatever model you have in the database or ORM or the DTOs your services return. You should end up with simple inline expressions. e.g., ${model.foo} rather than ${_POST['foo'].replaceAll("\\.","-")} or ${fooList[i].bar.baz.foo}. Move all that ugly logic into your controller. That part is easy. The hard part is resisting the urge to go back your service layer or even your ORM mapper and develop a factory and mapping system that trasforms the output to perfectly match your templates so you can just write Foo.list(params) in your controller and have it be oh-so-elegant. It will be done in 2012. Remember: Worse is better.

The Ugly American Programmer: Bill Burke

2009-07-20T10:55:00.004-05:00

I originally tried to post this as a comment on Bill Burke's Polyglotism is the worst idea I ever heard, but as far as I know, my comment is still awaiting moderation. So I'll go ahead and dissect it here. Burke starts with an alleged tongue-in-cheek:

Let me first start off with some tongue in cheek…I’m an American. I have no need to speak another language because well, I speak English. 99% of the civilized world speaks English so WTF should I ever learn another language? Case in point, I minored in German in college. For two semesters I went over to Munich and worked at Deutsche Aerospace so that I could learn German better. The thing is, besides the fact that my coworkers spoke damn good English to begin with, all the documentation they put out was in English, the units used were feet, pounds, miles. When the French came over to work with us, we also spoke English. So what is the freakin point of learning German? I was pretty damn disappointed that I wasted all this time in college learning German when in reality it was just a freakin useless exercise…

So we've taken the ugly American attitude and are now applying it to software development. Now, if he was trying to make the point that there is a lot of benefit to learning another language, his tongue-in-cheek disclaimer would make sense. But instead the above ignorant statement is followed by more ignorant rambling about using multiple languages in a project. He seems to think tongue-in-cheek means "I can say something ignorant and not be criticized for it".

Which brings me to the point of this blog. Polyglotism in software has to be the worst idea I ever heard. The idea of it is that you use the language that is best fits the job. Some say this is a huge boon for the developer as they will become more productive. In practice though, I think this is just a big excuse so the developer can learn and play with a new language, or for a language zealot/missionary to figure out a way to weasel in his pet language into a company. Plus, you’d probably end up being average or good at many languages but a master of none….But lets pretend that it is a benefit to the developer. Developers need to realize that there are implications to being polyglot.

Notice his focus on the single developer who is out to ruin things through his own selfishness (he must hate America!) No consideration is given to the benefits that a team and organization can derive from using multiple languages (which we will get to). His next point, maintenance:

So, you’ve added a Ruby module to that big flagship Java application or product your company is so proud of. You did it fast. It works. And management loves you for it. They love you so much for it, they’ve promoted you to software lead and now you are running a brand new project. Now that you’ve left your polyglot project, somebody needs to take over your work. Unfortunately, your group is a bunch of Java developers. For any bug that needs to be fixed, these developers need to be retrained in Ruby, a new Ruby developer needs to be hired, and/or a Ruby consultant/contractor needs to be brought it. Multiply this by each language you’ve introduced to your project.

Oh no, a Java programmer has to learn some Ruby! If you have any talented developers, they'll probably jump at the chance to escape Java hell and learn something new, assuming they haven't already learned Ruby on their own. "Multiply this by each language" is meaningless scare language. This assumes that skills are not transferable between programming languages. It's not at all difficult to be proficient in multiple languages. Many Java/PHP developers are also well versed in JavaScript (if you discount JavaScript at this point, I hope you have a career path that doesn't involve any programming for the web). Knowing multiple languages demonstrates mental flexibility and teaches a developer to think about problems in different ways. I would never hire a programmer that was only proficient in a single language.

The JVM is pretty cool now. We can run Ruby on it, Python on it, and even PHP on it. Your JRuby apps can work with Java APIs. Same with Jython and JPHP. Great. So now your developers can use any one of these language to build out extensions to your Java app. But what happens when you want to refactor one of your re-used Java libraries? OOPS!!!

Groovy and Scala are conspicuously absent from Mr. Burke's list. Could that be because they both compile to bytecode, and the resulting classes can therefore be used by any JVM language? So you could use Java, Scala and Groovy together in the same project in a completely interchangeable way, right now.

Ah, so you’ve weathered through the maintenance and refactoring nightmares and you’ve finally shipped your product. Hmm, but you’ve just added the complexity of installing multiple runtimes on your user base. Its not hugely bad if you’ve used the JVM as your base virtual machine. But you still need to package and install all the appropriate Java libraries, Ruby gems, and Python libraries on the appropriate places of your user’s machine. You also need to hope that all the desparate environments don’t conflict with anything the user has already installed on his machine. And that’s just with languages running in the JVM. Imagine if you used vanilla Ruby, Python and Java all as separate silos!

These new languages are used to reduce maintenance by dramatically reducing the size of the codebase, and simplifying application setup and ramp up times. Give me a domain model and I can have a functional Grails or Rails application running in less than an hour. And the code base will be a fraction of the size of the equivalent Java project. As for packaging, Maven handles Groovy and JRuby quite well. I'm sure PHP and Jython will have support soon enough, if they don't already (I don't know).

A support call comes in for your product or application. Its obviously a bug, but where is the problem? Is it your application code? Your JVM? Your Ruby VM? Your Java library? Your Ruby gem?

What an absurd statement. Where is the bug? I don't know, how do you normally find the bug in a "pure" application? Lets see, the date is parsed incorrectly and we're using some Ruby code to parse dates... Where could the bug be? Is it in the PHP templates? Only if you're an idiot. You could make the same argument about the MVC pattern and Hibernate. 3 layers? How will I know where the bug is? What if the bug is in a library? I better write it all myself in C. (Note that this is the correct way to make a tongue-in-cheek statement).

All and all, let me put it this way. We all work in multi-national environments. What if each developer documented their projects in their own native language, because lets face it, they are most productive in that language. Where would we be? Doing what’s best for oneself isn’t always best for the big picture

This last point about documentation doesn't even make sense. You write for your target audience. The compiler/interpreter is the intended audience of a programming language. You write Ruby code for the Ruby interpreter, Java for the Java compiler, etc.

The whole thing is absurd. You might as well say that SQL is a waste of time. The simple fact is that modern programmers use multiple languages in the same project with regularity, and the only ones who have trouble are the poorly trained & ignorant (who can hardly be called master of even the single language they know), and the old fogies, like Burke, who consider learning something new a waste of time.

Buy my House!

2009-04-02T21:09:00.004-05:00

My apologies for posting something almost personal, but this is mostly as a note for myself.

If you live in the Austin in area and are looking for a good house, mine is for sale.

When to go dynamic

2009-03-10T11:41:00.006-05:00

The most typical complaint I hear from Java developers about Groovy is, "I spent X hours trying to figure out this problem that would've been a compile time error in Java." And they're right. Very little compile time checking is done in Groovy. It can't be done because methods may be available at runtime that aren't resolvable at compile time. That's part of what it means to be a dynamic language.

Some developers seem to think using the optional types will save them. e.g.,


int foo (String bar) {
 return bar.size()
}

Looks good, right? Try running this script:


int foo (String bar) {
 return bar.size()
}
if(false) {
 String s = foo(123)
 println s
}
println "Done"

Go ahead. Run that script. No error. Types aren't checked at compile time. If we change false to true, we get a groovy.lang.MissingMethodException at runtime, but that's it. Your compiler can't save you here. Good unit tests will catch some errors if you use the optional types, but you have to know what the error is before you can test for it, so you've already spent the X hours trying to track it down.

What makes Java so good for enterprise work is the static types. Static types enforce a contract. Static types make it possible for Eclipse to take you to the exact method that will be invoked by a particular line in the source. If the method is on an interface, Eclipse can tell you all types that implement that interface method and take you to any implementation you choose immediately. This is extremely useful when you're trying to navigate a complex, unfamiliar system. While a human may be able to easily figure out what file to open, having a tool that can do it for you automatically allows your concentration to remain on what the code does and lets you work at a higher mental level of abstraction.

If I have to learn a new framework that someone has written, I'll get through it 10x faster if it's in Java. No/bad documentation? Not a problem, I'll just look at the code for that method. Whereas if I come upon an undocumented Groovy method, what do I do? Ugh, I have to grep through all of the source to figure out where it's defined, how it's used, yada yada. Not quick. Breaks my concentration.

Java is very good at making large systems manageable. However, the trade off is that
any system that has been under active development for more than 2 weeks quickly becomes a large system. There is nothing concise about Java. There are very few shortcuts. The simplicity of the language and the rigidity of types means you have to write more code and produce more classes to get things done and God help you if you want anything remotely flexible. Code that needs to do something even slightly dynamic can quickly become unreadable and strewn with all kinds of reflection objects, odd exceptions and other nastiness. Or worse. You could end up in XML hell.

Enter Groovy. Dynamic languages do best when there are conventions and consistent ways of doing things. Grails is proof of this. Controllers are in the controllers folder, services in the services folder, etc. Each class follows a certain arechtype and there are good idioms in place that imply what is going on. However, when you start writing plugins, or doing things outside of the normal scope of Grails, things can get messy fast.

Anytime you're coding in uncharted territory, consider going back to Java. Writing a plugin? Implement as much of it as you can in Java. Only go back to Groovy if the equivalent Java code gets too ugly or would be many times longer. Most of your plugin/framework/new invention's plumbing should be in Java. The plumbing is where you connect everything together, and it's important to verify that thingDoer is of type ThingDoer, and that all interceptors properly implement the ThingDoerInterceptor interface. You want your API to fail fast so your users don't end up with a NPE exception deep in your code that they have no hope of debugging.

Keep Groovy at the fringes, handling the nasty implementation details, and protected by static types and good assertions. Make sure your Groovy class implements an interface, or better yet, have a Java class implement the interface and wrap the Groovy implementation so you can jump to it in Eclipse. Use Java to provide enforce the type contracts and provide a well defined framework. Use Groovy to make the implementation readable.

Mock object are making your tests stupid

2009-02-12T20:15:00.005-06:00

Mock objects were created because when you're writing unit tests (especially in a statically typed language), it can be a pain to create a subclass/implementations to use for your tests. You need a mock whenever the class you are testing depends on another class. You want to guarantee that changes to the other class don't break your tests for this class, because that would defeat one of the purposes of unit testing: isolation of the failure to a particular class/method. So you want an object that, whenever getTemperatureOutside is called, returns 32 degrees.

Sometimes you also want to make sure that the method you are testing causes some side effect. e.g., if the temperature outside is less than 32 degrees, it should call deployIceScraperNarwhals(). So, you want a method that records that deployIceScraperNarwhals() was called (but doesn't actually deploy the Narwhals) and something that checks that it actually was called, failing the test if it wasn't. Thus, the idea of verifying method calls was inextricably bound to the idea of providing fake methods to isolate behavior and every genius-pants writing a mock object framework decided that the verification was the most important part, to be emphasized throughout the documentation.

Then you came along, read the mock object documentation and proceed to go and write unit tests like this (mock object syntax is made up):


class UnderTest {
 int calculate(Foo foo,Bar bar) {
    int a = foo.getA()
    int b = bar.getB()
    return a + b
 }
}

class UnderTestTest {
  void testCalculate() {
     Foo foo = createMock(Foo).getA().andReturn(3)
     Bar bar = createMock(Bar).getB().andReturn(2)
     assertEquals(5,instance.calculate(foo,bar))
  }
}

Honestly, I'm looking for the stupid thing you did here, and I can't find it. The test passes, you didn't have to write MockFoo and MockBar classes, and all is right with the world. Until you decide to make an optimization:


class UnderTest {
 int calculate(Foo foo,Bar bar) {
    if(foo.getA() == bar.getB()) {
      return foo.getA() << 1 // a+b = a*2 = a left shifted 1 bit     
    }     
    int a = foo.getA()     
    int b = bar.getB()
     return a + b
 }
}

Ignoring the fact that this probably hurts performance instead of helping, lets rerun the test and see what we get (error message made up because I'm lazy): "Unexpected call to getA." That's right, you only told it to expect one call to getA, not two. You didn't change the result, you didn't remove an important side effect, in short, you did nothing that should break a test. Everything is still working perfectly. But it broke the test.

This is the part where I get to tell you you're an idiot. Instead of realizing that what you really want is a stub object, you decided that you must have written the test wrong so you rewrite it to expect between 1 and 2 calls to getA(). Because these mock object things can be tricky, you proceed to write all of your unit tests from then on by examining the source of the method you are testing, creating a mock for each dependency, adding a method call for each method call in the source, and basically rewriting the method in mock form. You idiot. You just wrote a test that confirms that the method was coded a particular way, not that the method behaves correctly.

Here's an example:


class OtherUnderTest {
 /**
  * @return a Map with a result entry, which is a message to the user.
  */
 Map deployNarhwals(Foo foo,Bar bar,Narwhals narwhals) {
    if( underTest.calculate(foo,bar) < 32 ) {
       narwhals.deploy()
       return [result: narwhals.getStatus()+' narwhals deployed']
    }
    return [result:'ok']
 }
}

class OtherUnderTestTest {
  void testDeployNarwhals() {
     Foo foo = createMock(Foo).getA().andReturn(3)
     Bar bar = createMock(Bar).getB().andReturn(2)
     Narwhals narwhals = createMock(Narwhals).getStatus().andReturn('Deadly')
     narwhals.deploy()
     Map result = instance.deployNarwhals(foo,bar,narwhals)
     assertEquals([result:'Deadly narwhals deployed'],result)
  }
}

You did so many things wrong here I'm going to have to make a list:

You mocked the parameters, but didn't stub the internal dependency. See that instance of UnderTest in there? You're not testing it's calculate method, you're testing deployNarwhals, so you need to replace it with an instance that returns something like 31 so that you'll always be testing the condition (which should be a part of your API documentation) "if the temperature outside is less than 32 degrees, the narwhals will be deployed." Stupid.

You verify that the side effect occurs, but you also couple your test to your implementation.What if I decide that the user doesn't need to know the Narwhal's status, I change the wording of the message, or make an extra call to getStatus()? Broken test. But the test shouldn't break just because your marketing department decided that 'released' was a more green term than 'deployed', or you cleaned up your code and change the number or order of non-side-effect-causing method calls.

You check for equality on a MapDo you really care if there are extra keys in the Map? Probably not. Instead, check that all the key value pairs you expect are there.

This is how you should have written your test:


class OtherUnderTestTest {
  void testDeployNarwhals() {
     boolean deployed = false
     Narwhals narwhals = [
        getStatus: { return "Deadly" },
        deployNarwhals: {
          deployed = true
        }
     ] as Narwhals
     instance.setUnderTest([calculate:{ return 31 }] as UnderTest)
     Map result = instance.deployNarwhals(new Foo(),new Bar(),narwhals)
     assertTrue(deployed)
     assertNotNull(result.result)
  }
}

You could use a mock object framework for the Narwhals object, but you should configure it to only require the call to deployNarwhals, and not to fail when other methods are called. The above test will not fail unless the API contract is broken. I only assert that the expected side effect occurred, and that a non-null message was returned.

The problem is that you morons seem to think that more assertions == better test. That's simply not true. A good test covers as many cases as possible, but only asserts what is defined in the API contract. If you start asserting things that are not part of the API, like getA() is called, you make the test brittle. Even worse, someone could come along and see your assertion and think that getA() being called is part of the API and code accordingly! Now you've introduced a bug somewhere else in the code that your tests wont catch and could become a complete mess if it isn't caught quickly. The reality is you should assert only what you have to to catch a bug. Sometimes tests don't even need assertions, or it's not worth the effort to write them. It's enough that they show that the code under test ran without error, which is no small matter. Knowing that retrieveNarwhals(null,null,null) doesn't throw an exception can be very important.

Now, get back to work and try to stop writing brain dead tests. You thick ninny.

EDIT: Here's a good alternative: Mockito.

Kill the Singletons. (And those things you thought were Singletons too)

2009-01-30T11:51:00.000-06:00

You guys think you're so smart. You read something about design patterns once, and now you can justify any code you write by showing the pattern it implements. Except you only remember the Singleton, and a few others. You probably remember the Factory, because you usually implement it as a Singleton. Maybe you remember the Facade and/or the Adapter pattern. You may even have some code somewhere that implements the Visitor pattern, probably where it was not at all appropriate. But Singleton is the one that everyone knows. Even those of you who never even read a poorly written article on design patterns. And you use it everywhere.

Or you think that you do. Probably you're using it as a Factory or a Locator. Or you have a class with static methods that you use to load resources or something like that. Those are singletons right? No, but you thought they were. There are even some clever and occasionally useful Singletonish things implemented using ThreadLocals, but that's for later. The point is, you've used Singletons as an excuse for what amounts to a bunch of global variables and methods. Bad programmer! No copy and paste for you!

The real problem is in how you're using them. Singletons can't (easily or at all) be proxied, decorated, replaced or mocked. You're losing all kinds of flexibility, without gaining any sort of benefit. This is your code:


public void doSomething() {
MyConfig config = MyConfig.getConfiguration();
config.doSomething();
// etc....
}

This is your code on inversion of control:

private MyConfig config;

public void doSomething() {
config.doSomething();
// etc....
}

public void setConfig(MyConfig config) {
this.config = config;
}

Of course, somewhere else you need to myClassThatDoSomething.setConfig(configInstance);, or have your container do it. This pattern is also called dependency injection, but unlike the Singleton, it's easy to get right. Don't call static lookup methods (or any lookup methods if you can avoid it) and make everything you need an object property. It's a simple change, but it gives you tons of flexibility. The only downside is that you have to learn how to use an IoC container. Fortunately, they're getting more idiot proof by the day, so with a little work, you can probably figure it out.

Now, before you go and delete every last one of your singletons, there are a few situations where IoC wont work. For IoC to work, you have to have control of the creation of the object, or at least be able to get a reference to it before it's used. A good example is logging. Logging has to be available before the container even initalizes, so you can't dependency inject logging components (easily). And sometimes, the thing you need to inject can't be managed by the container. For example, suppose you need access to the current HTTP request object, or the session object? Ignore for a second that your container can usually mange to inject special objects like these and recognize that what makes these object special is that, in a particular scope, there is only one of them at a time. By their very definition, there can only be one HTTP Request and one Session at a time. To have two requests, or two session doesn't make any sense. That means you actually have a good candidate for a singleton. A filter like this can be very helpful (some types & casts omitted for brevity):

private static ThreadLocal request = new ThreadLocal();

public static HttpRequest getRequest() {
return request.get();
}

public void doFilter(request,response,chain) {
request.set(request);
try {
 chain.doFilter(request,response);
} finally {
 request.clear();
}
}

That lets anything that's executed inside the filter call MyFilter.getRequest() to get the current HTTP request object, regardless of whether the framework was kind enough to pass it in. Even this should only be a last resort. There may be a legitimate case where someone needs to wrap, replace or mock the request, but your singleton code gets in the way. How will you unit test code that accesses the request in this way? There is no real HTTP request when you're running unit tests. Dependency injection solves a lot of problems.

Sometimes you can't avoid the Singleton, but considering the mess that you've made of things so far, maybe you should ask someone smarter than you next time you think you really need a Singleton. You probably don't. Now get back to work, and when you have some time, get the original (and best) design patterns book, read it, and stop abusing them. Design patterns are meant to describe good code, not magical pixie dust that you sprinkle on to make your crappy code suddenly good. Alas, that is another post.

JS Tips & Tricks

2009-01-29T21:30:00.004-06:00

http://javascript.crockford.com, Secrets of the JavaScript Ninja and JavaScript: The Good Parts will cover almost everything you need to know about JavaScript. Still, there are a few tricks that I use almost every day that I feel are often underrated:

&& and || aren't your father's boolean operators. Thanks to JavaScript's notion of truth (anything but false, null, 0, undefined and the empty string), they don't have to return boolean value, and will not unless you coerce it into one with !!. That means you can use them inline as part of assignments and in parameter lists. e.g,
```
function(someParam) {
 // provide a default value
 someParam = someParam || {};
 // ...
}

// guard against null objects
var count = foo && foo.count || 0; // if foo is null, count == 0, else foo.count
```
function scope:
```
(function($){
$('#foo').doSomething();
// ...
})(jQuery);
```
The key here is that you're taking a global/wider scoped variable and making it a local variable, which allows you to rename/replace the original variable with little to no impact to your code. For example, suppose you need to use 2 versions of jQuery in your page. If you have the good sense to wrap all of your scripts in a function, you can include jQuery version A, then include your scripts that need version A, then include jQuery version B and your other scripts. Or if you wrote all of your scripts using $ as a reference to jQuery and then wanted to use Prototype in the same page.

The other nice thing about wrapping your scripts is that any vars and function your define wont bleed into the window scope unless you specifically assign them to window. e.g., window.GLOBAL_FOO = 123;

Sorry for the short post. I have some more code heavy stuff I think I'm working on for next time, I promise.

Quit being lazy. lern2javascript noob

2009-01-28T15:21:00.002-06:00

[Note: This is a blog post from a year ago, but I'm reviving it because I'm seeing developers and companies lose work because they don't have JavaScript experience. If you work on web applications, you need to learn JavaScript. In addition to this post, which targets Java developers, I've added links to good getting started resources at the end.]

Prologue: Why you need to learn JavaScript

For the foreseeable future (probably the next 5-10 years), the majority of software development will be done on web applications. Many of these will be ports of existing desktop applications or designed to replace such applications. Developing desktop-like applications on the web is going to require that Ajax thing you've been hearing about, something called DHTML, DOM, etc. That means you'll be using JavaScript.

We Java developers have been conditioned to flee at the first mention of JavaScript which has meant that it has long been the job of the 'creatives' to do the JavaScript work. That's not going to cut it anymore. While the work UX experts do is crucial to developing any application, they have different training, priorities and experiences. RIAs need solid technical developers who understand best practices, patterns, unit testing, etc. and can apply them to JavaScript.

One way Java developers try to avoid having to deal with JavaScript is through frameworks like JSF or GWT. There are certainly advantages to using a framework, but they are no substitute for being comfortable with JavaScript. Developing RIAs through a framework without knowing JavaScript is like using Hibernate without a solid understanding of SQL and relational databases. You may get by, but it wont be easy and it wont be pretty.

Hopefully you're now convinced that JavaScript is in your future. I want you to know that you don't have to be dragged into it kicking and screaming. It will be fun. You know those closure things that they're talking about putting in Java 7? JavaScript has always had them. Ever get tired of writing the same Java code over and over? JavaScript code is far shorter and less repetitive.

Differences: The Good, The Bad, and The Ugly

I'm going to focus on the immediately important differences for now. We'll pick up the more subtle ones as we go.

JavaScript, like Java, has a C like syntax so you should feel pretty much at home. It has fewer keywords than Java, but adds a few new ones. The most important one is function, which we'll talk about more in the next chapter, but there's also var(declares a variable), with (to generally be avoided), and in (multiple uses).

JavaScript is loosely (weak) typed. For a UI, that's a good thing, because most of the data you're working with is string data, so you don't have to declare String everywhere and if you need a string to occasionally be a number, it happens automatically. Every object has a boolean value of true except 0,the empty string and false. An undefined or null variable also evaluates to false, so you can check to see if an object has the method/property that you want like so:
```
if(!foo.bar) {
 throw "foo must have bar";
 // you can throw anything to raise an error
}
```
Of course, you can do whatever you like if the object you were passed doesn't have a particular method.

There are only Numbers in JavaScript. There is no distinction between int,float, double and long. Math.round, Math.floor and Math.ceil are there if you need them. The maximum value is 1.7976931348623157e+308 so don't worry too much about overflowing. You shouldn't be doing any heavy computation on the client anyways. If you try to do arithmetic with something that isn't a number, the result will be NaN (not a number). NaN and anything is NaN. That includes undefined variables:
```
var foo = 1;
foo + bar; // == NaN, since bar is undefined
```
There is no char type. You can use either single or double quotes to delimit a string. 'f' is always the string "f". charCodeAt(index) on strings will give you the character code as a number.

In addition to functions, objects and arrays are first class citizens (technically functions and arrays are also objects). You can define array literals using the [] syntax: var myArray = [1,2,3];. For objects you use {} and provide a list of name:value pairs: var myObj = { foo: "bar", baz: [1,2,3] }. Both array and object literals can span multiple lines. It's generally good form to break lines after a comma (if you must break lines):
```
var foo = {
 bar: "baz", qux: 123,
 quxx: { a: 1, b: 2, c: 3}
};
```
Perhaps the most important JavaScript data type is the function. We wont get into it now, but you need to know that you can treat functions like any other variable. e.g.,
```
var foo = function(a,b) {
 return a + b;
};

foo(1,2); // == 3
```

When a function is called as a property of an object, it gets access to this. e.g.,

var foo = new Object();
foo.bar = function(a) {
 return a + this.b;
};
foo.b = 2;
foo.bar(1); // == 3
foo.bar(2); // == 4

Note that while you can get put a reference to foo's bar function in a variable, it will no longer have access to foo via this.
```
var bar = foo.bar;
bar(2); // == NaN (not a number)
```
Since this.b is undefined, 2 + undefined is NaN. You might be wondering why this.bar doesn't result in an error. Well, it's because this is always defined. Since JavaScript executes in the context of the browser window, the default this is the window. This is where all global variables live. Incidentally, you can reference the window explicitly with the window object.

There is a difference in scoping in JavaScript as well. You're used to having block local variables. Well, in JavaScript variables are either global or function local. Blocks share their enclosing scope, so
```
var foo;
if(true) {
 foo = 1;
}
```
is the same as
```
if(true) {
 var foo = 1;
}
```
foo will still be visible after the else block. If you fail to define a var inside a function, a global variable is created (or if it exists it is overwritten). e.g.,
```
function() {
 foo = 1; // sets window.foo = 1
}
```
So, think carefully about scope when you declare and use variables.

JavaScript is interpreted, which means that you don't even know 100% if a line of your script is valid until it actually runs. That makes unit testing crucial. Fortunately tools like JUnit, Javadoc, etc. all have JavaScript equivalents that will seem very familiar. There are also a host of development tools including Eclipse plugins, Aptana, IntelliJ support, etc. to help you along.

Lastly, you should know that JavaScript inheritance is prototypal. It is fully possible to achieve class based inheritance, which we will do with the help of a library called Prototype, but the prototypal nature of JavaScript gives us some interesting options which we will explore later.

Your Environment

We're Java developers, so we know the power of a good IDE. Eclipse, NetBeans and IntelliJ all have decent JavaScript support. There are a variety of plug-ins for Eclipse (I use JSEclipse), but I'm going to tell you right now that none of them will give you the code completion that you're used to. It's simply not possible with a dynamic, weakly typed language. You'll have to learn APIs the old fashioned way, by reading the docs and using them.

Fortunately, JavaScript gives you an amazing debugging and exploratory tool that you have never had and likely will never have with Java. Its name is Firebug. Go ahead and install it. Done? You need to restart Firefox. OK. Good. Now see in the bottom right corner? Click it. This is Firebug. It does a lot of things, but for now we're just going to mess around with the 'command line'. Do the following:

Click the in the bottom right corner of Firebug.

Paste this into the command line and press Ctrl+Enter or click 'Run':

$('demoSteps').childElements().each(function(item,i) {
 item.insert("("+i+")");
});

OK, it's not the most impressive example, but you did just execute code without having to reload the page or anything. Here we're making use of a library called Prototype. Bookmark the API immediately. What the above code does is call a function named '$' (yes, that's a valid variable name in JavaScript) with the argument 'demoSteps'. When $ is given a string, it looks for the element in the HTML with id="demoSteps". We get the child elements and iterate over them, calling the anonymous function we defined there on each element, along with it's index. We then append the index in parentheses to the body of each element.

The equivalent JavaScript without Prototype is a lot longer. You could start learning JavaScript without any libraries, but then you'd quickly get caught up in fixing bugs and writing the same code over an over. Fixing bugs doesn't give you a good appreciation for a language, nor does clunky repetitive code. In any serious web project you will use Prototype or a similar library, so it doesn't hurt to know it. After you're comfortable, we will delve deeper into the JavaScript language and you'll learn what Prototype hides from you.

Before we go, drag this onto your bookmarks toolbar: Insert Prototype. Now you can go to any web page, click that bookmarklet, pop open the console and start using Prototype to play. For homework, explore the Prototype API, especially the utility functions and the Enumerable methods. Try to write as many snippets using the API as you can.

Resources

http://javascript.crockford.com/ Read these essays. They're all pretty short, but they are essential. Also, get a copy of Crockford's book.
http://jquery.com/ jQuery is very popular. It has tons of plug-ins, with new ones popping up everyday. You will need it.

You don't know what a controller is for, do you?

2009-01-26T13:13:00.000-06:00

This is my second post in what I'm going to call the "You deserve verbal abuse because your code is dumb" series. These are things that frustrate me greatly, like creating spaghetti code because you don't understand CSS. (In fairness, my solution may not be obvious, but there is no excuse for highly fragmented spaghetti code in the view layer; someone should have come up with something better).

Last time I talked about how you can use CSS to get rid of most of that if/else logic you have in your page templates. Today, I'm going to explain controllers to you. That's the C in MVC. You should know this stuff. The controller is the thing that receives the HTTP request and decides what to do with it. Most frameworks will map URLs to different controller classes and methods. The controller is responsible for picking the view to render and loading the right data based on the request (and maybe the URL).

Most of you seem to think that it ends there, and that it's OK to take whatever convulted data structure your service layer spits out and pass that right on to the view. Lean in really close so I can hit you. No! Bad developer! The whole reason that we moved away from CGI/Servlets is because mixing the processing code and the view markup is a bad idea. Unfortunately, you Philistines didn't get it, so you invented crap like Struts that just puts a layer of abstraction around HTML code and then get yourselves back into the same mess.

This is what your templates look like:

<div id="${result.product.productInfo.productId.toString()}">
<h4> ${result.product.productInfo.productName.getLocalizedString(request.client.language)} </h4>
etc....

and here is how they should look:

<div id="${productId}">
<h4> ${productName} </h4>
etc....

The controller is responsible for getting and creating a model that makes sense for the view. Even the most technically averse designer can look at a view with a smart controller and figure out what's going on and how to edit it. It's when the expressions start to get long and the if/else logic creeps in that they start to get confused and scared. So, have pity on the designers, keep the logic in the controller. If you're careful, maybe someday the designers can get in there and actually change the templates themselves, instead of having to file a bug report for every single HTML change. Wouldn't that be nice?

One liners for readability

2009-01-23T12:16:00.011-06:00

(For clarity, when I say one liner, I mean single statement. For various reasons, including readability, it may be desirable to split statements across multiple lines)

Which is easier to read? Is it this multi-statement snippet:

def params = request.params
def config = whatever.getMyConfig()
def request = MyLongTypeNameRequestTransformer.toMyRequest(params, config)
def result = myLongNamedServiceThatDoesSomethingCool.execute(request)

or this single statement:

def result = myLongNamedServiceThatDoesSomethingCool.execute(
  MyLongTypeNameRequestTransformer.toMyRequest(
      request.params, whatever.getMyConfig ) )
// line breaks added to protect the page-width. 80 columns

Now your answer to that question probably depends on how you interpreted 'easier to read'. Most of you probably thought that I meant "Which format makes it easier to understand what this snippet does?" Typically people are going to say that the first one is more clear in that respect. However, I want to consider a different question, "Which format makes it easier to understand/follow/debug/grok/refactor the overall codebase?"

I would argue that the second form is better in that respect, and that overall manageability of a codebase is generally more important than a small readability improvement for a section of code. I see one liners as an implied abstraction. It hints to the reader that, despite the complexity involved, we are really only interested in one result (either a return value or a side effect), so they can mentally collapse all that code into "get X" or "do Y". It says, "Skim over me, the details are not important."

There are two alternatives to one liners. The first is to use a bunch of local variables. While this can certainly make an algorithm more readable, each one adds one more thing that I have to keep track of when I'm reading your code. I have to remember that config came from whatever.getMyConfig(), and I have to consider that it may be used later in the method. If config is inlined, I don't even have to mentally register its existence unless I'm trying to parse the one liner.

The second alternative is to encapsulate the snippet as a function. This gets rid of the mental scope pollution from stray variables, but it introduces two new problems. Firstly, if I want to know what the function does, I have to go to another place in the file (or another file), which breaks the continuity. Often times one liners also need several of the variables in the current scope, so you end up with a lot of parameters, which adds even more clutter. Secondly, you end up poluting the scope of the class or script with yet another function that is only used in one place. You've just moved the clutter up a level.

Now, there are certainly times and places where you don't want to inline everything all willy nilly. If an expression appear multiple times in a function (e.g., whatever.getConfig()), don't repeat yourself, use a local variable. If you can extract a function that can be used in more than one place, go right ahead. If you have code that is several indentation levels deep, first consider a redesign and/or refactoring that simplifies, but if that doesn't work, extract a function to make it more readable. However, declaring variables and functions should be things that you do out of a clear need, not just because you think you'll need it, or because your CS professor always told you to (CS professors are concerned with algorithms and minutia, not codebase managability). Remember, premature optimization is the devil.

Most modern languages offer syntax that can assist in creating readable one-liners. If your language has map/has and/or list literals, you should avoid building those constructs procedurally. e.g., in JavaScript:

var array = new Array();
array[0] = 'foo';
array.push('bar');
myMethod(array);

is lame. Write that as

myMethod(['foo','bar']);

Groovy's map literal:

myMethod([foo:'bar',baz:'quxx']);

Even Java can be slightly improved thanks to anonymous inner classes and initializer blocks:

myMethod(new HashMap() {
{
 put("foo","bar");
 put("baz","qux");
}
});

Hopefully you have the idea by now.

Whadya know, this semantic markup crap is actually useful

2009-01-20T14:31:00.002-06:00

why you should use a rules engine

The UI of a web application generally consumes 60-90% of the development effort. UIs are complex, and they're big. All the cross cuttng concerns of the application combine in the UI to give you as many versions as there are variables. For many sites that means there can be hundreds of unique ways for a page to be rendered. As a consultant, I see a lot of you writing templates that look like this:

<div id="product">
<c:if test="${hasErrors}">
 <div class="errors">
  Bid must be a number
 </div>
</c:if>
<c:if test="${forSale}">
 <input class="buyButton" value="Buy!" type="button">
</c:if>
<c:else>
 <span id="saleMessage">Sorry, not for sale.</span>
</c:else>
<c:if test="${isAdmin}">
 <input value="Delete" class="deleteButton" type="button">
</c:if>
</div>

You get tag/conditional soup, because the there are a bunch of different states that the page could be in, and you have to show/hide lots of things, or give things different colors, or whatever based on whatever state the page is in. Or maybe you list several things on a page, and some of them will be in different states than others. Or you have some button that the user isn't allowed to click because they don't have the right role.

If you have tag soup, you're doing it wrong. All of the things I described above are state, or semantically useful information. With the exception of hiding information from those who are not allowed to see it, there is absolutely no reason not to render just about everything and use CSS rules to hide and otherwise transform the page based on the current situation. Here's an example:

<div id="product" class="user${userRole} ${forSale ? '' : 'notForSale'} ${hasErrors ? 'error' : ''}">
<div class="errors">
 Bid must be a number
</div>
<input class="buyButton" value="Buy!" type="button">
<span id="saleMessage">Sorry, not for sale.</span>
<input value="Delete" class="deleteButton" type="button">
</div>

And you have CSS rules like so:

.errors, .deleteButton, .notForSale .buyButton, #saleMessage { display: none; }
.error .errors { display: block; }
.notForSale #saleMessage { display: inline; }

You can't tell me that isn't simpler. It's easier to read too. All those business rules that you have translate nicely to CSS rules. Imagine that, rules begin easily represented as rules. This approach combines nicely with Ajax as well. Receive an update from the server and suddenly the item is for sale? $('#product').removeClass('notForSale'); and the browser takes care of the rest.

You might tell your self such an approach would never fly, what about security? Anyone could look at the source and see the delete button, or use Firebug to show it, then suddenly they're deleting things! Hopefully you understand the problem with that argument, but for the naive: if you're not checking permissions and validating data on the server, you're wide open to all kinds of attacks. Your rendered HTML is just the preferred interface. The real interface to your application is HTTP, and an attacker will quickly figure out what a real delete request looks like, so unless you're also checking credentials, you're screwed.

The exception, as I said before, is information that needs to stay secret. You shouldn't render some element with user passwords and hide it with CSS. Don't do it with trade secrets, launch codes or your secret pictures of yourself dressed as a drag queen either. For anything else, take advantage of one of those things about semantic markup that actually works well, and make your life simpler.

Next time: You obviously don't understand what a controller is for.

But first: Writing one liners; for readability.

'Opt-In' Changes and the Importance of Testing Suites

2008-08-27T16:45:00.000-05:00

A development mistake I see all the time is when a developer fails to make changes to core classes 'Opt-In' or backwards compatible. For example, suppose you find a bug in your table/grid widget that causes one of the several instances of that widget to fail mysteriously. You come up with a solution that involves an extra method call or two at a certain point, or changing an option/property to another value. Your first instinct might be to make that change the default behavior, but in most cases, that's wrong.

Minimize Affect

Summary: How many instances of that widget do you have in your application? Are they all failing? Why not? Were they functioning properly before? If they ain't broke, why fix 'em?

Most likely the change you are going to make really should have been the default behavior all along, however, you now have working code that was developed and presumably tested against the old behavior. If you change that behavior now, even if it is now the 'right' behavior and was previously the 'wrong' behavior, you now need to test every single instance and subclass of your widget in all possible scenarios to ensure you haven't introduced a bug. Do you have a comprehensive automated testing suite? If you do, great, but you probably don't.

Even if you do have a good test suite, why fix something that isn't broken? Suppose, instead of making that change the default, you instead add a useFooHack property to your base class. It defaults to false, but if it's true, you use your new behavior. When you discover that other instances have a problem caused by the old behavior, you just change the flag to enable Foo, and it's fixed. Easy as that. If you discover that every last instance has the problem, you can remove the useFooHack=true line from your instances and if(useFooHack) from your base class. Simple. The alternative is to risk introducing bugs into every instance that you thought was working.

Testing is a waste of time

Now some people will learn of problems like this and say you should have done more testing before committing. You know, you should have gone through all the major paths of your application to ensure that they're working before committing. Most of the time those people are managers who don't have to waste time doing the testing themselves, but expect you to do it.

Suppose a developer spends X time testing before a check-in and finds some bugs that take Y time to fix, it has cost him X+Y time. However if he instead verifies that his code does what he intended it to do and checks in, but someone discovers the bugs, emails him, and he spends Y time fixing it, it only cost him Y time. Now if X is small, say 30 seconds, because all the developer had to do was kick off the automated testing suite, then the developer will do the testing to avoid getting angry emails. However, if testing takes more than a few minutes, most people will realize they can get more work done by letting someone else hunt for bugs. You can argue about it all you want, you're not going to change human nature, and the optimal choice most of the time is to skimp on the testing.

In short, you have to work with people the way they are. No amount of training or lecturing will keep people doing things that are not in their own best interest. If your project's big problem is that people keep breaking the build, make not breaking the build the easy path. You may have to sacrifice some hours to developing and maintaining a test suite, but you will generally save more hours of lost productivity from broken builds.

Java Fun Continued - Groovy FTW

2008-08-20T17:28:00.004-05:00

Here's the equivalent Groovy for my previous Java abomination that prints the Java wrapper methods for me:


new File("src/main/java/Foo.java").text.
    replaceAll(/(?sm)\s*public\s+(\S+)\s+(\S+)\((.*?)\)\s*\{/) { 
       s,ret,name,params ->
  print """
    public ${ret} ${name}(${params}) {
        """
  if("void" != ret) {
    print "return "
  }
  print "foo.${name}("
  // -1 hack avoids index out of bounds when params is ''
  print params.trim().split(/\s*,\s*/).collect { 
    it.split(/\s+/)[-1] 
  }.join(',')
  print """);
    }
"""
}

I could change the prints to concatenation and reformat the whole thing as a single line, so it only counts as one statement. Technically I cheated a little because this version doesn't look up the methods by reflection, but it doesn't make a difference in the result, and wouldn't significantly shorten the Java code. Definitely more readable than the Java. The JavaScript is a little messier since you don't have multi-line string literals, GStrings, and closures are function(it){ return it.foo; } instead of { it.foo }.

Java Fun: When all you have is a hammer...

2008-08-20T12:24:00.004-05:00

I ought to be ashamed, but I was bored and lazy, so I thought I'd test my somewhat dormant Java skills. Can you tell me (without running it) what this code does?


  Method[] methods = Foo.class.getMethods();
  String source = FileUtils.readFully(new FileReader(
    "src/main/java/"
      + Foo.class.getCanonicalName().replaceAll(
        "\\.", "/") + ".java"));
  PrintStream out = System.out;
  for (Method method : methods) {
   Matcher matcher = Pattern.compile(
     "public (\\S+?) " + method.getName() + "\\((.*?)\\)",
     Pattern.DOTALL | Pattern.MULTILINE).matcher(source);
   if (matcher.find()) {
    String ret = matcher.group(1);
    String params = matcher.group(2);
    out.printf(" public %s %s(", ret, method.getName());
    out.print(params);
    out.print(") {\n  ");
    if (!"void".equals(ret)) {
     out.print("return ");
    }
    out.print("foo.");
    out.print(method.getName());
    out.print('(');
    if (!StringUtils.isEmpty(params)) {
     out.print(StringUtils.join(CollectionUtils.collect(
       Arrays.asList(params.split(",\\s*")),
       new Transformer() {
        public Object transform(Object input) {
         return ((String) input).split(" ")[1];
        }
       }), ','));
    }
    out.println(");\n }\n");
   }
  }

I'll post the equivalent Groovy (and maybe JavaScript) later. I imagine I can do it in a single statement function. I don't miss Java very much.

New (to me) Bike: Issues letting go

2008-08-20T08:04:00.002-05:00

My friend John is letting me borrow his fancy Specialized bike that he doesn't ride anymore. I'd say it's about a third of the weight of Old Creaky, my bike that I built at the Yellow Bike Project, and is definitely nicer in every way, but I'm having some problems letting go.

Firstly, this thing is a true road bike, everything is built for speed. The problem is that the tires have little to no tread on them (by design) so if it's remotely wet or gravelly, I make myself slow way down because I'm afraid this thing will slip out from under me. Old Creaky had some decent tread, but not too much. Just something I need to get used to I supposed.

Secondly, the shifters are those neat kind that are built into the brakes, so naturally they're indexed. I've been riding for the past year with friction shifters, so I'm used to being able to go from 5th back to 1st in about 2 seconds just by pushing a little lever down. With these indexed things, I have to click, click, click, and then I'm not sure where I am because they don't have any sort of indicator, so I'm constantly looking down to see where the chain is. I suppose I'll get used to it, I just hope I don't get hit by a car in the meantime.

Lastly, John is about 6'5", and although I'm 6'3" Old Creaky was not built for a 6'3" person, while John's bike is built for John, so I'm bent over a lot further than my back is used to. I'm sure I'll get used to it too.

So, three things to beware of when you get a new bike. I wouldn't keep riding it if I didn't think it was a net positive, but I also never thought any of these things through. I'm working on my fear of change.

Cisco VPN Client Shell Script on OS X

2008-06-17T13:29:00.005-05:00

I like to start and stop the VPN client from the command line, but sometimes, for whatever reason, the VPN service isn't running or isn't responding. I then have to go Google it to find the incantation to restart it. Rather than do that every time, I use this script to connect to the VPN:


CMD="sudo vpnclient connect MyProfileName user myUserName pwd myPassword"
yes | $CMD
if [[ $? -eq 8 ]]
then
    # exit code of 8 means it couldn't connect to the vpnd
    sudo /System/Library/StartupItems/CiscoVPN/CiscoVPN restart
    yes | $CMD
fi

Static vs. Dynamic: The Line

2008-05-15T08:56:00.003-05:00

If you read many programming blogs, you've probably heard about the recent rekindling of the static vs. dynamic debate. In particular, it started with Steve Yegge's Dynamic Languages Strike Back, followed by Cedric Beaust's Return Of The Statically Typed Languages and finally Ola Bini's A New Hope: Polyglotism. Ola does a good job of summarizing the other 2 and then proclaims that the correct answer is somewhere in between, using both static and dynamic languages in different situations.

As a Java developer who knows JavaScript far too well, this is something I've believed for quite some time. I'm going to go one step further and say that I have a vague idea of where the line is where it is appropriate to switch from static to dynamic. If you've done much JEE development (if you're reading this, I know you have!), then you also know too, but you may not realize it.

Think about a typical web application. Say we're using Struts, Spring, and Hibernate, but it really doesn't matter. In the view we have this mix of XML/HTML and some sort of code, be it logic tags or inline code. The type of everything is a string. We produce a big string as the output, all the inputs are strings, we validate strings. We have to do conversion in varying degrees to pass things on to the service layer, but by the time we get to the persistence layer, you better not be passing a String when you really need a Long. Did you spot it? We get virtually no benefit from types in the view, which is why no one uses Servlets anymore. Declaring that everything is a String is kind of pointless, so a dynamic language is a good choice for the view. On the other side, types help us ensure the integrity of our persistent data and let us avoid nasty things like injection attacks. The persistance tier has to be fast and it has to be correct. A UI glitch is annoying, but if we commit corrupt or invalid data, we're in big trouble.

That leaves the middle tier, which is where the blurry line between static and dynamic lies. One option would be to use a language with optional static types like Groovy (i.e., persistence layer is pure Java, middle tier is Groovy, top is JavaScript or Groovy with minimal typing). Another would be to use a library like DWR that takes care of most of the type conversions so you can have a statically typed service later but a fully dynamic view. The JVM really shines here because it gives us so many options. What languages you choose are largely going to be affected by what people know (i.e., what's easy to support) but picking the right tool for the job is important too. I think one thing is certain though, the days where developers can be effective knowing just one language are drawing to a close.

RegExs for Common JavaScript Bugs

2008-04-16T17:07:00.006-05:00

I use these 2 quite a bit:

:\s*[\[{]\s*[\]}]

This catches empty object and array literals in another object literal. Chances are you put this in a class/prototype definition like so:


Class.create({
  myHash: {},
  myList: []
});

That's roughly equivalent to declaring 'myHash' and 'myList' as static in Java. Unless you really want every instance of that class to share the same hash/array, that initialization needs to happen in the constructor:


Class.create({
  initialize: function() {
    this.myHash = {};
    this.myList = [];
  }
});

,\s*[\]}]

Catches [1,2,] and {1,2,}. The trailing comma causes IE to fail silently, which is always fun to debug. Since this is always wrong, you can change that to ,(\s*[\]}]) and do a find and replace with $1 as the replacement.

Post another regex findable bug if you know one.

A Java Developer's Guide to JavaScript - Functional Programming and JavaScript Mechanics

2008-02-25T20:00:00.000-06:00

Homework Check

Last time we introduced the Prototype library. Hopefully by now you've at least looked over the Prototype API and are semi-comfortable with Firebug. If not, stop and take a few minutes to look around and write a short script or two.

Functional Programming

Functional programming is one of those things we don't really do in Java, since Java lacks first class functions. Essentially, when you can pass functions around like any other variable, you often end up sticking them together ("composing" them) to create new functions. The result of one function gets fed into another, etc. Programming effectively in JavaScript requires recognizing that many patterns that are explicit in Java are implicit in a functional language like JavaScript.

There are several 'classic' functions that are usually the basis of any good functional programming library:

map/collect

A map function takes a value and returns a new value. It's complement is reduce/inject. Here's an example using Prototype's collect function:


var foo = [1,2,3,4,5];
var bar = foo.collect(function(a) {
 return a * 2;
}); // bar = [2,4,6,8,10]

reduce/inject

Takes two values (at least one is usually a collection, but they both can be) and combines them into one value. When called inject, the first value is usually the 'accumulator' and the second is a singular value. With Prototype, you'll use map/collect and inject on collections, but that's not the only way it is used. Map-Reduce is what powers Google searches. Here's an example:


var foo = [5,4,3,2,1];
var bar = foo.inject(1,function(a,b) {
 return a * b;
}); // bar == 120

A little explanation for this one. The first call, the parameters are (1,5), the next call is (5,4), then next is (20,3), etc. The above is how inject works in Prototype, other versions of reduce can pass two outputs of reduce instead of the previous output and the next input. e.g., it might do (1,5), then (4,3), (2,1), resulting in 5,12,and 2, then apply the reduce function on the results and so on until we have a single value. To see how map/reduce powers Google, read Can Your Programming Language Do This?

select/findAll

Used to select/find a subset of a collection. e.g.,

[1,2,3,4,5].select(function(a) {
 return a % 2 == 0;
}); // = [2,4]

all and any

Returns true if any or all of the elements in a collection match a given predicate. e.g.,

[1,2,3,4,5].any(function(a) {
 return a == 4;
}); // == true

With Prototype, if you do not pass a predicate, then the value is evaluated as a boolean. You could use this to detect if a collection contains all (or any) non-zero, non-false and non-empty string values.

There are other common functions, but these cover the conceptual core. Most other functions will build off of these in some way. As you get to know the common functions, you'll discover uses for them that are extremely powerful.

Manipulating Functions in JavaScript

Before we talk about design patterns, we need to quick take a look under the JavaScript hood and see how we can manipulate the scope and arguments of functions. Remember that when we call a function as a member of an object, the this keyword lets us access variables in the scope of that object. But if you have only a reference to a function and you try to call it, this will refer to the global (window) scope. To allow us to set the scope of a function, we can use call. Here's an example:

var foo = { a: 1};
function aPlus(b) {
 this.a = this.a + b;
 return this.a;
}
aPlus.call(foo,5); // == 6 && foo.a == 6

This would be almost equivalent to

foo.aPlus = aPlus;
foo.aPlus(5);

However, the second example will overwrite aPlus on foo. If aplus happens to be an existing variable or method, we've now overwritten it.



call

can be used whenever you know the exact number of arguments you'll be passing to a function, but what if you don't? You might say, "When would that happen?" What if you have a function that takes a function as an argument, and returns another function? This is something we do a lot in functional programming, so we need a way to keep our function producing functions DRY.

Before we discuss how you would call a function without knowing the number of arguments, we need to talk about a special variable: arguments. Inside any function, the arguments variable is automatically defined. arguments is an array-like structure that has all the arguments passed to the function. In JavaScript, the parameters you declare fo a function are just convenient labels for indecies into arguments. The JavaScript interpreter doesn't care if you call the function foo(a,b) with 1,2, or 7 arguments. a = arguments[0] and b = arguments[1], and if arguments[1] is undefined, so is b. Like an array, arguments has a length, so you can iterate over it. e.g.,

function sum() {
 var sum = 0;
 for(var i = 0; i < arguments.length; i++) {
   sum += arguments[i];
 }
 return sum;
}

Suppose we also have this function:

function double(a) {
 return a * 2;
}

and we want to create a function that takes the sum of 2 or more numbers and returns twice the result. To reuse our existing functions, we need some way of creating a function that passes all of it's arguments to sum regardless of the number of arguments passed to it. apply to the rescue! Like call, apply lets you call a function and set the value of this, but instead of taking the arguments from the 2nd argument on, it takes an array (or array-like object) as the second parameter. e.g.,

function doubleSum() {
 return double( sum.apply(this,arguments) );
}
doubleSum(1,1,1,1,1); // == 10
doubleSum(1,2,3); // == 12
doubleSum(); // == 0

We haven't made use of Prototype here yet because we haven't needed it. However, you may have noticed that we keep referring to arguments as an "array-like" object. That's because it's not a real Array and doesn't have any of the instance methods you can expect an array to have. Prototype provides a utility function for just this purpose, $A. $A(arguments) returns a true array with all of the arguments in it, so you can call all the standard Array methods and Prototype methods on it. We can rewrite sum like so:

function sum() {
 return $A(arguments).inject(0,function(a,b) {
   return a + b;
 });
}

Using apply we can even write call:

function myCall(func,scope) {
var args = $A(arguments).slice(2);  // creates a copy of arguments from index 2 on
 return func.apply(scope,args);
};

That covers call and apply. Next time we'll see how we can use what we now know about functional programming and JavaScript mechanics to implement the design patterns we know and love from Java.

Techdirt: The Formula

2008-02-18T14:30:00.002-06:00

If you read Techdirt very often, you know that they are the first to catch it when a corporation (or government, to be redundant) is trying to hang on to a business model that is losing relevance. While I often find Techdirt insightful, they too often use the formula "Why doesn't $company stop wasting resources doing $X and do $Y instead?" Where $X is something like filing a lawsuit, which often involves a lawyer on retainer or a few thousand in legal fees (i.e., a drop in the bucket of a corporate budget), while $Y is something like investing millions and billions in building new cable/telephone infrastructure. The implication is that $X is somehow keeping them from doing $Y, and that the two are comparable strategies, which is rarely the case. It would be like me saying: "Instead of whining about the flaws in corporate business strategies, Techdirt should fund startups to compete using their superior ideas."

That said, I like Techdirt most of the time and can generally count on them to be one of the first to break significant tech news, they could just use some fresh material.

jQuery Greasemonkey Template & Netflix Queue Shuffler

2008-02-09T21:11:00.000-06:00

jQuery is quickly becoming my weapon of choice to do DOM manipulation/enhancement. I'm still a huge Prototype fan for general code, but jQuery really does the DOM well.

So, when I set out to write a Greasemonkey script to randomize the order of the movies in my Netflix queue, I needed to find a way to include jQuery. The end result is fairly brief:


// ==UserScript==
// @name           
// @namespace      
// @description    
// @include        
// ==/UserScript==

/**
 * @param {Function} func the function you want to execute 
 * with jQuery. The only argument passed will be jQuery, so
 * your function should look something like this:<pre>
 * withJQuery( function($) {
 *   $(".foo .bar")...
 * } );
 * </pre>
 */
function withJQuery(func) {
  if(!unsafeWindow.jQuery) {
    var s = document.createElement('script');
    s.src = 'http://code.jquery.com/jquery-1.1.2.js';

    s.addEventListener('load',function() {
      // lots of people like to have their own $ function
      unsafeWindow.jQuery.noConflict(); 
      func(unsafeWindow.jQuery);
    },false);

    document.getElementsByTagName('head')[0].appendChild(s);
  } else {
    func(unsafeWindow.jQuery);
  }
}

// your code goes here, use withJQuery when you need it

The trick was figuring out that in a Greasemonkey script, you can't do element.oneventname = handler (you have to use addEventListener) and that once jQuery loaded, it would be in unsafeWindow, not window.

If you came here for the queue shuffle script, sorry to have bored you. I'll do another 'JavaScript for Java Devs' chapter soon.

Grails, DWR, and the dynamic web

2008-01-14T15:45:00.001-06:00

This post used to be titled "Using DWR with Grails", because the Grails DWR plugin lacked support for custom converters and creators. I wrote this patch (EDIT: Nabble screws up the patch formatting so that it doesn't take. You can download the patch here) adding this functionality and I was going to blog about how to apply and use it. However, before I posted this, I did a bit of searching, and apparently my patch got applied the same day I submitted it. It would have been nice if someone had told me they took my patch, but presumably this means that the next release will have the functionality. If you can't wait, I presume you can build from the trunk, or use my forum post. Enough of that.

I'm not big on making predictions, but this one seems pretty obvious to me. The reason there are so many Java web frameworks is that they all suck at being web frameworks. There is no clear winner in the Java space. We started with Servlets, which were terrible for web development because you couldn't tell what the heck the final HTML would look like. Then we had JSP, which is better, but once you start trying to apply DRY, get into a templating system and tag libraries, you're suddenly just as removed from the final result as you were with Servlets. In both cases, if I want to know what is being produced, I have to fire up my JEE container and do a request. Not to mention that creating templates and tags requires some lovely combination of XML and Java code. I don't know about you, but that feels off to me.

Why? XML doesn't give me much flexibility. DRY gets violated all the time because there is no mechanism in the design of XML for reasonable reuse. Java on the other hand lets me refactor and reuse to my heart's content, but as we've already established, it's crap for actually producing HTML.

And what about Ajax? JSP is fine if your site uses tables for layout and all links result in a full page refresh or occasional popup window, but if you want to build a responsive, dynamic site (using say, Ext) you'll end up having rendering logic in two places, WhateverCrapFramework and JavaScript. The best possible outcome is that you hardly use JSP (or whatever your 'controller' is) and rely on your Ajax toolkits.

Again why? Because static typing everything is not appropriate for user interfaces. A user only has one input type available to them: text. They can't enter FooBarDtos directly, they don't even know what one is. You have to take some text and convert and validate it. Granted, they can also click things, but HTTP seals the deal for the web because all data submitted must be in text form. Your types cannot help you until you have already converted and validated the data, at which point it's ready to pass on to your service layer. So why the heck do you need types everywhere in your controller? You don't, but Java forces them on you anyways.

I think Java is great for web applications by the way. Spring+Hibernate remains my first choice for implementing a solid service and data access layer (I like Guice too). Static typing is great for helping ensure that your services are solid, and your data remains valid.

Back to Ajax. I made a big investment in JSF in graduate school. I nearly killed myself and my wife building a framework for creating Ajax enabled components for JSF. It took me a few months after finishing to realize that JSF still doesn't make Java web development any less like wearing plate mail while figure skating.

When you get right down to it, the best language for developing a web UI is the language that was always meant for the web: JavaScript. DWR completely bridges the gap between Java and JavaScript. It converts all that pesky string data into nice typed data for your Java service layer and then turns around and converts those types that the user doesn't know about into pleasant little strings for web browsers to gobble up. You can expose your Spring service beans directly, secure them with Acegi (you better be doing that anyway, if you're relying on your controller to secure access, you're using a sieve as an umbrella), and call them from JavaScript without even having to know how HTTP requests work, much less Ajax.

In conclusion, we need real developers on the client side. Good designers/UX folk don't usually make the best technical developers, and the dynamic web (especially for enterprise) requires a solid, coherent client side codebase, and things like DWR make that easy. If you're a Java web application developer that's still afraid of JavaScript, you'd better learn to face your fears.

The IE script load event

2008-01-07T14:29:00.000-06:00

This is just a short rant. I promise I'll post more about Grails latter.

I really hate IE. Take this code to create a script tag and fire a callback once it's loaded:


var script = document.createElement("script");
script.setAttribute("type","text/javascript");
script.setAttribute("src", fileName);
Event.observe( script, "load", function() {...});

Works great in Firefox. But in IE6? Never fires. You have to do this instead:


Event.observe( script, "readystatechange", function() {
  if(script.readyState == "complete") {
    ...do stuff...
  }
} );

It's not that there is extra functionality. It's that a decision was made to provide a complicated interface that can handle like two extra edge cases, but the much simpler, obvious and consistent 'onload' handler gets left out.

Custom DWR Converters

2008-01-04T08:01:00.002-06:00

DWR is great. If you're full bore into Ajax development, it eliminates the need for a awkward controller like JSF or Spring MVC. You can write static DHTML that talks directly to your service layer, which keeps the UI code clear and simple. Secure your services with something like Acegi, and you're almost there.

I say almost, because the way DWR marshals your beans is either all or nothing. You can tell DWR, convert only these classes, and only these properties and that's it. What if you have different user roles that have different read permissions? e.g., detailed user info should be accessible to the admin and no one else. What if you want summary information when an object is nested inside another one? e.g., my Label BO has a name, description and a couple of flags associated with it. When I retrieve a labeled thing, say a message, I really only care about the label name so I can render a list of labels on the message. But when I retrieve a Label for editing, I need all the information about the label. Let's see if we can fix this.

Because DWR is open source, I can browse around and find the source for BeanConverter. It has a lot of hooks (including isAllowedByIncludeExcludeRules, which would be helpful for role based filtering), but what we really need here is a hook to control what converter is used. Because we have the source code, why don't we add one? I'll just create a subclass of BeanConverter and override convertOutbound like so:


public OutboundVariable convertOutbound(Object data, OutboundContext outctx)
      throws MarshallException {
  Map ovs = new TreeMap();

  ObjectOutboundVariable ov = new ObjectOutboundVariable(outctx);
  outctx.put(data, ov);

  try {
      for (Entry entry : (Set>) getPropertyMapFromObject(
              data, true, false).entrySet()) {
          ovs.put((String) entry.getKey(), getPropertyValue(
                  (Property) entry.getValue(), data, outctx));
      }
  } catch (MarshallException ex) {
      throw ex;
  } catch (Exception ex) {
      throw new MarshallException(data.getClass(), ex);
  }

  ov.init(ovs, getJavascript());

  return ov;
}

The code is pretty much the same as in the superclass, but with 1 key difference. My version calls getPropertyValue. In the template class, I keep the default behavior, loading the converter from the converter manager, but we'll create a subclass in a moment that takes a different approach.

First, let's look at what dwr.xml offers us in terms of configuration. It looks like you can set properties on converter instances using param elements. I prefer to use a concise format for configuration whenever possible, so our format will be something like "propertyName:includeExcludeProperty1,includeExcludeProperty2 otherProp:more,of,the,same". Supposing this was an include, then when we create the converter for the otherProp property, it will include the more,of,the and same properties and no others. Got it?

We need to provide parsing for our custom format in the converter:


public class SubBeanConverter extends TemplateBeanConverter {

protected Map subInclude = new HashMap();
protected Map subExclude = new HashMap();

public void setSubIncludes(String defs) {
this.subInclude = parseIncludeExclude(defs, true);
}

public void setSubExcludes(String defs) {
this.subExclude = parseIncludeExclude(defs, false);
}

private Map parseIncludeExclude(String defs,
boolean include) {
Map map = new HashMap();
for (String def : defs.split("\\s")) {
String[] split = def.split(":", 2);
BeanConverter bc = new BeanConverter();
bc.setConverterManager(getConverterManager());
if (include) {
bc.setInclude(split[1]);
} else {
bc.setExclude(split[1]);
}
map.put(split[0].trim(), bc);
}
return map;
}
...

TemplateBeanConverter is our original subclass of BeanConverter. Since we're using the same format for includes and excludes, the same method is used to parse them both. For each property we list, we create a BeanConverter instance that we will use in place of the usual converter.

The last step is to override getPropertyValue and use our special BeanConverter instances:


protected OutboundVariable getPropertyValue(Property property, Object data,
OutboundContext outctx) throws MarshallException {
Object value = property.getValue(data);
if (value != null) {
BeanConverter exConverter = subExclude.get(property.getName());
if (exConverter != null) {
return exConverter.convertOutbound(value, outctx);
}
BeanConverter inConverter = subInclude.get(property.getName());
if (inConverter != null) {
return inConverter.convertOutbound(value, outctx);
}
}
return super.getPropertyValue(property, data, outctx);
}

Easy right?

Next time I'll talk about my experience using Grails with DWR.

UPDATE: I mentioned role based filtering above, but I will leave that as an exercise to the reader. One hint: if you're using Acegi to control access to pages,

org.acegisecurity.context.SecurityContextHolder.getContext()
 .getAuthentication().getAuthorities()

will get you the list of granted authorities (AKA roles, e.g. ROLE_ANONYMOUS) for the current user.

More than you wanted to know about JavaScript object creation

2007-12-13T13:23:00.000-06:00

I began this journey because I needed to create a new instance of a JavaScript class with a variable number of arguments. Not knowing the number of arguments, I couldn't just write new MyClass(arg[0],arg[1].. etc.), so I had to figure out how to replicate the behaviour of new in some other way.

You may have heard that JavaScript inheritance is prototypal. If you don't know what that means, thin of it as one object inheriting from another by making a copy of the original object. Everything in JavaScript is an object, including functions. You may think that JavaScript has classes. It doesn't. It really just has functions.

JavaScript is prototypal, but it wanted to appeal to programmers that were used to "classical" inheritance, so it included the 'new' keyword that we all love so much. That means you can do things like:


function MyClass(arg1,arg2) {
this.foo = arg1;
this.bar = arg2;
this.baz = 3;
}
var mine = new MyClass('foo','bar');
// mine.foo == 'foo'

So what's going on? MyClass is called, but there is this 'this' object that isn't declared anywhere. Where did it come from? new created it for us and put it into the scope of the MyClass function. If a function called with 'new' doesn't have a return statement, it returns 'this' by default. You could try to call MyClass by itself, but you would end up creating 3 global variables because this defaults to window. So how do we redefine this? There are two methods on all functions (remember, functions are objects, just like everything else) that let you control the value of this: call and apply.

var foo = { qux: '123' };
MyClass.call(foo,'foo','bar');
// foo.foo == 'foo' && foo.qux == '123'
// if you don't know the exact number of arguments you can use apply
var args = ['foo','bar'];
// apply takes an array for the arguments
MyClass.apply(foo,args);

You might now be thinking that MyClass.call({},'foo','bar') is the same as new MyClass('foo','bar'). Well, new does a bit more than that. Remember the prototype? Calling new more or less creates a copy of the class's prototype, which is then used as this when calling the function. e.g.


MyClass.prototype.myFunc = function(a,b) { return a+b; };
var mine = new MyClass(...);
// mine.myFunc == function(a,b) { return a+b; }

One brief digression from our quest: John Resig blogged about a way to create classes that are instantiated correctly even if you forget new. An interesting read, but not quite what we need since we may need to instantiate classes not under our control.

So if you create a new object, copy all the properties of MyClass.prototype, and then call MyClass.apply(...), is that the same as new MyClass(...)? Nope. new does some things behind the scenes that you cannot replicate without calling new. e.g.,


var mine = {};
for(var i in MyClass.prototype) {
mine[i] = MyClass.prototype[i];
}
MyClass.apply(mine,'foo','bar');
// mine instanceof MyClass == false

So, what we have is an instance that is almost, but not quite identical to an instance created with new. If you know that the object will only be duck typed, then this is good enough. However, if it's possible that it might end up in an instanceof or have some other low level type check, this isn't good enough. The final trick that you need is new:


function newInstance(type,args) {
var f = function() {};
f.prototype = type.prototype;
var o = new f;
return type.apply(o,args) || o;
}

By creating a no-arguments function with the same prototype, we can safely use new to get the prototype copying, and then call the constructor using apply. instanceof type (and other checks, like prototype.constructor) will return true, and the new instance will walk and talk like a duck of MyClass.

I'm a nerd. Or do I mean geek?... Hi, my name is Noah.

2007-11-22T00:08:00.000-06:00

Aaaand it's Thanksgiving. I'm up late because I just got my slice and I can't stop tinkering.

I think my master's project is dead meat. Yes, JSF component creation is woefully painful and slow. Yes, a framework helps tremendously. Yes, my framework was good, and let you choose your Ajax implementation, and so on.

The problem with all that is that the JSF Ajax implementations suck. And the learning curve for JSF (much less components) sucks. It's just too quirky. Hopefully JSF 2.0 will make component creation easy, enable RESTful services, and will make it easier for newbies and old hands to just get things done. I'd like to hope all of that, but the only positive things I tend to hear about 2.0 are from Ed Burns, who seems like a very nice guy, but whose rhetoric reeks of corporate "gotta toe the line/I think my company is the best ever, is not a dinosaur who is slowly and inconsistently embracing open source, and is a bit too bureaucratic to be able to save Java." JavaScript is it, ya know.

Which brings us to my next topic. Kevin at work is the new Grails evangelist. I have to admit, Groovy is a cool language. If not for the fact that JavaScript is the NBL, I think Groovy would be the weapon of choice. Optional static types, intuitive C like syntax, closures everywhere, functional programming, etc. It's a readable LISP, IMHO. Since we'll probably be using Ext for my next project, (in wonderful freezing cold Pittsburgh, no less...) I thought that I'd take the opportunity to get familiar with it by working up a sample project. It's going quite well, so I must remember to get a few entries out of it (yes the separate tech blog is pretty much dead).

Groovy is a great complement to JavaScript/Ajax programming primarily because the thought processes are the same. There are some more syntactic niceties in Groovy, and there is sadly no Prototype for Groovy, but switching is completely effortless. And if I need to, I can always turn the static types back on and write some Java code. If you are a JEE developer and haven't looked at Grails yet, you need to.

More bulletins as events warrant. Cheers.

In the swing of things

2007-10-31T18:46:00.000-05:00

I'm finally starting to feel settled in Austin. It helps to actually live here. Working downtown is pretty cool, and I like the bus. Despite the smells, it's usually a nice ride. I get a lot of reading done; too much in fact. I need a side project, so I'm trying to revive my master's project. We'll see how that goes.

November 10th (Saturday), we are having a BBQ from 3ish to whenever I turn into an angry drunk and kick everyone out. If you're reading this, you're invited. If you're not reading this, something is wrong, please stop before you destroy the universe. Thanks.

I need to read something intellectually stimulating, but I'd prefer fiction. Any ideas? Oh, and happy Halloween.

Living in Austin

2007-09-07T11:37:00.000-05:00

There is so much to do here. In the last month I've learned more about bikes, built my own road bike, taken the dogs to two of the off leash dog parks, taken the bus downtown a few times (~30min in which I can read, write or just enjoy the scenery; in a car any of those will get you killed), and fixed 98% of the problems with the house, of which there were several dozen. Oh, and I've been very, very lazy, played video games a lot, and spent probably a week's worth of time on reddit. You may not believe it, but not having a job gets old after a while.

Last night we saw Quiet, Lovely for the first time in a few years, although we had to duck out early as H's stomach bug decided to grace us with one final encore. As an aside, The Mohawk seems like a nice place to get a drink, but until they finish their outside stage, it's a bit small for a decent show. Also, why the heck would you start a show at 10 PM on a Thursday night?!?!?! People who like music sometimes have jobs, do you really think you'll sell more beer by forcing them to leave before the show is over? Tonight, Meryll plays at Stubbs. Going by last night, we should show up at about 3 AM.

I fly to Boston on Sunday to start work. According to my parents, I've been to Boston, but didn't get to see much as I was only -2 months old. Think about it. I hope I like traveling as much as I think I do.

it's not so bad...

2007-08-06T20:00:00.000-05:00

One of my committee members dropped out. Since my defense was supposed to be tomorrow, that means postponing it a few weeks while we find another person and give them time to get up to speed. Disappointing, but not that big a deal. We still move on Friday, I'll just have to make one last trip up to Hueco later this month.

Yes, my life will be marginally more stressful between now and then, but there's nothing to do about it, and it's not too bad.

Update: It's really not so bad at all. Another professor, hearing my plight, took pity on my poor soul and has undertaken the herculean effort of digesting my 68 page project document in less than two days. It looks like he'll actually make it too! My defense is postponed only until Wednesday. That doesn't mean it's a sure thing, but it gives me a good shot.

blog, blag, blech

2007-08-05T20:37:00.000-05:00

Grad school doesn't leave a lot of room for a social life. Certainly nothing to write about, unless you care about my frustrations with technology. That should change soon. That ends Tuesday, then we move to Austin, have a whole month off, and then start a job that sends me all over the country. My first day is in Boston. That's pretty cool.

Hopefully this will be a journal of cool stuff I do while traveling and when I'm home in Austin. I'm sure I'll also resume a bit of the philosophical bent this blog had long ago, but probably with less of the bad poetry/stream of consciousness/"look how emo I am" content. Stupid emo kids.

I'll be 25 this year. There's no avoiding it, I'm a grownup now. But, that doesn't mean we have to act like our parents.