When I do get the urge to write, nothing derails it faster than getting irritated with the actual writing process.  Getting your thoughts onto paper should be the easy part.  The hard part is ensuring that those thoughts don't make you look like and idiot.

So this blog post serves 2 purposes.

1. Announcing that I'm going to be putting more time into blogging.
2. Starting to lay out a plan for making blogging less painful

First off, let's get a feel for why the current experience isn't working.  Here's what my current blog interface looks like.

Not too shabby looking right?  I'm running the blog app that comes with the Axiom Stack framework.  I've written about Axiom Stack before and I think it's pretty sweet.  I substituted the fckeditor for wymeditor, because it produces more standard xhtml.  This is a nice addition for Axiom Stack because it uses e4x to parse and manipulate xhtml on the back end.  But there loads of problems here (no offense to the axiom software guys, this is a free blog app that's really just meant to be an example).  The rest of this blog focuses on the most egregious offender.  The WYSIWYG editor.

WYSIWYeG: or What You See Is What You (eventually) Get

WYSIWYG editors are terrible for any significant writing.  The behavior is often inconsistent.  They're often buggy.  And you spend a lot of time fiddling with toolbars to get your content to look the way you want.  Or worse yet, you have to flip over to the dreaded HTML view to fix things.  However, they've become very popular on the web for 2 reasons.

1. They give the user lots of controls for doing other important things, like working with tables, links images, bulleted lists, etc.  All those other things that make up "content".  In short, wysiwyg's give us an environment similar to (or at least inspired by) Microsft Word and other desktop word processing software.
2. It's a hell of a lot better than writing HTML by hand.  HTML is the language of web content, but it's not what you want to be thinking about when you're writing.  For instance just making one word bold in that last sentence requires 17 extra keystrokes than normal (I prefer <strong> to <b>, don't you?)

There are good things about wysiwygs but my biggest complaint with the one's I've used is that none of them hide the HTML well enough.  There are always quirks that remind me that I'm dealing with code.  And that code needs to be formatted properly or I'm going to have issues.  What I want is a language that allows me more flexibility with my content without shoving HTML in my face.

As far as I can tell, that's Markdown.  Markdown is gaining a lot of traction as a language for facilitating written composition on the web.  You just write like you would normally, double newline for paragraphs, simple markers for formatting. So this:

This paragraph serves to *emphasize* some of the nicer features of markdown.  It is _not_ meant to be a full-fledged [tutorial](http://daringfireball.net/projects/markdown/basics)

Produces this:

This paragraph serves to emphasize some of the nicer features of markdown. It is not meant to be a full-fledged tutorial

The key feature with markdown is that you can write html if you need to.  But if you don't, you'll never see it, and you can still do the usual formatting of your content; bold, italic, lists and links.  99% of blog content is made up of words and these simple elements.  That should be easy.  And with markdown, it will be.

So my first goal is to replace wymeditor with a decent Markdown editor.  As it turns out, a cursory google search couldn't find any that do what I want.  Which is to provide a native markdown interface, syntax highlighted, with a function that will preview it in HTML.  So that's my first project on the road to a better blogging experience.

Other posts in this series follow the changes I'm making as well as touching on anything else I run into that's interesting.  Other stuff on my list include:

• Better integration and support for code snippets
• Easy dictionary look up
• Easy image inserts (uploaded to my server)
• Re-usable formatting elements

Most of this stuff can be easily accomplished with a few plugins in popular systems like Wordpress or Movable Type.  But this is my technology blog.  We're looking for something a little different here.  Hopefully future installments will be more interesting than what this post turned into.  But hey, give me a break, I haven't done this in like 5 months.

I don't write much Java.  I'm all about dynamic languages (hence the Uneducated tag).  But sometimes I have an opinion on things I don't know a whole lot about.  So here are my thoughts.

I ran across a proposal to eliminate checked exceptions in Google's experimental noop language.  I think this is a good forum to discuss the issue because the language doesn't have any baggage that comes with worries about backwards compatibility and that type of thing.  We can talk about the problem that checked exceptions try to solve and thus think of new ways to approach that problem.

Anyway, I left a comment there and I'm just going to reproduce it here for posterity and possible discussion:

I've been reading a lot about checked vs. unchecked exceptions lately. This isn't my field, but I'd just like to offer some questions/comments.

Quick question. With the above proposals (original and gabrielh), how would unhandled runtime exceptions propogate? Since they aren't explictly thrown, they're just set as scope variables.

Some people argue for specific exception types. But in my experience, most calling code trys to discern 1 thing. Is it a problem I can recover from or not? To try and clarify a little:

• The caller can't recover from a file not found error. You could suggest that the caller may be able to create the file, but that's probably giving the caller too much responsibility. Essentially another code path should be started that may eventually culminate in calling the function again. But the original path is essentially dead in terms of exception handling.

• A caller can recover from a Timeout exception in a network call. The method is idempotent in most cases, so it can just call again (and hope it was just a hiccup). Or it can pass a higher timeout. I think these are reasonable actions that are still within the caller's responsibility. If these don't work, it's time to execute another code path or propogate the exception.

So in my line of thinking, checked exceptions force a caller to consider whether it can handle any problems up front. You're more likely to include exception handling code because it's part of fulling the method signature. Only we've seen in practice that this is not the case. In many cases, the try/catch structure encourages programmers to treat all exceptions as "unfixable". Basically instead of handling the exception, most try/catches do one of the following.

• Clean up before re-throwing
• Wrap the exception in your own to throw down the stack
• Ignore It

So if you buy all of this (and I'm not sure i do, this is just a line of thought) it looks like checked exceptions aren't that helpful. But having a mechanism for explicitly specifying the exceptions that are possible is very helpful.

So this is my list of must-haves before I'll consider using another browser setup on a regular basis.  I'm a professional web developer, and like many people reading this, I'm also an internet junkie.  So this isn't going to be the typical feature set.

High Performance and Responsiveness

Obviously this is the biggest issue and the real reason I'm considering the switch.  My typical browsing session involves 8 or more tabs open to the most resource intensive sites you can think of.  Web applications that are on the cutting edge of browser capability, vendor products that provide massive functionality behind the scenes, huge multimedia sites choc full of giant images, audio, video, flash and javascript.

And that's just my day job.

The fact is that for whatever reason, Firefox can't handle this level of activity anymore.  It slows to a crawl.  It eats my RAM like candy (and I have a lot of RAM).  And on the days when I'm being particularly patient, it just calls it a day and crashes.  Unacceptable.

Fortunately, Firefox has some real competition these days.  Internet Explorer 8 isn't completey terrible.  But I'm on a Mac, plus there's no way I would use it for web development.  I've been using Safari 4 for the last week or so and it's zippy pretty much all the time.  When I was a Windows guy, using Google Chrome was a joy.  If they ever release a decent version for OS X, it'll definitely be in heavy rotation.  Opera has always been one of the better performing contenders from what I hear.  Firefox is easily beat in this category.

Web Developer Tools

This is pretty high on my priority list since I can't do my job without decent development tools built into the browser.  Sure, I used to build websites before Firebug came along.  But I now recognize that era as The Dark Ages of web development.  I'm as eager to go back there as I am to see what the Black Plague was like.  Fortunately, the other browsers have developed some nice tools as alternatives here.  Safari and Chrome offer a console that allows you to inspect the DOM, see some applied css rules and debug javascript.  The Chrome console is missing some key things though.  My guess is it comes pretty much out of the box from Webkit.  The Safari version is much more full-featured.  But on closer inspection, they both have deficiencies that can't stand up to the combo of Firefox with Firebug and Web Developer Toolbar.

• Edit in place: With Firebug I can inspect html and edit in place.  I can see tweak things in real time instead of the usual edit-save-reload cycle.  This goes for css too, which is really helpful.
• Full CSS view: Selecting a DOM element in Firebug shows the full cascade of css rules that apply to a given element rather than the truncated view in the Webkit console.  It excludes inherited styles (e.g. default from the body element)
• Network Profiling: The Net view in Firebug is great for getting a real sense for how your page loads and what the pain points are.  The Webkit console has a page load view, but it seems buggy and wrong a lot of times (reporting 0 byte files and 0 load times).
• View All Javascript/CSS: Not sure how many others use this feature.  But the Web Dev Toolbar has an option that will give you a page with all of the javascript or css for a page all stitched together.  This is awesome for searching for that one pesky line that's giving you trouble.  Especially if some of the code is loaded from external sources and you're not very familiar with it.  (To be fair, this can be replicated with a few bookmarklets though.)

Add YSlow and other Firebug extensions this list and that's a lot to give up.

Major Extensibility

This section pretty much covers the rest of my gripes with other browsers.  The greatest feature that Firefox brings to the table is that it is infinitely customizable.  All of the features of Firefox that make it so indispensible were developed by third party developers.  Over the last several years I've amassed a prodigious collection of add-ons that turn my firefox into exactly what I want it to be.  My first couple of days trying to use Safari or Chrome, I could barely get anything done.  I was missing the familiar environment that was tailored to my workflow. Here are my favorites features and add-ons:

Web Development

• Firebug and extensions plus Web Developer mentioned above.
• It's All Text.  Let me use emacs wherever possible.
• Tamper Data.  Cause sometimes HTTP can be a PITA.
• Not to mention MeasureIt, Colorzilla, Pixel Perfect,...

I don't use all of these all the time.  But when I need them, they make my life much easier.  And when I don't have them, it makes me contemplate finding a new profession.  I've been searching for replacements on the Mac and what I've found is that most of the good ones cost money.  Yes, I've paid for a few I thought were good.  But none of them are integrated into the browser very well.

Productivity

• Bookmark keywords:   This is actually built into Firefox and it may seem like a small thing. But Firefox lets you give any bookmark a keyword, which is more like an alias from the unix world.  When you type the keyword into the address bar it replaces it with that bookmark.  So for instance, to visit facebook I just type "fb".  No clicking, no googling, I'm immediately there.  And I never have to remember anything.  "What's that long stupid url for our dev wiki?" Type "devwiki".  Easy enough.
• OpenBook and XMarks.  Combined with the above, I'm much better at managing links and navigating to my most frequent sites.
• Tab Mix Plus.  Firefox has pretty decent defaults for getting navigating around and managing tabs.  But after I got used to my TMP settings, it felt like things were broken whenever I didn't have them.  Middle click on a tab to close it.  Double click on a tab to duplicate it (including the history.) Middle click on the tabbar to re-open the last closed tab.  If you aren't doing this, you are missing out.  Firefox saves your closed tab history by default, but this makes it much more accessible.
• Better Gmail.  Makes GMail... better.  CustomizeGoogle.  Makes the rest of the google products better.
• QuickRestart.  This is handy when you need to restart Firefox.  It used to be needed when you installed new add-ons.  But they have long since added a convenience button to do that.  Now it's useful when your Firefox has consumed all of your memory like some kind of black hole, and it needs to be restarted so you can keep working.  The reason this is notable is because QuickRestart gives you a button that can be added to your toolbar.  Just right-click and select Customize and you get a whole list of things that can be pulled onto the toolbar. Many other add-ons do this too, giving you very convenient access to their functionality.  This is a clutch feature in my opinion.

Web Surfing and Social Media

• Shareaholic.  A couple of clicks to share anything I want instantly on Reddit, Digg, Facebook, Twitter, Gmail and 50 other sites I've never heard of.  This is essential for minimizing my goofing off time.
• Search Cloudlet.  Adds tag clouds of relevant keywords to Google searches and Twitter.  I thought this was just a novelty at first, but it actually prompts you to add words to your searches that you may not have thought of. It can help refine your thoughts about what you're looking for.
• Xoopit for GMail.  This add-on crawls your emails and pulls out all of the media, making it organized and searchable.  You can see all the pictures or videos people have sent you in the last week, month or year.  Maybe not very useful in general.  But it is very cool to look over the entire history of the random stuff people wanted me to see over the years. That's how I found this nugget of joy.

Everything Else

Greasemonkey.  This is probably the clincher.  Userscripts.org has tons of scripts contributed by people who made the web work they wanted.  I can't even use Facebook anymore without the Fixer.  Many of the best add-ons started as greasemonkey scripts.

The bottom line is that in my version of Firefox, I can have things my way.  I've been trying to get other browsers to behave for a while now, and it's not working out very well.  So I guess it's back to slow performance, frequent crashes and even more frequent restarts.  I'm pretty confident that Mozilla is working on fixing these issues.  But they had better step things up.  The latest build of Chromium for Mac looks better every day.

There are several usable implementations of the early ServerJS standard available right now.  So I sat down this weekend to see if I could integrate one of them into Axiom Stack (the open source server-side javascript framework that powers this site.)  I chose Narwhal simply because the primary maintainer, Tom Robinson, is focusing on support for the Rhino JavaScript Engine.  Getting things up and running ended up being pretty straight forward, so I figured I'd outline things here.

On the surface Narwhal, and any other ServerJS compatible implementation, is just a library of javascript files that provide the standard APIs.  The underlying implementation relies on java packages though, which is why the Rhino engine is key.  Narwhal includes Rhino as a dependency and it also includes scripts for bootstraping the javascript environment in a JVM and loading the necessary js files.

That's all great for getting started with Narwhal, but Axiom Stack already provides the Rhino environment, and in fact it already provides a mechanism for loading js code into your application.  So my task was to find the merge point between Axiom modules and Narwhal packages.

Axiom Stack modules are pretty simple. Include your code in the modules folder under the Axiom install dir.  You need to follow a certain package structure that Axiom recognizes.  Then add the module name, which is just the name of the module folder, into your app.properties file.   It should look something like this:

AXIOM_R00T/
  modules/
     narwhal/
        Global/
           packages.js
            ...
            ...

And in your app.properties:

modules = narwhal

Now to figure out which Narwhal files to load.  Download the narwhal source from github.  Narwhal actually includes support for several javascript platforms including Rhino, Spidermonkey and v8.  Since I'm interested in the Rhino version, I'm checking things out in NARWHAL_ROOT/platforms/rhino/.  The lib/ folder has all kinds of good stuff, so I tried copying these files into my narwhal module Global/ folder.   That didn't work.  I got all kinds of awesome error messages about missing functions or variables.  So I read up a little and finally came across this helpful thread.

I think that for Jaxer and ASP, you can bypass bin/narwhal (which just finds itself and calls platforms/*/bin/narwhal-*), and you can bypass platforms/*/bin/narwhal-* (which just finds the engine and executes platforms/*/bootstrap.js) and go straight to platforms/*/bootstrap.js.

So it looks like we need to bootstrap the narwhal environment first.   But after that, the bootstrap file knows where everything else is.  So after some more futzing around and breaking things, I did what any self-respecting integration artist would do: I cheated.

All I needed was for Axiom Stack to load and execute that bootstrap file.  Axiom Stack loads whatever is in the modules/narwhal/Global/ folder.  So here's my setup:

AXIOM_ROOT/
   modules/
      narwhal/
         Global/
            bootstrap.js -> NARWHAL_ROOT/platforms/rhino/bootstrap.js

Yes, I simply symlinked the bootstrap file into the Axiom Stack file tree.  Almost there, but there's one more step.  The Narhwal loader looks for the "NARWHAL_HOME" environment variable to know where the install is.   So set this in your environment and restart Axiom Stack and you should be rocking and rolling.

Yeah, I know this is not ideal, and certainly wouldn't be acceptable for a production setup.  But hey, we're talking baby steps here.   This is pretty good for a couple of hours on a Saturday.  I've already got ideas on how make the integration a little more stable.  Hopefully in the future, Axiom Stack will be fully ServerJS compliant from the start.  Until then, check out Narwhal and make sure to give feedback to the group.

<!-- This is a comment. -->
<!--
        They can also span
        multiple lines.
-->

Pretty straight forward. And you would think it's easy to identify when rendering. Once you see the "<!--" then ignore everything until you see "-->".  Except Firefox doesn't do that exactly.  It turns out if you have 2 dashes or hyphens contained in the comment, Firefox will prematurely end the comment.  And probably break your page.  Example:

<!-- This is a fine looking comment--Until you get here. Now you're screwed! -->

The above comment Will break in all versions of Firefox up to the current stable version of 3.5.  Take a look (You are using Firefox right?)  This doesn't happen in any other modern browser including Internet Explorer.  But okay, so there's a rendering bug in Firefox.  It happens right?  No big deal.  I'll just fulfill my responsibility and go on over to the bug list and make sure it's there.  Sure enough, I found the bug, and that's when things got interesting.

Apparently people have been arguing about this bug for 6 years!  And it still isn't fixed!  Because, like all good stories about programming minutia, some really smart guys don't agree that it's a bug.  HTML is a tag-based language originally based on SGML, and apparently the spec for HTML relies on SGML to specify how comments should be handled.  Here's the choice excerpt from the spec.

White space is not permitted between the markup declaration open delimiter("<!") and the comment open delimiter ("--"), but is permitted between the comment close delimiter ("--") and the markup declaration close delimiter (">"). A common error is to include a string of hyphens ("---") within a comment. Authors should avoid putting two or more adjacent hyphens inside comments.

You see the issue here is that the end delimiter of a comment in SGML isn't "-->", it's just "--". After the double hyphen your text starts to mean something again. And you can include as much text as you want after that. But according to HTML you're still inside a tag, because it hasn't seen a closing ">" yet. Hence the weirdness. In fact it's even more complicated than that. Check out this more in-depth breakdown. Which explains why one set of double hyphens breaks comments while a second set will fix them. <!-- So something -- like this -- would work just fine. -->. Remember, if you're going to include double hyphens inside your HTML comments, make sure they come in multiples of 2.

Oh... well... fine then.  That's kind of stupid, but okay.  Bug closed as invalid.  And it was.  Until someone else realized it was stupid.  In fact, several people did.  Which is why it was successfully removed as a requirement for passing the Acid 2 compliance test.  And finally it was fixed where it should've originally been fixed, in the spec.  HTML5 addresses this issue.   Hooray!   Now the official end delimiter for comments is "-->".   Except you're still not supposed to use double hyphens... whatever.

So now the smart guys at Mozilla are listening to reason and plan to fix the bug in the next version of FF.  But only after the HTML spec supports it.  That's the most interesting thing to come out of all of this in my opinion.  It raises a few questions.  Is it good to hang on doggedly to a spec written years ago, when clearly it does more harm than good in modern practice?  And more importantly, when is it time to stop being pedantic and fix the damn bug already?  Apparently you can give it at least 6 years.

While I do really like the ExtJS APIs the generated markup makes me cry. A button should not be made of multiple 's. #fail

I had this same gripe a while back when I was hacking facebook with greasemonkey.  I felt like facebook was flouting everything we've learned about web standards and structured content.  Everything is a <div> wrapped in multiple other <div>'s.  In fact, most of the content is generated with Javascript.  With their last major redesign, Facebook eliminated many of the explicit page reloads.  Now they literally pull in the entire page with ajax and regenerate the markup.  And a couple of weeks after I finished my greasemonkey script, they changed the interface again and broke it!

But then I started really considering what apps like Facebook are trying to accomplish.  They're task-oriented, not content oriented.  Most of Facebook requires a user to be authenticated.  fb is more concerned about people coming to the site, not robots.  And even considering that, the public pages still manage to rank pretty high in Google.  Search for marco rogers and you'll get my profile link as the first result.  Even though the page is pretty much entirely generated with javascript. Today's web apps put the application first and the web second. Many of them are basically meant to be portable desktop apps. When was the last time you cared how the visual elements in MS Outlook were put together?

Unless I miss my guess, the markup spit out by fb is probably tied pretty closely with some proprietary content framework they're using to generate the pages.  And generating semantic markup with javascript is no mean feat.  If you can get away with every element being a <div> or <span>, you're almost certainly doing yourself a favor.

That's how I read the promise of Ext JS.

Ext JS is a cross-browser JavaScript library for building rich internet applications.

Ext JS is offering to do you that favor. You can build those complex applications with an interface that will be consistent in functionality as well as look and feel. As long as you're not worried about search findability. And if you're building a web application, you should really ask yourself that question sooner rather than later.

Ever since the most recent Facebook redesign, you can't see your current status from the home page.  You have to go to your profile or find out through some other method.  This has been irking me, because the only time I update my status is when I got to my home page to read something and realize that it's old.

Anyway, I was looking for something to do so I wrote a greasemonkey script that adds the status back to the home page.  You'll need the firefox add-on if you don't already have it.

And you can get the script here: http://userscripts.org/scripts/show/45971.  I'll probably write a follow up about the headaches I ran into trying to get this to work.

A Few Requirements

First a little setup. Ideally this summary would be written specifically for the article and formatting nicely for it's space in the page. But since we live in the real world, we are generally given a few requirements for our system to make this easier. These are the general guidelines I used:

• The summary should be generated automatically from the first paragraph of the full text (or first 2 or first 3 ...)
• The summary should preserve any html that may be present (i.e. links, bullet points, etc).
• The editor (me) should have the option of overriding this auto-generated summary with a custom one.

The Tools

I wanted to see how simple I could make this with the system I'm using. I have a few things going for me.

Almost all of the html content is available as well-formed xml. This means I can manipulate xml node objects instead of parsing text. Using text manipulation techniques to modify tag-based html has always bothered me. You end up either writing lots of code to handle lots of different possibilities or relying on complex regular expressions. By representing your content as xml (or in this case xhtml) you can be a little more confident that your output will still be well-formed and valid. And you can probably manage your changes with less code.

It's not DOM, it's e4x. This system is Java based. And yes, that's a good thing. Because that allows me to use the Rhino JavaScript Engine. In my opinion, javascript is a very nice scripting language. And working with it on the server as well as the client side is pretty nice. But more importantly, Rhino has native xml support through the e4x specification. E4x is much easier to work with than the w3c DOM spec or most language APIs for manipulating xml. This makes some of the code look a little funky.

It's fully object oriented. This isn't really an objective advantage. I just prefer it. All it means in this case is that all of the functions I'm using are actually methods on an object. The object represents my blog article and I have access to any of the properties of the article through the "this" keyword. I'll talk about this more in the second part to Why you should be using an Object Database instead of ORM.

The Code

The first step is a function that returns the summary. I'm calling it "abstract" and the first pass just returns it.

function get_abstract(){
    return this.post_abstract;
}

This is just a getter function. But what if no abstract has been provided? Then we're ready to generate it. So let's update this a bit.

function get_abstract(){
    if(!this.post_abstract){
        this.post_abstract = this.generate_abstract();
    }

    return this.post_abstract;
}

A few things to notice here. First off, the "post_abstract" is a property on our blog article. That means when we give it a value, it's automatically saved into our database. There's nothing else we need to do to be able to access the value of this property again later. Second, the "generate_abstract" function is also a method of our blog article, so it needs to be accessed through "this".

So we've got a simple function that returns our abstract. And if there isn't one, it calls a function to get one. Let's check that out.

function generate_abstract(){
    var limit = 3;
    var tmpbody = this.body.elements();

    var abs = <></>;
    for(var idx in tmpbody){
        abs += tmpbody[idx];
        if(abs.length() >= limit) break;
    }

    return abs;
}

This is a pretty simple function. It grabs all of the elements out of the body of the article. These are xml elements representing our xhtml. Probably paragraph tags. But we don't have to actually worry about it. It could be any tpye of element. We loop through these elements and grab as many as we need. Then we add them to our temporary abstract. This "abs" variable is the only place i this function where you see the e4x peeking through. We start by creating an empty XMLList. This is essentially an xml fragment with no top level container element. That describes the html body of our blog post perfectly. Here's a snippet of the html of this post.

<p>In the world of online content management, we often want
 to introduce an article with a summary or abstract.  
The best example of this is the front page of a blog.  
You are presented with a list of the most recent articles 
with just a few paragraphs from each to get your interest. 
 If want to continue you can click through and get the full
 text.  Until recently, this blog didn't do that.  Since I 
I haven't seen a lot of</p>
<h2>A Few Requirements</h2>
<p>First a little setup.  Ideally this summary would be 
written specifically for the article and formatting nicely 
for it's space in the page.  But since we live in the real 
world, we are generally given a few requirements for our 
system to make this easier.  These are the general guidelines
 I used:</p>
<ul>
    <li>The summary should be generated automatically from 
the first paragraph of the full text (or first 2 or first 
3 ...)</li>
    <li>The summary should preserve any html that may be 
present (i.e. links, bullet points, etc).</li>
    <li>The editor (me) should have the option of overriding
 this auto-generated summary with a custom one.</li>
</ul>
...

So in our list of elements would look something like this [<p>, <h2>, <p>, <ul>, ...]. We grab the first 3 to form our abstract, and return the new XMLList. That's our summary! The first 3 html elements from our body text. It gets saved on our blog article object so that any time we need it, it's available. No sense in doing this work every time. So that's it. I hope this has given you a little taste of how things work on this blog. This doesn't even scratch the surface of the power and convenience in using e4x to manipulate your content.

Before we wrap, I have something to confess. The get_abstract code doesn't work. This was a weird gotcha moment in the e4x spec that I had to work around. When working with an XMLList, calling the elements() function doesn't return the first level children of the list. It returns the children of all of those elements. So instead of my <p> tags I got the text nodes inside them. Once I figured it out, there was a simple addition to account for it. Just wrap the body in a top level node first. Then the call to elements will give you the child nodes you expect. As an after-thought, I've included the ability to specify the length of the abstract. Here's the final version of the function.

function generate_abstract(elimit){
    var elimit = elimit || 3;

    var tmpbody = <div></div>;
    tmpbody.appendChild(this.body);
    tmpbody = tmpbody.elements();

    var abs = <></>;
    for(var idx in tmpbody){
        abs += tmpbody[idx];
        if(abs.length() >= limit) break;
    }
 
   return abs;
}

Oh yeah, replacing the full text on the home page with the abstract text was pretty simple. Instead of calling this.body, call this.get_abstract. Pretty straight up. Enjoy the shorter home page.

The problem with RDBMS

First of all, I don't have a problem with RDBMS.  I think it has served us very well up to this point and will continue to do so for the forseeable future.  It's a perfectly workable solution to the problem of storing and retrieving persistent data.

I don't think anyone will take issue with that argument.  So what's wrong with relational databases?  The answer has been getting a lot of attention lately so I'll keep it simple and let Wikipedia define it for me.  Impedance Mismatch.  I pretty much agree most of the points listed, and several of them can be rolled up into one argument.  Essentially, RDBMS doesn't fit the object-oriented way we like to use data on the web.

The last sentence is important, because it puts some necessary context on the argument I'm making.  I'm gonna go out on a limb and say that the majority of working web professionals have bought into object-oriented programming.  That's not to say most people program this way.  PHP is still the most popular web language and many people still generate pages in a very top down imperative way.  (I'm one of those people who thinks PHP lends itself to this style, but that's a different rant... I mean discussion).  Also, some functional languages like Haskell and Clojure are getting attention as being viable for web programming.

But whenever I see examples of people doing anything useful, I see a trend.  We like to work with object-like data.  More and more websites are being built to provide access to bigger and more complex collections of data.  Movies on Netflix, People and their friends on Facebook even boring stuff like all of Physicians in your Insurance network under your employee health benefits plan.  So it's more and more essential for programmers to be able to create a good mental model of the data and how to work with it.  Creating objects representing these different data collections and then composing them together seems to be the best thing going.

When using an RDBMS, in the simplest form, "object-like" just means lists or arrays that represent rows in the database.  This is generally where PHP falls.  You use PDO or some equivalent, write your SQL statements and you can access your data like this.

$get_my_employee = 'select * from "Employee"';
$emloyee = $db->query($get_my_employee).fetch(); // simplified on purpose. you get the point
echo $employee["First Name"];
echo $employee["Last Name"];
echo $employee["Spouse"];

Not too bad, except for that first bit, but it seems harmless enough so lets ignore it for now.  It's just a collection of related data points representing some information on an employee.  But things are rarely this simple, and that last line looks suspicious too.  Because we're all about normalization and proper organization of our data, we've split all of our Spouses into a separate table and linked them through a foreign key. So making a long story short, that first line isn't so harmless anymore.  You have to do your join and specify the fields you want and put your necessary filter clauses on it until you get all your data in one fell swoop.

$get_my_employee = 'select e."First Name", e."Last Name", e."Phone Number", ... s."First Name" from "Employee" e, "Spouse" s where e."SpouseID" = 'schmoopie' and e."SpouseID" = s."ID"...';

I put in ellipses in place of some details to speed things along.  But depending on the complexity of your database schema, those ellipses are hiding some headaches.  Not to mention the fact that you have two columns with the same name that represent different data points ("First Name").  It's left as an exercize for you to determine which is which by adding even more code.  Fun times.

But eventually you end up with one long array with the employee you're interested in along with all the data about their Spouse as well.  You completed your task but the journey was a pain, and don't even get me started on debugging and maintaing that long SQL string.

The Problem with ORM

Okay, I think I made the last section a sufficiently irritating explanation of what we already know.  I wanted to go through this to drive home a point though. Problems like this are the reason that Object-Relational Mapping (ORM) is all the rage right now.

ORM allows us to take those nasty details and abstract them away.  We can get nice simple objects to work with and get back to the real business of making our web page do something cool.  More importantly, it allows us to deal with the relationships between our data items in a sane fashion.  No more jumping through hoops to get all of our data points into one row.   Let's switch to Python/Django just because I haven't worked with any PHP ORMs yet.


employee = Employee.objects.get(id=1)
print employee.first_name
print employee.last_name

spouse = employee.spouse
print spouse.first_name

Ah, that's more like it.  Even my non-programmer friends (who's eyes glaze over when I talk about code) can piece together what's going on here.  And that first line replaces the previous unpleasantness with all the ellipses. So what's my beef with this?  On a theoretical level, nothing.  I'm a fan of Django precisely for this reason. But from a more practical perspective, this sets off that nervous twitch I get when something is way harder than it needs to be.  That RDBMS is still there.  Lurking.  The ORM evangelists will respond with "yeah, but you don't have to worry about it".  First off, I call bullshit on that.  It never works out that way.  But let's say we're drinking the kool-aid today and I believe that.  The spirit of ORM is to literally take something that was designed to work a certain way (RDBMS + SQL), and take lots of time and effort to make it work a different way (Objects + simple filter functions).  Wrapping it in layers of abstraction until it doesn't even look like itself.  Let's talk about some of the problems.

It's an Abstraction.

When I first started looking at ORM, I thought this was a good idea.  Maybe because I was already working with objects as data on a regular basis.  One of the platforms I started out on was Zope, which is built on a datastore that represents actual Python objects composed together in a manner of your choosing.  It has it's own issues, but in terms of web programming, it felt natural.  It took a while before I got "comfortable" with SQL. It has always felt, for lack of a better word icky. The developers of Django's ORM understand this. If you look at the documentation for Django Querying, the api goes through great pains to insure that you can avoid the ickiness.  You can represent almost any complex query without writing SQL.

## The example assumes you have a Model representing a Blog and this has a number of Entries related to it.
##  To select all blogs that contains entries with "Lennon" in the headline and were published in 2008, we would write:
Blog.objects.filter(entry__headline__contains='Lennon', entry__pub_date__year=2008)

I'm sure the last of my non-programmer friends have drifted away after that. But if you are a programmer and you're trying to get a handle on how this works, you run into the following bit of nastiness before you even get to this example.

To handle both of these situations, Django has a consistent way of processing filter() and exclude() calls. Everything inside a single filter() call is applied simultaneously to filter out items matching all those requirements. Successive filter() calls further restrict the set of objects, but for multi-valued relations, they apply to any object linked to the primary model, not necessarily those objects that were selected by an earlier filter() call.

That may sound a bit confusing, so hopefully an example will clarify. (the above code example follows)

Even the developers of one of the best schemes out there (in my opinion) have to admit that they've got some crazy juju going on under the hood.  This is because they have to take what you're trying to do and put the ick back into it.  These simple filters get translated into not so simple SQL.  And to top things off, they frequently try to help you along by giving you the equivalent example in SQL.  So you frequently end up doing double work. 

• Step 1) How would I do this with a complex SQL query?
• Step 2) How do I translate Step 1 to my nifty ORM api so I can get my data as objects?

It leaks

The complexity of the ORM scheme leaves you wide open for leaky abstraction problems.  And often times, when you run into these, you lose a lot of the time you saved by using the abstraction in the first place.

If it doesn't work, you're back where you started

Is this really an acceptable trade-off to be able to work with object-like data?  Absolutely.  Hence the popularity of Django and Ruby on Rails and Symphony (or whatever php framework is actually in style these days)  But to boil it down to the point of this post.  Why are we choosing to deal with this impedance mismatch when we don't really have to?

In part 2 to this post, I'll take a shot at talking about object databases and making a case that they should become a more common addition to our web framework architectures.

For the past several years weve seen tremendous growth in the area of web development.  As we started to regroup after the bubble burst (which I just missed), there has been a pretty dramatic shift in the way web programming is done as well as what is possible.  We've seen a paradigm shift from table layouts to semantic markup with css.  Newer and better browsers have given us a more stable environment to work in as well as new and exciting capabilties.  Someone figured out what AJAX was good for and blew everybody's mind.

It's an exciting time to work with the web.  For the reasons stated above and for one other.  It seems to me that it's been pretty easy to follow the tide of innovation.  With this new renaissance has come an exponential increase in online media as well as professional and personal blogs dedicated to our field.  These days the flood of information comes right to your door.  We work in an industry where it's possible to stay on the cutting edge of knowledge simply by putting in a few hours of online reading a week.  Okay maybe several hours of reading a week, but you get my point.

It's become apparent that the ongoing conversation about web development has become pretty essential to me being good at my job.  I followed AJAX through it's stages: from a curiosity, to a fad, to an established technique, to a mandatory resume headline.  And like many, I worked at a contracting/consulting company where being a step ahead of everyone else was critical.

So shortly after I had this realization, I had the thought that eventually became my resolution for 2009.  To put it simply, it's important to join the conversation.  I think if you're a professional web developer and you're involved with anything remotely interesting, you should share it.  Throw up your own website and fill in the gaps in our collective knowledge.  There's almost certainly something you know that others can benefit from.  And who knows, you might even become so popular that you can quit your day job.  And isn't that what we're all striving for?

So I put up this blog to share my thoughts and ideas, and I noticed a few things right off the bat.

• I don't know why it took so long to get around to it.  The hardest part was picking a name for the blog (thanks Walter).  If you've been procrastinating... stop it.
• If you're purely a developer like me, then forget what I just said.  The design will be the biggest pain in the ass.  I'm not using an established blogging platform that has free design templates available, so I cobbled this together myself.
• When you're working on something of your own, you're more motivated to improve it.  This one was pretty big for me so I'll elaborate.

I mentioned that I used to work as a consultant.  Well now I'm part of an in-house IT shop, and there is a marked difference.  I'm not sure if others who work in-house have seen this.  I found that the things I do at work are so far removed from my current personal interests in the field, that I wasn't finding the motivation to work on my own projects as much.  Perhaps it was too much of a context switch.  Now it makes more sense to stay abreast of new developments in php and postgres than javascript and REST.  So not only did I have less time for side projects but less ideas and less inclination to pursue them.

But putting together this website has rejuvenated my enthusiasm (and evidently my vocabulary as well).  Not only do I have a platform to implement my own things, but I have the impetus to do so.  In my mind, joining the conversation doesn't just mean talking about what you already know.  It means talking about what you're doing.  It means letting us in on the things you're learning as you continue to gain experience.  For instance, the wysiwyg I'm using to write this post doesn't have a tool for hyperlinking text.  Can you imagine!  What is this 1999?  Well this will be remedied in the near future.  And you'll be able to feel my sense of satisfaction.  You'll hear about other things too.  Like what it's like to work with a real server-side javascript platform.

Anyway, if you're interested, stick around.  Otherwise, go get your own site started and join the conversation.

Oh, and let me know if anything is broken on the site.  It's still pretty raw.