JSConf 2009: the best conference you couldn’t attend!

(sorry for the lack of links in here. I wrote this on a plane and haven’t had a chance to do anything else to it…)

I just returned from JSConf 2009, the first JSConf conference. It was possibly even the first conference to feature JavaScript as a general scripting language in the same vein as Python or Ruby.

Overall, it was a very good conference. The organizers did a terrific job and paid great attention to detail. The sponsored evening events were an awesome idea and well-executed. (At least, Friday night’s was, I flew home Saturday evening). It was a relatively small conference at 130 people. I think they can easily have double that number next year, if they want to. But, that would require a change of venue, because the Hotel Palomar’s meeting rooms were filled almost to overflowing. From talking with Chris Williams on Friday evening, it doesn’t sound like they’re interested in changing venues next year, which is a shame because a lot of people will have to miss out on a great conference.

Francisco Tolmasky from 280 North gave the perfect kind of talk to kick things off. I’ve been following Cappucino’s development, so I was not surprised in the least with what I saw. But, Francisco is a polished speaker and many people had not seen Apple’s Interface Builder used to create webapps (via Cappucino’s nib2cib tool). We didn’t get a demo of Atlas, which would certainly have wowed this audience.

One thing that pleased a great many people in the audience, myself included, was word that Safari’s debugger would start looking at a “displayName” on functions to determine what name to show in the debugger/profiler. JavaScript has many places where it’s impossible to guess a reasonable name, and it’s nice to have a way to give the debugger a hint like this. Let’s hope we get this in Firebug soon.

Toward the end of the second day, there was a talk about SproutCore, so we had a chance to two different ways to apply the style of Apple’s Cocoa to building webapps. As with anything, there are tradeoffs. Cappucino builds on Objective-J, which gives you a more concise syntax than straight JavaScript for things like Key Value Observing. If JavaScript today had getters and setters, then this particular benefit of Objective J would go away. For now, though, using SproutCore effectively means calling get* and set* methods to get variable values rather than just looking up the value directly.

There were two presentations on server side JavaScript, a topic that people who know me know that I am currently very into. Nick Campbell showed off the Axiom Stack, which builds on Helma. Axiom is presently AGPL-licensed, which means I won’t go anywhere near it. But, that’s just me. On the plus side, Nick has been peripherally following the activity on ServerJS and is quite in tune with our goals. Nick’s company, by the way, has a unique and useful sounding web marketing-related product coming out soon, so keep an eye on those folks if you’re a marketer.

The second Track A presentation about server side JS was James Duncan’s presentation about Joyent’s Smart platform. I must say that this looks like an excellent offering, and I’m looking forward to seeing a lot more of it as it hits general availability. The best parallel I can draw is “App Engine on JavaScript”, but that doesn’t really do it justice. They have a key-value store that they will scale transparently for you. No more manual sharding! Just start tossing data in. Of course, that’s the promise… and the devil’s in the details with such things. I came in a bit late to James’ presentation and I forgot to ask him afterwards about their data store’s indexing capabilities and whether it is eventual-consistency based or more immediate than that.

The Smart platform is SpiderMonkey-based, and they pull some interesting tricks to overcome the lack of a decent stack of libraries for SpiderMonkey. Their web server interface looked very much like Jack, which is a bonus. It would be nice if we can harmonize it with Jack in some fashion. Intriguingly, their web framework, which is apparently based on Sinatra, looks an awful lot like the home grown one that I made for Bespin in Python and then duplicated in JavaScript.

I didn’t lump my former SitePen colleague Kris Zyp’s talk in with the other server side JS talks, because Kris was talking a lot more about JS that spans from client to server and using standards such as JSON Path, JSON Query, Persistent JS, and JSON object referencing to move data around seamlessly. Of course, he used Dojo and Persevere as his demo platform, but the ideas he presented can be applied anywhere.

Brian LaRoux’s talk on PhoneGap was quite interesting and entertaining. Brian’s talk had a refreshing lack of gravity, while still providing useful content. For one, he mentioned that Dashcode actually offers good tools for making iPhone web apps. He talked about a variety of iPhone-related JS toolkits, and gave a demo of Nitobi’s iPhone “stimulator” which does a better job of representing how an iPhone app will behave than Apple’s own simulator.

I should note that there were probably 50% fewer bullet points than what I have seen at some other tech conferences I’ve been to. I think the message is sinking in that bullet points suck (except for actual lists of things).

My Mozilla colleague, John Resig gave a wide-ranging talk about JavaScript performance testing, games and his project of the moment, TestSwarm. TestSwarm looks fantastic and fills a gap: it will provide a way to do cross-browser continuous integration tests. People join the test swarm by opening their browser up to the TestSwarm page, and the server will send them test jobs as they come in. So, for example, a revision gets checked into jQuery, and the TestSwarm server will pull out the tests and send them down the wire to a collection of testers who are using different browsers. The results from all of the browsers will come back and get logged. This tool will be useful for a lot more than just jQuery, and John offered help connecting it up with, say, DOH for Dojo’s tests.

Another former SitePen colleague, Pete Higgins gave a Dojo roundup at the very end of the conference. I saw half of his talk before I had to go to the airport. There are lots of good things afoot in Dojo-land. The new conditional compilation stuff seems useful for a variety of things. For example, Dojo can be built in a super-slim variety (6K) that loads everything dynamically. Or you can dump all of the IE compatibility stuff. With the Bespin project, we have a plan to ship a variety of packagings, and I can see this being useful for that as well.

Pete’s Plugd (which he pronounces “plug-dee” as opposed to “plugged”) project provides a bunch of extra convenient ways to use Dojo and I do hope to see that stuff included in 1.4. Pete says that Plugd will likely add 4K to Dojo’s gzipped size, but I think it’s likely worth it.

Malte Ubi likely takes the prize for JSConf attendee who came the longest distance to attend, having flown in all the way from Hamburg, Germany. Malte has been doing some fantastic work on Bespin. Actually, the things he’s been doing go beyond the realm of fantastic and into “crazy”: using Narcissus (JavaScript parser in JavaScript) to read your JavaScript file in a web worker to provide completely client-based outline views and code completion. Awesome.

Malte did a Track B talk on Joose, his JS object system that is built on ideas from Perl’s Moose. It looks like a very powerful system that provides things like type coercion and traits (called roles here) that you don’t find in the type systems that typically come with JS toolkits. If I recall correctly, Joose weighs in at about 16K gzipped, so it’s not a small package.

Nick Carter gave a Track B talk on his JS ORM, JazzRecord It’s a direct descendant of Rails’ ActiveRecord. It looks like a nice enough package, but seeing his samples made me that much more convinced that Atul is right: SQL does not belong in the browser. The sqlite storage engine may very well, but SQL itself does not. TaffyDB, dojo.data, CouchDB, whatever… just as long as the principal form of expression, persistence and querying is JS. The needs of a typical web client are very different from the kinds of things to which we apply SQL on the server. And, even then, people are starting to realize that SQL is not the best tool for every job.

I spoke on Track B about ServerJS. I called the session “a standard library for JavaScript in non-browser contexts”, or something catchy like that, because it is clear that what we’re building applies just as much to command line tools and other kinds of non-browser programs as it does to the server side of web apps.

My talk came immediately after James Duncan spoke about dynamic loading of C code into a SpiderMonkey environment. I lamely brought up ctypes as one approach, but dynamic loading of binaries is not my strength. I suggested that James would get useful feedback for his idea on the ServerJS list, given the number of people who are linking C/C++ libraries up with SpiderMonkey and v8.

As for ServerJS itself, I spoke about the project in general and our biggest milestone to date: the “SecurableModule”. I had a simple “math.js” module with a fibonacci function in it. I picked a lovely O(n^2) function to show the performance difference between Rhino and v8 which is considerable. Since math.js is a “pure JavaScript” module, I was able to demo that module being loaded into: Narwhal on Rhino, Narwhal on k7, Persevere, Helma NG (as a Jack app, no less) and GPSEE. I also demonstrated how with Narwhal/Rhino, I could “forget” to put a “var” in and my variable would not suddenly leak into the global namespace.

I also mentioned that we’re working on binary objects and files and that Jack (the interface) has good prospects given how proven the technique is in Python and Ruby.

I hope that at JSConf 2010, we’ll be able to see some significant apps built on a fairly complete platform.

Conference co-organizer Chris Williams thanked me for my endorsement of the conference, saying that I pushed Mozilla over the edge on sponsoring the conference. I had no idea I had such pull :). Anyhow, Mozilla’s sponsorship apparently had a direct impact on the conference food, which was quite beyond typical conference fare. Thanks to whomever it was at Mozilla who gave the a-ok on this.

As with any conference, the hallway track is among the most important, and I had a good time meeting new people and talking about a range of things. Community-driven conferences do bring in a good collection of people to meet.

I am doubtless leaving people out of this roundup, and I apologize for that. I am sure there are some other JSConf roundups that will provide additional insight. Also, the videos will be showing up online over time, so keep an eye out for that.

Speaking at JSConf on Saturday at 1:45PM

JSConf is coming up on Friday, and I’ll be there. I was too late to get a normal “track A” talk in, but I am scheduled in the “track B” room for Saturday at 1:45PM to talk about a standard library for JavaScript development outside of the browser. It’ll be a 30 minute session, and I’m hoping to have some cool stuff to show!

I hope to see you there!

This Week in Bespin March 2, 2009

For the week of February 23rd:

  • Roberto Saccon ported Bespin to Dojo! Dojo leader Peter Higgins helped to clean things up a bit. This has significantly changed (and improved) the use of namespaces in the frontend code, which will make Bespin much easier to drop into other pages.
  • For people keeping score at home, there is now a 0.1 branch in the Bespin repository that we will be using to update the server. The default branch has the Dojo work and all of the other forward-looking work going on. Until the trunk work stabilizes a bit, bespin.mozilla.com will be running a bit behind on the changes here.
  • autoindent setting. When you hit return/enter you’re placed at the same indentation point on the next line.
  • Select an area and hit TAB and it all indents. SHIFT+TAB and it unindents. Thanks to Jay Bird!
  • Special support for certain keys so German keyboards etc can get in ‘{‘ and the like. Thanks to Julian Viereck  
  • For people following the trunk/tip, the project build script has changed a bit. Be sure to follow the instructions in the top-level README file to get the Bespin server running.
  • PHP, Java and Aduino syntax highlighting, thanks to Sean Burke and Olle Jonsson, respectively. The Arduino support is optional. Use include(“bespin.syntax.arduino”); in your config.js to load.
  • Multithreaded development server, which speeds up page loads a bit now that there are more modules loading.
  • Config work has short cuts now so you can simply use: include(file), execute(cmd), publish(), subscribe() 
Coming up the week of March 2nd:
  • I’ll be working on backend changes to make VCS integration possible (and to allow us to start using Bespin as our primary editor when working on Bespin itself)
  • Joe will be working on collaboration implementation
  • Ben and Dion will likely have their hands full just keeping up with your patches! (I’m sure they will sneak some time in to improve the editor and Thunderhead as well 🙂

By the way, “This Week in Bespin” is not just for those of us at Mozilla. If you have anything you want to see show up in these summaries, just send me an email message!

Embarrassment Driven Development

Googling “embarrassment driven development” (EDD) does not return as many hits as it should. I think it’s a very powerful development technique. I first heard the expression from the Plone guys at PyCon 2006, and Google did turn up this match:

[ArchipelagoSprint] Time to get cracking on Plone 3.0!

Wrt. timelines, I was hoping that we could try to have a “Tech preview” release before the Plone Conference 2006 in Seattle (October 25-27) – I’m going to be on-stage there talking about the exciting new features of Plone 3.0 – and I’d like to not be booed off stage. Yes, this is embarrassment-driven development – as usual. 😉

That’s Alexander Limi illustrating the prime motivator for EDD.

The idea behind EDD is simple: if you have to demo something in front of an audience, and that something sucks, you will move hell or high water to make sure you don’t look like an idiot.

Every product has rough edges and warts, but no one wants a demo to be all warty and to have to say “yeah, I know you shouldn’t have to click to the left of the button, but we just haven’t gotten to that yet”. EDD ensures that, at least for the parts you have to get up and show, the rough edges will be smoothed in time for the show.

I’m going to be practicing EDD leading up to JSConf. I want to be able to show some useful, non-trivial bits of ServerJS work by then.

280Atlas looks amazing

If you haven’t seen it yet, you really should watch the demo of 280Atlas. It’s basically Interface Builder written for the browser. Just watch their demo and tell me that that doesn’t look like a great way to build webapps (and I do mean apps and not sites).

A lot of work goes into designing good software. 280North are taking the great design work put in on Apple’s developer tools and making it available to people building browser-based apps. They’ve saved themselves the work of inventing every little piece of the model, instead focusing on the work of making it work well in JavaScript.

Awesome work, guys!

This Week in Bespin (Feb 23)

I’m crossposting this from the Bespin googlegroup.

Before moving on to the week that is coming, let’s look back at the week that was:

Week of February 17th:

  • Version 0.1.2 of Bespin was released. Notable changes:
  • System clipboard copy and paste now works in WebKit. Firefox will almost work. Vote up this bug:   https://bugzilla.mozilla.org/show_bug.cgi?id=407983 
  • Project names now need only to be unique per user. Previously, they had to be globally unique. This reflects a bit of a change to how we were viewing collaboration. (More on collaboration to come).
  • Thanks to that change, the sample project has been renamed from SampleProjectFor:yourusername to just SampleProject.
  • Speaking of project renaming, you can now rename projects from the dashboard or the editor using the “renameproject” command. New “createproject” and “deleteproject” commands make project-level changes much clearer.
  • Every user now has a visible “BespinSettings” project. You probably      won’t be surprised to hear that it stores settings for Bespin! You can put custom code (set autoconfig on and put code in config.js) and commands (use the cmd* commands) in there.
  • The login form is now a real form, which means that your password manager can save your Bespin login information.
  • Client and server implementations can make sure they’re in sync with the X-Bespin-API HTTP header that is returned from the server. The current API version is 2 (which adds the renameproject API).
  • key repetition works for arrows and backspace and lots of other minor fixes.
  • As Dion noted, Boris Bokowski and Simon Kaegi coupled the Bespin Editor with a headless Eclipse . The screenshot showing Java compiler warnings within Bespin is worth at least a couple thousand words!
  • Jerome Velociter got the Bespin editor running in XWiki . Avi Bryant hooked the dashboard and editor up to Squeak :. It’s been amazing to see the integration going on so far.
  • Oscar Carballa kicked off the creation of a User Guide for Bespin
  • Mozilla’s Joe Walker started working on the collaboration spec. The model that he’s looking at is designed to allow you to share a project with many people and give them access to view it/collaborate on it in different ways. So, you’ll be able to selectively use collaborative editing with some people, do a “webcast” style sharing with others and more.
  • Three more google groups have been announced: bespin-core covers the development of Bespin’s code, bespin-dev is for people who want to develop extensions for Bespin. Community member Guilherme Santos created the bespinbrasil group for Brazilian Bespin users.
  • Coming up this week:

    • Ben and Dion are in Miami for Future of Web Apps where they will be showing off Bespin.
    • Joe is continue to work on collaboration and is experimenting with Neil Fraser’s mobwrite.
    • I will finally be getting to work on the version control spec, and hope to be starting on the version control backend code.
    • And we’re all going to be working on bug fixes and minor enhancements.

    Bespin: code in the cloud

    Despite working for an open source company, I have been pretty quiet here about what I’ve been doing in the Mozilla Labs web developer tools group. No more. We’ve gone public!

    Mozilla Labs » Blog Archive » Introducing Bespin

    Bespin proposes an open extensible web-based framework for code editing that aims to increase developer productivity, enable compelling user experiences, and promote the use of open standards.

    I started working on Bespin as soon as I joined Mozilla, hitting the ground running with a new Python server. Ben and Dion had already done a lot of work and experimentation on Bespin prior to joining Mozilla in December, so I must confess that I am still fairly ignorant about the Canvas-based magic that they’re doing in the UI. But, Bespin has an architecture that lends itself well to selective ignorance: the server provides a RESTful API, and the client is responsible for all of the presentation. For their part, Ben and Dion have been able to be blissfully ignorant about the inner workings of the Python server.

    Of course, I’m not a JavaScript noob and have done some work in the client, but my focus has been the server. Now that we’re out in the open, you can definitely expect that we’ll be talking more about how things work and how you can bend Bespin to your will. Bespin is honest to goodness open source (MPL-licensed), so it becomes an open and collaborative effort starting right away.

    The initial reaction has been fantastic. There are tons of people hanging out in #bespin on irc.mozilla.org, and the mailing list has grown to a couple hundred members already. Thanks to everyone for jumping in with your thoughts and patches!

    Here’s some of the coverage:

    Dion’s post at Ajaxian:

    From Dion’s personal blog:

    Foolish chaps and companies have come to me in the past thinking that open source will be a silver bullet for “getting other people to do our work.” Those that have been involved in open source know that it isn’t the case. It is often more work. But, it is worth it.

    From Ars Technica:

    The project is still at an early stage of development and there is clearly a lot of work to be done before it will be able to deliver the same practical value as existing desktop editors. Despite the limitations, it shows an enormous amount of promise and has the potential to eventually deliver a user experience that rivals even the best text editors.

    From Five Questions with Dion Almaer:

    Now the browsers are moving fast again and building a first class platform for us to develop, the Open Web Platform. Instead of getting bogged down thinking about what IE 6 gives you, take some time to think about what you could build with the latest technology. I realise that you have to be pragmatic and get things working with your audience, but browsers are changing, and so are expectations.

    From What Mozilla’s Bespin Bespeaks (ComputerworldUK):

    You can see that Bespin is ticking all the Mozilla boxes, but what’s also striking is that this is a Web-based project: Mozilla is entering the cloud. It’s a further shift to viewing the Web as a platform for doing, well, just about anything. Clearly, against that background, open standards are even more important. And not only for the code: another issue that Mozilla will need to start addressing publicly is that of open data. As more stuff moves into the cloud, it become imperative to establish minimum standards for access to the data that is held there. I look forward to hearing Mozilla’s views on the subject.

    While I certainly don’t speak for Mozilla, I would be extremely surprised if there’s anyone at Mozilla that believes that users should have anything less than full access and ability to take their data with them. There can be technical issues involved in providing the data, but the data should be available in some reasonable form. Bespin, for its part, makes it easy to export a project in a tarball or zipfile.

    I was surprised to see Bespin covered even on Lifehacker:

    Primarily, Bespin is a text editor—the kind you’d use for editing code or managing text-based todos. Using Bespin, developers could collaborate on projects through a unified interface (that still supports plugins!) no matter where they are—so long as they’ve got a browser.

    cnet has the story, too:

    For example, what about integration with open-source software repositories? If it’s flexible enough, Bespin could essentially act as a source code viewer that repositories such as SourceForge or Google Code could employ.

    A nice writeup on the ReadWriteWeb as well:

    It’s clear that a great deal of thought and attention went into this early version – and it’s a safe bet that it will only get more impressive as time goes on.

    RWW last month surprised me with their coverage of me joining Mozilla.

    I’m having a great time at Mozilla so far, and it’s great to be out in the open working with so many people now on Bespin and ServerJS.

    ServerJS: one week into building a better JavaScript

    It’s actually been 8 days now since I introduced the ServerJS group. As Peter Michaux mentions in “A Bright Future for Standardized Server-Side JavaScript”, the group has gotten off to an absolutely blazing start. As it stands this morning, there are 224 members who have posted 653 messages in 8 days.

    There’s something more important than the numbers, though. Many people who have joined the list are deeply experienced in JavaScript-outside-of-the-browser. Everyone comes along with experience from other programming languages and environments. And all of knowledge and experience is being put to work in figuring out the approaches we want to take with modules, files and binary objects, as just a few starter topics.

    You can see some of the work in progress on the ServerJS wiki. Note that the wiki has moved to wiki.mozilla.org.

    The interest in server side JavaScript is self-evident, given all of the different projects and frameworks that have grown up around the concept. What’s new here is that these projects are getting together to form a real ecosystem, rather than standalone tools.

    I think that better server side JavaScript tools will ultimately help web developers produce better user experiences more quickly, which fits in quite nicely with our goals in the Mozilla Labs web development tools group.

    I expect that the coming week will have some even more interesting discussions than the last, so now is a great time to join up if you have ideas on what you want to see in JavaScript as a server side scripting platform.

    Server JavaScript wiki and IRC

    The server side web development chatter over the past few years has been centered around Ruby, Python, the latest frameworks in Java and Groovy, plus the endless swirl of activity in PHP. At least, that’s the chatter I’ve been seeing.

    Clearly, there is pent up demand to make JavaScript into a not only competitive but truly awesome server side web dev environment. I launched the ServerJS mailing list yesterday and as of now it has 136 members and a lot of active discussion. This is a very promising start.

    Of course, discussion is not the same as code. Luckily, there’s no shortage of code. It’s a matter of picking the right code and sticking with it.

    To help collect up all of the useful information that’s coming up, I have set up a wiki at Mozilla’s Developer Center:

    ServerJS – MDC

    At Tom Robinson‘s suggestion, we are also meeting on IRC in #serverjs on freenode.

    The energy behind this project is huge and I think a great new programming environment will ultimately emerge.

    What Server Side JavaScript needs

    Server side JavaScript technology has been around for a long time. Netscape offered server side JavaScript in their server software back in 1996, and Helma has existed for a number of years as well. But, server side development has changed a lot over the past few years.

    Aptana’s Jaxer gives an innovative view of how you can leverage a JavaScript environment running on both sides of the wire (client and server). Very convenient communication and the ability to easily share code between client and server are big benefits of running JavaScript on the server.

    Jaxer and Helma are interesting projects, to be sure (and there are many others!). But what I see missing from JavaScript on the server is not interesting projects, but rather a useful ecosystem. People working in Python like to talk about the fragmentation of web frameworks and whatnot, but that’s nothing compared to the fragmentation of JavaScript.

    For example, JavaScript needs a cross-interpreter standard library. Thankfully, some amount of standard library exists (the part inherited from the browsers). So, you get regular expressions and dates. But, what about files and directories? Why can’t the same API be made to work in Rhino, Spidermonkey, V8 and JSCore?

    A handful of standard interfaces. Connecting to a database and running queries is a well understood and common problem. In Rhino, you get to use JDBC. But, JavaScript really should have its own cross-interpreter standard like Python’s DBAPI. It should also be possible to take a webapp that was originally deployed behind an Apache module running Spidermonkey and then put it behind Jetty thanks to a standard web server/web app interface.

    JavaScript needs a standard way to include other modules and for those modules to live in discreet namespaces. There are easy ways to do namespaces, but there’s no standard programmatic way to load a module (once!). This is really important, because server side apps can include a lot of code and will likely mix and match parts that meet those standard interfaces.

    There needs to be a way to package up code for deployment and distribution and further to install packages. Linux people will correctly point out that they can just type “apt get” (or yum, or whatever) and their work is done. But there are a lot of people using Macs and Windows who need a convenient way to get their development environments set up and to package up the code they write for deployment and for other people to use.

    Part of the distribution and installation problem is a package repository. I don’t know if JSAN is the answer there, but I do know that an easy way to install a package and its dependencies makes a huge difference in how many libraries people will likely pull together in their apps.

    And, on top of all of this goodness, we’d get template engines, object relational mappers, middleware, packaged apps, etc. In fact, many of those things already exist. But, the problem is that they have no common basis to sit on. And that’s what prevents an ecosystem from growing.

    If you search the Python Package Index for WSGI (the Python standard for connecting webapps with web servers), you’ll find 180 packages today… servers, middleware, full blown applications. And those are just the packages that have “WSGI” in their listings. That’s what an ecosystem looks like. And Java has one, and Ruby has one, and JavaScript needs one.

    It’s also worth noting that many of those WSGI components can run unchanged on CPython, Jython and IronPython, thanks to a common standard library. JavaScript has a collection of implementations in C, as well as Java and .Net implementations and there just needs to be a little agreement on some interfaces and such. Libraries that run in all of those places can attract more users and, hopefully, more committers to help the libraries grow.

    What I’m describing here is not a technical problem. It’s a matter of people getting together and making a decision to step forward and start building up something bigger and cooler together.

    To that end, I’ve set up a new ServerJS group in hopes of getting the interested parties talking and maybe even to get us together face-to-face to produce some code and settle on some interfaces. There’s a tremendous collection of JavaScript code out there already, and let’s see if we can make all of that code that much more valuable.

    In the web developer tools group at Mozilla, we have a wide open charter to help software developers make the best use of the open web. Doing our bit to help the server side JavaScript community grow and flourish can certainly be a part of that.

    (Before someone says “why not just use Ruby/Python/Java/C#?” let me just say that that is an entirely different question and I won’t address that in this post.)

    Update: The group is now called CommonJS.