Side Effects vs. Promises

This past weekend, I saw Chris Armstrong (@radix) talk about Side Effects as Public API. From the description on the Strange Loop page for the talk:

Haskellers have been isolating their side-effects from their pure code for decades, but most people write code in languages that allow ad hoc side-effects anywhere. In a lot of cases, these side-effects are practically a part of the public API — they put restrictions the way the code can be used, often obscurely, and have effects that are observable to the caller. It helps a lot to acknowledge this and explicitly treat them as a part of your public API. Here are the basic ideas that I’ll cover:

  • represent your side-effects as “intents to perform an action” — transparent objects that expose the details of the side-effect, without actually performing it
  • combine effects with pure code to describe the execution order and data dependencies (yes, like a monad)
  • write unit tests for your code without mocking, by specifying the expected content, results, and order of side-effects performed by a function

Chris has made a Python library called Effect that implements the ideas he was presenting.

In JavaScript-land, we’ve embraced the idea of promises now to the point that they’re a standard part of the language, already implemented in multiple browsers (about 66% of those in use!) and in Node. Promises can clean up your asynchronous code and if you’re not familiar with them, you should go read one of the many tutorials about them.

A promise is a placeholder for a value to come along at some point in the future. Using the then method on the Promise, you can pass in a function to be called when the value is ready. Here’s a hypothetical example:

function getItemOne() {
    let promise = get("http://foo.barbaz/api/item/1").then((value) => {
        // Do something with the result
    }, (error) => {
        // Do something with the error
    });
    return promise;
}

This example above is not very different from the use of callbacks and isn’t intended to show off the nice attributes of promises. It is a good example to illustrate the use of an Effect rather than a Promise.

The first thing I’ll note is that getItemOne is not a pure function. Calling it produces a side effect: an HTTP request. Testing this function, or any function that calls it, is difficult because of that impurity. The usual solution for testing something like this is to replace get with a mock function. That doesn’t really fix the impurity of the function, but it at least makes it testable.

We can make getItemOne pure by making a small change like this:

function getItemOne() {
    return new SideEffect({
        type: "http-get",
        url: "http://foo.barbaz/api/item/1"
    }).on((value) => {
        // Do something with the result
    }, (error) => {
        // Do something with the error
    });
}

I’m not suggesting that we’d use the API above, because I think we can do better than that. In fact, this version could look the same as the previous version, if we wanted it to. But, I wanted to clearly show the difference between promises and the side effects about which I’m writing.

The function above is pure. Every time you call it, it will return exactly the same value. Calling this from a test is easy and you can verify that the side effect returned matches your expectations. You can also pass in values to test what happens in the success or failure cases, which would also be pure functions.

In a real system, somewhere up the call chain there will be a dispatcher called to actually process the side effect. The Python Effect library points out that this approach allows you to easily swap out how any given side effect is processed.

I asked Chris if he knew of any JS implementations of Effect. He wasn’t aware of any, but he pointed out that it’s basically just the IO monad. Here’s an implementation of the IO monad in JS. If you look at the Python Effect library and its examples, though, I think there’s a need for a JS library that fits in better with the overall JS ecosystem (test runners and whatnot).

In practice, returning side effects rather than performing them and returning promises can increase the size of the “functional core” of your application, which is a win in my book.

The Tale of the Merry Squid: Teaching Programming with Minecraft

The Merry Squid

This past January and February, I taught a class that introduces children to programming through Minecraft. I called the class “The Merry Squid”, and this is my story of how it went. For the benefit of those who don’t want to read the whole thing, I’ll note up front that the class is available online.

Programming in Minecraft

There are so many things that I think are great about Minecraft. Since it’s written in Java, people could use a bit of perseverance (okay, a lot) to figure out how to hook in to Minecraft’s normal operation and change it. If you’ve played Minecraft, you’ve probably already heard about “mods” that do this.

One Minecraft mod is called ScriptCraft. ScriptCraft is a mod that lets you write mods, in JavaScript. Even better, it lets you run bits of JavaScript code from within Minecraft itself.

Minecraft is set up for multiplayer games, so there’s both the client part that runs the graphics and the server part which keeps track of the state of everything in the world. ScriptCraft is a server mod, so that means you can only change things that affect the state of the world and not the graphics that appear on screen. There’s an awful lot you can do within those boundaries, though!

Possibly the best thing about Minecraft for teaching programming is that many kids love it. There’s a strong built-in motivation for them to try things out and see what they can create.

From Logo to Minecraft

Logo was possibly the first “teaching” language that was geared toward kids. Using simple commands, children could draw pictures by moving a “turtle” around on the screen or even an actual physical robot on a piece of paper.

ScriptCraft includes a JavaScript object called “the Drone” that feels a bit like the turtle from Logo. Using functions on the Drone, you can “draw” with Minecraft blocks. Want to make a 10x10x10 cube? Normally, that would mean placing 1,000 blocks, one at a time. Using the /js command provided by ScriptCraft turns that into just a quick bit of typing.

That instant feedback of typing a command and immediately seeing the result in your Minecraft world is fantastic. This seemed like the perfect entry point for getting children to do some JavaScript.

Setup Pain

As easy as it was to start coding with ScriptCraft, getting ScriptCraft itself running was not easy for many.

Most of the parents of the children in my class are not “computer people”. I created step-by-step instructions to set up Minecraft with the required mods on both Windows and Mac, but setup was still not easy. An automated installer would have been nice, but I didn’t have time to build one.

To make matters more challenging, the class met in a building that has barely functional Internet access. USB flash drives are not quite a thing of the past yet.

Also, some of the computers launched programs very slowly. I didn’t have time to track those problems down either, because I had a class full of kids, so my focus was on making sure that everything could run, even if launching was slow.

It’s worth noting that Minecraft is a fairly resource intensive program. That’s probably why Minecraft Pocket Edition was a ground up rewrite in C++: desktop Minecraft takes a pretty beefy computer to run acceptably.

Other local people who teach Minecraft classes bring their own equipment and teach with Minecraft: Pi Edition. This certainly gets rid of the setup pain, but the Pi Edition is nowhere near as interesting as the things you can do with ScriptCraft.

LearnToMod launched just as my class was starting. It sounds like they run the server for you which certainly makes life easier for the participants.

Graduating to code in files

There’s a limit to how much you can do from Minecraft’s little command line. It’s like writing programs in tweets. To do anything really cool, you need to edit JavaScript code in files.

The experience of modding Minecraft with JavaScript is still pretty great. You don’t quite have the immediacy of entering code in the command line, but you can dynamically reload your code without having to restart Minecraft. The iteration loops are still quick.

I got everyone set up with Brackets because I knew the editor well and I knew that the out-of-the-box JavaScript listing would come in handy.

Everyone was successful in running the new scripts that I provided each week. Sometimes they made their own customizations to the scripts, and some of the children clearly were writing some code of their own with varying degrees of success.

My “flipped classroom” experience

I taught this class as a “flipped” class: I’d post a video each week which provided the new material to learn and then planned for us to do more together during class time. My thinking with this was that by doing the experimentation at home after watching the videos, the kids could spend as much time as they wanted playing around with it.

When class times came around, I found that sometimes I needed to do a fair bit of tech support and that the environment hadn’t quite encouraged the kids to work together on things. There was a bit of “hey! look what I did!” that happened, but not quite as much as I’d have liked.

My Egg Hunt mini game from the last session was ridiculously complicated, but I think it did serve well to show off what was possible and was a fun activity that we all did together on the last day in class.

The flipped classroom approach is great because I could put the class online once it was done. At least one parent and child have taken a look and found it helpful:

Fun was had by all!

These children were already fans of Minecraft, but were entirely new to writing code. I tried to balance doing fun things with Minecraft that they couldn’t have done before with learning some JavaScript, putting the greater emphasis on fun-with-Minecraft than learning-JavaScript.

Despite the challenges that I’ve mentioned here, I had a good time creating and running the class and the impression that I got from the kids was that they enjoyed taking the class.

What did I learn?

I learned as much as the children did through this experience.

  • The ScriptCraft drone with the in-game REPL is brilliant. Kids love it and can make good use of it.
  • LearnToMod sounds awesome. Their approach (run the servers on behalf of the learners) makes a whole lot of sense. Alas, with the poor Internet access we had, doing the class in conjunction with LearnToMod would not have been an option.
  • The creator of ScriptCraft has since published a book (A Beginner’s Guide to Writing Minecraft Plugins in JavaScript). Teaching a class based on pre-existing material like that could have made my life easier.
  • I shouldn’t sign up to do a class when I’m already oversubscribed. At the time that I started exploring the class, I thought there would be no trouble… but I had so much going on when the class started that it was a struggle to sneak in my work for the class. The timing was just not quite ideal.
  • Even though the flipped classroom approach wasn’t as smooth as I hoped, I would definitely do that again. Especially when it comes to homeschooled kids, it makes sense for the kids to spend as much time exploring as they want. I would want to make sure that the in-class activity time could be smoother than the way it went for this class.

Minecraft is used for teaching a wide variety of things, and ScriptCraft offers a lot for people who want to learn programming in Minecraft, once you get past the setup.

Measuring and patching phase for React components

On the latest episode of JavaScript Jabber, Amit Kaufman and Avi Marcus talk about how your web app performance is killed if you interleave your reading and writing from the DOM. For their now React-based app, they needed to measure certain elements for their layout work. Their solution sounds clever: components specify what information they’re going to need and they batch up all of the DOM reads and do them all at once and then they do all of the adjustments that come as a result of those reads all in one separate pass. This minimizes the amount of churn and recalculation that the browser needs to deal with.

Custom messages for Jasmine expectations

jasmine-custom-message gives you a “custom failure message on any jasmine assertion”:

describe('the story', function() {
  it('should finish ok', function() {
    since('all cats are grey in the dark').
    expect('tiger').toEqual('kitty'); // => 'all cats are grey in the dark'
  });
});

Overall, I like Jasmine as a testing library, but this particular limitation has really bothered me.

How the Web Evolves

Two years ago, I made a mistake in posting this overly succinct statement to Google+:

Web SQL Database needs to die. The sooner IndexedDB is in the hands of developers the better.

Tweet-sized statements often don’t capture enough of the nuances of a thought to communicate well.

My big mistake was in not being clear enough about why Web SQL Database needed to die. I tried to explain why the proposed standard was problematic for some browser vendors and such, but fundamentally my opinion is really that we needed to have some standard way to store reasonable amounts of data for offline and online uses and to be able to access that data efficently. Web SQL DB would meet that criteria if it weren’t dead. I’m sympathetic to the issues that some browsers have with Web SQL DB (I did work for Mozilla, after all!), but at the end of it I just really want the web platform to have all of the capabilities it needs. Data storage is a pretty basic thing.

Of course, reality is more complicated than “Web SQL DB is dead”. All of those hundreds of millions of iOS devices today only support Web SQL and not IndexedDB. Many people on the Google+ thread have a strong preference for SQL vs. the API that IndexedDB has to offer. But, the fact remains that the Web SQL Database proposed standard has had a giant disclaimer at the top since 2010 stating that it has reached an impasse.

But, this blog post is not about WebSQL DB vs. IndexedDB. Web platform features like these don’t just poof into existence. With today’s process, these features are designed, tested in browsers, formalized, argued about and standardized by various groups of people. The web is not like some proprietary platform where a vendor suddenly drops a new version with a bunch of new features on everyone. By knowing how the standards come to exist, you can help ensure that the platform does what it needs to do for your apps.

Alex Russell spoke with the people of JavaScript Jabber about TC-39 (the group that standardizes JavaScript), but Alex also has a lot to say about the evolution of the rest of the web platform as well. If you’ve ever had trouble with HTML5 application cache while trying to make an offline web app, Alex has been working on a new API, Service Worker, that will make your life better. He’s also been quite involved in the Web Components work. And, of course, as one of the founders of the Dojo Toolkit, he’s been at this for a bit longer than just about anyone.

Yehuda Katz gave a talk a few months ago (“The Future of the Client Side Web”) in which he spoke about how the standards are made and where they’re going. Yehuda also has tons of experience with both server side and client side development and he’s part of both Ember and jQuery core teams.

Alex and Yehuda are real-world web developers who have taken the step of helping to build out the standards themselves. They are both part of the W3C’s Technical Architecture Group (TAG). Speaking of which, TAG elections are coming up and the super-sharp David Herman (who has done a ton of amazing work on modules for the next version of JavaScript) and Domenic Denicola (who has helped tremendously in pushing Promises for JavaScript) are running for the TAG.

The web platform is built by real people who want the platform to be the best it can be. Understanding this is the best way to ensure that the web gets the features that you need for your applications.

Optimizing JavaScript Performance Through Custom Memory Allocation

(Translations available, see the end of the post)

Mozilla’s Kannan Vijayan had an intriguing result in running SunSpider ported to C++, asm.js and Dalvik. In the “Binary Trees” test, asm.js was the fastest by far. Kannan’s untested theory is that it boils down to memory allocation performance:

In asm.js, the compiled C++ is executed using a Javascript typed array as the “machine memory”. The machine memory is allocated once and then used for the entire program runtime. Calls to malloc from asm.js programs perform no syscalls. This may be the reason behind asm.js-compiled C++ doing better than native-compiled C++, but requires further investigation to confirm.

In the Hacker News discussion there were some comments there about the memory performance. duaneb said:

Anyone who has implemented a memory allocator knows how expensive it can be. If you have a simpler algorithm that works with your data, allocate a huge chunk and manage it yourself.

Some game developers reuse objects as a way to avoid unnecessary allocations/GC. An article was just posted in Smashing Magazine about Emmet LiveStyle which talks about how they reused objects in order to save allocations.

I don’t think that selective reuse of big chunks of memory is a tool in most JavaScript developers’ toolboxes right now, but it seems like a good idea in cases where you need consistent and smooth performance.

Update: Vyacheslav Egorov emailed me a link to Kannan’s graph that includes the JavaScript engine performance. In the binary tree case, the JavaScript implementation was the fastest by far. Perhaps the C++ was not well-tuned.

I don’t want to get too hung up on the benchmarks, because my main point is not “asm.js is faster than C++” (in fact, I’m not stating that at all). My point is that there ways to control memory management in JS that may be a non-obvious way to improve responsiveness.

(Russian translation available)

???? ???????? ?? ??????? ????? via ?????????????? ?? Softdroid: ???????????
?????????????????? JavaScript
.

Tools for Web Developers (1DevDay)

This past Saturday, I had the pleasure of speaking at 1DevDay Detroit 2011. The event was held at the fantastic Compuware headquarters in downtown Detroit. It’s a beautiful building, and the training center facility works well for events like 1DevDay.

I spoke about “New Tools for the Open Web”. My goal was to show off a bunch of tools that people might find useful and to talk about tools for the future. It’s impossible to talk about all of the great stuff that’s going on. The world of open source tools for the web is huge and growing, and there was a standing-room-only crowd for my talk (thanks for packing in there and letting me run over my timeslot a little bit!)

I started with some tools that aren’t new, hopefully showing off some cool features along the way.

  • The Web Developer Toolbar has an incredible number of utilities packed into its menus. It’s not new, but it should be in your toolbox.
  • Firebug has had a huge influence on tools for web developers and continues to get awesome new features (such as more fields on the Net Panel and the ability to select which ones you want). I also pointed out Firebug’s non-standard but amazingly useful control for quickly selecting script files on the script page.
  • I spoke about a couple of neat JavaScript features in the Chrome Developer Tools (the ability to tweak JS on the fly and the ability to prettify JS while you’re debugging)
  • I showed off the Web Console and Scratchpad tools from Firefox, in particular showing how Scratchpad provides a very clean, usable interface for experimenting with JS.
  • I gave a sneak preview of the new Firefox styling tools that will very soon be in Firefox Aurora builds.
  • I showed of the Firefox style editor which should be in the next Aurora. For now, it’s available as an add-on
  • I also demoed Tilt, an amazing 3D visualization of the DOM

In a category all its own is Weinre which provides a way to debug apps running on a mobile device.

Then, I stepped into a collection of CSS tools.

  • ReCSS is a bookmarklet that’s been available forever. One tap of the bookmarklet reloads your CSS without refreshing the page.
  • LESS extends the CSS language with a variety of useful features and compiles down to standard CSS to run in the browser. It’s written in JavaScript, so you can do the processing client side or server side. Neat trick: if you are doing client side processing, you can add #!watch to your URL and LESS will periodically refresh the CSS from the server automatically. Make a change on disk and see the change almost immediately in the browser. I also mentioned Sass and Stylus as alternatives.
  • CSS Prefixer helps you out by expanding your CSS3 declarations to work in all of the browsers that support them with a prefix (-moz, -webkit, etc.)
  • Prefix Free is an alternative to CSS Prefixer that works client side and adds only the declarations needed for the user’s browser.
  • Rainbow is a Firefox add-on for working with colors. Among its many neat features, it will give you the color palette for the page you’re looking at. The Colour Bookmarklet does something similar as a cross-browser bookmarklet.
  • DOM Monster is a cross-browser bookmarklet that will take a look at your page and give you specific advice on how to make it faster

I put out a call on Twitter and G+ for suggestions of tools that people rely on for their web development. Several cited their editors, so I thought I’d mention a couple.

  • JSFiddle takes the pastebin concept a couple of steps further by giving you a way to easily share snippets of HTML/CSS/JS that are put together into a preview pane. JSBin is an alternative
  • The Eclipse Orion project is building a browser-based IDE with workflows and plugins that feel very much “like the web”.We ship the Orion text editor component in Firefox
  • Cloud9 IDE provides a beautiful and feature-rich IDE for working on both the client and server parts of your application… all in the cloud!

I didn’t talk a whole lot about JavaScript-specific tools, because a lot of what we use for JS are libraries and not tools.

  • I showed JSHint, a fork of Douglas Crockford’s ever-popular JSLint. JSHint provides nice static analysis of your JavaScript code to catch some errors early
  • CoffeeScript is a nice language that provides a friendlier syntax on top of JS without changing the semantics. This means that the JS it produces is quite readable (and quite close to what you’d write by hand). I’m still hoping that some of CoffeeScript’s best parts make it into ECMAScript.next (and there’s a good chance that a lot will).
  • Using a language like CoffeeScript makes debugging a bit harder. I showed off our not-quite-ready-yet source maps feature for Firefox. Source maps will also be really useful for debugging minified JavaScript.

I spoke a little about docs. Documentation is important, and the more we do to have better docs for our tools and libraries, the better.

  • I think JQAPI is awesome. They took the standard jQuery docs and reformatted them to be more useful.
  • Docco lets you do literate-style programming in JavaScript (or Python or any of the other ports that are available).
  • Eloquent JavaScript is a book that teaches you JavaScript and provides interactive exercises to help with the learning

Then, I spoke a bit about the future.

Almost everything I showed was created in HTML, CSS and JavaScript. If you can make web apps, you can make tools. Even the tools that are built into the browsers are built of the same basic stuff.

Invest in your tools:

  • Find the good ones, use them and tell others!
  • Give feedback (this really helps people who make tools to make them better)
  • Make them better
  • Make entirely new ones

I mentioned how some tools, like Matthew Claypotch’s localStorage bookmarklet are very simple to write but can meet your needs precisely and even show toolmakers what you’d like to have.

Other tools mentioned to me by others but not part of my talk:

  • Glimpse is a client-side view into a server-side ASP.net applicaiton
  • Edit This Cookie is a Chrome extension for editing cookies. There’s also Firecookie for Firebug
  • YSlow is a tool from Yahoo! for spotting performance problems with your pages
  • stat.js is a bookmarklet that gives you information (such as frames per second) for the page you’re looking at
  • h5o provides an outline view for HTML5 documents
  • LiveReload automatically reloads your JS and CSS as you change it (oh, and it will recompile your CoffeeScript, Sass or LESS for you at the same time)
  • ReloadEvery is a Firefox extension to reload periodically automatically
  • redbot spots problems with the way you’re serving up your pages
  • HttpFox is a Firefox extension that provides an alternative view of network requests
  • ColorZilla is another set of color tools for Firefox
  • Fiddler is a web-debugging proxy

And there are many, many more. It’s a good time to be building web applications.

Browser-based command lines are awesome

Command lines are an amazingly great way to get things done using the keyboard, and using the keyboard is a way to do many things faster than reaching for the mouse. Of course, I’m a geek and not everyone likes command lines. But, many geeks do… and, I think that what we think of as a command line can move way beyond what a command line was in the ’70s.

TermKit recently got a good deal of attention, and rightly so. Steven Wittens has created a beautiful command shell that runs in a dedicated browser.

We did a good bit of command line experimentation on the Bespin project, and Joe Walker has continued that experimentation with the GCLI project. GCLI has been pushing in some directions that Joe has had in mind for a long time (a command line interface that is also mouse-friendly). At work, we’ve got some big plans for that one. Ajax.org also has a command line (not yet GCLI-based) in their slick Cloud9 IDE.

There’s also been Ubiquity, which had a “natural language” take on the command line. A few months ago, Ian Bicking created his take on a browser-based command shell with jsshell.

All of these are exclusive of the many JavaScript REPLs that exist and have varying and interesting sets of features.

All of these are something that we could loosely call a “webby CLI”. So, I’ve set up a webby-cli googlegroup for people who want to talk about the concepts, projects and niggling implementation details around bringing a great command line experience into browsery environments. Join us, if you’re interested in that sort of thing.

More challenging puzzle for the devtools job

Yesterday, I posted about the Developer Tools Engineer position we’re hiring for at Mozilla. The response to my little puzzle has been terrific, but “it’s too easy” was a common refrain. So, I decided to add two more little puzzles that follow the first. The puzzles are not intended to be time consuming, but they do use a couple of modern browser features.

Give it a whirl and say hi if you make it all the way through. One person completed it last night after I put it up but before I had a chance to write this blog post.

Update: it looks like this may only work in Firefox at the moment, and possibly only Firefox 4.

Work with me at Mozilla! Solve the puzzle

My first two years at Mozilla have been great fun and an amazingly positive experience. This year promises to be even more fun, where I define fun as going after some ambitious goals. This year, we’re going to be building some very cool tools and setting the stage to push the boundaries of web developer tools even farther in 2012.

Now is your chance to get in on the ground floor of this work. Be a part of the open web and help us make better tools. It doesn’t matter where you’re located… if you’re an amazing web developer or have deep developer tools knowledge, I want to talk to you.

There are two ways to apply:

  1. Read the job posting and apply normally
  2. Solve the puzzle for extra credit

I hope to hear from you!