Blue Sky On Mars

Is an evolution of the Open Web in progress right now? Undoubtedly… I took a look at some of the players in Flash, Silverlight and the Open Web on the SitePen blog.

While I don’t have first hand knowledge of why Apple is doing it, I think I’ve got a pretty good idea of Why Apple is Investing in WebKit Performance.

I’m at PyCon right now and don’t have much time to blog (I really try to maximize my limited conference time). But, I can’t go without bringing attention to: SitePen Blog » Commercial Support Now Available from the Co-Creators and Contributors of the Dojo Toolkit, DWR, and Cometd, because this is what I’ve been working on lately.

SitePen Support is the first “product” that I’ve worked on at SitePen (services can be products, too). We’ve got a great service to offer to people using the Dojo, DWR and Cometd open source projects and I’ll have more to talk about soon. We’ve been doing some really cool stuff with the UI for our support service, and this tied in directly with the talk that I gave at PyCon yesterday. That talk will be online sometime soon.

Yesterday, Adobe shipped AIR 1.0, an open source toolkit and runtime for creating desktop apps using web technologies. For a time there, I really wondered why someone would want AIR, rather than just building a normal webapp that runs from the server. I can now think of four reasons:

1. offline - This one is obvious. AIR is a Google Gears competitor as a way to take a webapp offline.
2. performance - Rather than relying on the browsers’ caching behavior to make parts of your software load snappily, just ship the whole app to the user so that the whole thing is local as they navigate.
3. browser consistency - Maybe you’re sick of trying to make IE work (I wouldn’t blame you for that, really). AIR uses WebKit, which is a nice browser platform to target. Of course, this is a lame reason to use AIR. Forcing one’s users to install the AIR runtime and accept the download of one’s app rather than just using a good JavaScript toolkit and common cross-browser coding practices seems like a bad attitude toward one’s users.
4. local data access - You can manipulate files on the user’s hard drive. This lets you more comfortably do things like build a password manager. Pownce uses AIR to allow people to conveniently drag-and-drop files into the Pownce service.

Installing your first AIR app is amazingly easy. I’m not sure what trickery Adobe pulled to do that, but it’s pretty cool. The only app I’ve used so far is eBay Desktop. In my opinion, there’s no reason that app is an AIR app rather than just a normal webapp. As far as I can tell, you need to be online to do anything useful with it. I don’t think any of the 4 criteria above apply to eBay Desktop.

It’s also notable that SitePen has announced AIR support with Dojo 1.1. That means that you can build cross-platform desktop applications using Dojo. That’s nifty.

AIR is a gamble for Adobe. From what I’ve seen, they seem to be putting a lot of money into it. Time will tell if any truly compelling apps appear.

My first blog entry just went up over on the SitePen blog. I expanded on two topics from CodeMash: concurrency and non-relational databases. If you read my brain dump on here a couple days ago, you might want to check out the SitePen entry if those two topics interest you.

I spoke at CodeMash for a second year in a row. I think this year’s conference was even better than last year’s, particularly for me because I was only able to attend for one day last year. It’s a shame that CodeMash has such regional attendance, because where else do you find a conference with talks on C#, LINQ, Silverlight, Python (3 talks!), Rails, Dojo, Concurrency in Java and more?

It might seem like such a broad range of topics dilutes the conference, but it’s really cool because it gives you a chance to gain exposure to things you wouldn’t normally encounter. There are a lot of interesting things going on in the Microsoft world, but I don’t normally encounter those since I travel mostly in the “open source” world.

The only thing that keeps CodeMash regional, I believe, is that it’s in Sandusky, Ohio. There’s nothing wrong with a cool conference in the midwest, but Sandusky is just not very easy to get to from most places. The biggest nearby airport would be in Cleveland, and that’s not a huge airport and is probably an hour away from the conference.

For me, personally, the location is nice. The Kalahari resort is nicely done and the waterpark is fun, and the drive is only 2 hours for me. But, this location will definitely skew the crowd to a midwest one.

Concurrency

The best talk I attended, not counting open spaces, was Brian Goetz’s talk about concurrency. Brian works for Sun and knows his concurrency (he’s a co-author of “Java Concurrency in Practice”). Concurrency is becoming an increasingly hot topic, as it’s common now for machines to have two or more CPU cores. He used Java as a backdrop and had a great example of how threads are hard. What made the example perfect was its simplicity: an object representing a bank account that had only three methods.

Fairly quickly, you come to the conclusion that shared state is hard to handle well concurrently. This is part of the reason that Python’s creator, Guido van Rossum, has favored using multiple processes for concurrency. Some promising approaches to concurrency mentioned by Brian:

1. Software transactional memory. It sounds like a silver bullet, for people who like the threaded model, but it’s still very much a research project. The idea is that you can declare that an operation is atomic and if two atomic operations try to manipulate the same data, one will fail. This is an optimistic kind of concurrency, because you don’t actively lock the data. You just assume everything will work out okay and deal with it if it doesn’t.
2. Erlang’s Actor model has proven to be quite a successful way for doing many things concurrently. However, Erlang is a functional programming language that doesn’t have many of the language features that people today are used to (like, say, class-based OO).
3. Scala includes an Actor library modeled after Erlang’s. It combines functional and OO styles and provides access to the Java libraries. If you write programs in Scala’s Actor style, you can get excellent, reliable concurrency… you just have to be careful about Java code that you import

Brian also pointed out that you can improve your concurrency picture in Java by making your objects immutable. If operations that change the object return new copies of the object instead of mutating in place, your code is much more concurrency friendly. Unfortunately, you’ll still need to watch out for libraries that other people created.

Software Engineering

The conference-opening keynote was Neal Ford’s talk about Software Engineering and Polyglot Programming. He talked about software development as an engineering discipline and said that, compared to things like bridge building, we’re still in the very early days. In the early days of any engineering, you need to do a lot of testing to see if what you’re doing is actually working, and that’s the stage we’re at with software. In other words, write tests for your code if you want to consider yourself a software “engineer”.

I’ve been a test-driven development fan for a while, and I definitely agree with Neal on the importance of testing.

The other aspect of Neal’s talk, polyglot programming, was all about using different languages for different tasks. We already do (HTML, CSS, JavaScript, Python and SQL is one possible stack of languages). Languages that fit the task can save us a lot of effort. I am a fan of DSLs and declarative programming styles, so this all resonated with me as well.

Neal framed a few of his examples as “static typing vs. dynamic typing”, using Java examples to represent the “static” side of things. I really dislike when the “static vs. dynamic” debate has more to do with Java’s painful syntax than differences in typing. If you want to frame a discussion around “static vs. dynamic typing”, use a language like Scala or even C# that has type inference, operator overloading, etc. so that the statically typed examples aren’t bloated messes.

Other Talks

I attended a couple other formal talks, but I don’t really have much to say about them.

Something I did learn from the Hobo talk is the idea of using small (say, 30 seconds), soundless screencasts for doing demos. You embed the screencast directly in your Keynote presentation. It’s a way to do a demo where you know everything will work, you don’t need internet access, and people can see how things run. The only drawback is that the demo is fixed and you can’t change it based on realtime audience feedback. For some demos, this is a good technique that I’ll try to keep in mind.

Open Spaces

Running in parallel to the formal talks, there were open space sessions. Anyone can post a topic and claim a room and people can join a discussion about something. As Bruce Eckel pointed out at the beginning of the conference, open spaces are awesome because they give you something you can’t have from your own home: a real conversation with other interesting people. You can view a presentation remotely, but having a real talk about something works much better with physical proximity.

Next year, there may be more emphasis on the open spaces. This year, I thought there was too much competition in each time slot (my introduction to Dojo was up against 5 other formal talks plus who knows how many open spaces sessions). I don’t think it’s good to have so many parallel tracks in a conference with 350 attendees. I think the CodeMash organizers are considering some alternative time slot layouts for next year.

Non-Relational Databases

I hosted two open spaces sessions. The first was about non-relational databases. We discussed when you might consider using a non-relational DB and many of the available options. CouchDB has gotten a lot of press, but Persevere seems more “done”. We talked a bit about tuplespaces and Gigaspaces. The ZODB is a nice option for Python programmers, though I still long for a ZODB storage that doesn’t require packing. Amazon has SimpleDB. There’s also MonetDB, a column-oriented database. We did also talk a bit about OR mapping, though the focus of the discussion was alternative database technologies.

The mismatch between the kinds of problems we’re solving and the kinds of development tools we use (largely OO languages today) make other kinds of databases an interesting line of experimentation. Some are ready for prime time use (the ZODB has been used in production environments for a decade), but you have to figure out which problems you’re trying to solve.

The Next Big Language

The second open space session I hosted was about the next big language after Java. This is a topic I’ve been thinking about a fair bit lately, and I have a lot more to say on the subject. The discussion was fun and interesting.

Python Web Frameworks

Mark Ramm hosted a Python Web Frameworks open spaces discussion. We talked about things people wanted to see, and we were fortunate to have a couple of Ruby people in the session to bring their perspective in. This was also good timing for Mark, who led an international TurboGears 2 sprint this past weekend.

The Experts Zone

A new feature at CodeMash 2008 was the “Experts Zone”, where the speakers can be found to discuss things in more depth. I didn’t see a whole lot of activity in the room, and I think that’s partly because it was one more thing in competition with the talks that were lined up. The schedule for the Experts Zone was also tucked away in the back of the program.

That said, Dick Wall had no trouble attracting people to his Experts Zone session where he was showing off Android tools a bit.

Software Documentation Tools

During my time in the Experts Zone, I ended up in a conversation with Bruce Eckel and Mark Ramm about programming tool documentation. There are competing needs for nice formatting, code samples that can be run by the reader and code samples that actually work.

Bruce writes his books in Microsoft Word, because he likes to be able to work on the document with all formatting intact. He uses a text editor initially on his code samples, but then copies the code into Word. Word is the authoritative source for the code in his books.

The magic here is that he’s got a Python script that uses COM to tell Word to export the document as text. Then, it rips through the text file, extracts the code samples, compiles them and tests them.

This seems like a good way to go when your primary output is the printed form. You get complete formatting capability, tables of contents and indices, and it’s easy to insert pictures and make everything look perfect. This might be even easier to implement with Apple’s Pages, since Pages documents are actually stored in XML (which I’m guessing is more readable than that of Word).

I wonder if the same can be applied to Keynote presentations?

Mark told us about another way to go that is more “open source compatible”. Mike Bayer put together tools for SQLAlchemy that let him write his docs in Markdown and have his code automatically colorized and executed to generate the SQL, which is also placed into the final HTML document.

This setup seems ideal for producing high-quality docs like the kind that SQLAlchemy has. You could easily support multiple languages and provide ways to handle setup and teardown code for tests. Some people really go for doctest, but I’d like the ability to have more code that does not necessarily appear in the final output.

Getting Flushed

This year, I actually got to spend a little time in the waterpark at the Kalahari. That was a lot of fun! The place was much bigger than I expected, and there were slides that others reported as being new this year.

My favorite was a slide where you slide down on a 4 person raft. The raft goes down a surprisingly steep hill and into what can best be described as a toilet bowl. That thing was probably 50 feet in diameter. You go sliding around it once and then there’s a big spray of water that drenches you, slows the raft down and guides you into a tunnel down to the pool below. I rode on that one with Bruce Eckel, Mark Ramm and Dick Wall while James Ward graciously stood off to the side taking pictures.

Of course, James didn’t just walk back down the stairs after taking pictures. He went down the one person version of that toilet bowl… I found that one a bit more disturbing. You start off in a completely dark tunnel before coming out into the bowl which just has a big hole in the middle. Gravity seems to want to pull you head first into the hole (I steered myself to fall a bit more sideways through the hole), and then you plunge into the pool below.

I also rode on that 4 person one with David Stanek and the other guys from American Greetings. We probably rode that one four times.

The other slides were fun as well… just avoid the waterpark if you’re afraid of the dark. Most of the slides have you in almost total darkness for much of the ride.

My Talks

Oh yeah, I had two talks this year: an intro to Dojo and another about Dojo Offline. The talks seemed well-received (we’ll find out when the evaluations come in!), but the crowds were relatively small given the serious competition for interest that existed in every timeslot.

Conclusion

CodeMash is a fantastic conference for people wanting to get together with other geeks who care about the craft. The organizers have already declared that CodeMash 2009 is a go, so start planning your trip to Sandusky for January 2009!

Back when I was in the thick of my involvement with TurboGears, I remember seeing an email from someone complaining that, unlike with PHP apps, TurboGears apps cuoldn’t be hosted on $1 per month hosting accounts. While I was always interested in seeing TG apps on inexpensive shared hosts, I had no expectation that people could host TG apps for $1 per month. And, frankly, I don’t think I care about that.

Software development is full of tradeoffs. As Ian Bicking points out, PHP’s deployment model has some good aspects. As long as you stick to the constructs provided by PHP, it’s fast and the programming model is very simple and side-effect free. Now, however, PHP users want the productivity goodness that you get from higher-level OO frameworks like TG, Django and Rails. To create frameworks like those in PHP means optimizing the framework so that it loads the minimal amount of code in any given request. This is a constraint that TG, Django and Rails don’t have to consider at all, since the code is loaded once and kept resident (in any sane deployment).

The DreamHost blog recently lamented the pain of Rails deployment in their shared hosting environment. I’m frankly not too sympathetic. WebFaction seems to be able to manage shared deployments of modern frameworks quite nicely, and I assume they’re making a profit at it. WestHost provides quite inexpensive VPSes for folks who want to do it all themselves. I like DreamHost, and I’m a satisfied customer, but if others can figure out reasonably priced ways to host Rails and TG, DreamHost should be able to as well. (Full disclosure: those links are the same affiliate links that I have in my sidebar. I mention them here because they are relevant to the topic at hand, but it’s worth mentioning that following those links and buying services there does give me some money.)

Maybe DreamHost shouldn’t be looking at it like “how can I get Rails into an existing $10/month package” but rather what kind of good Rails hosting setup can i do for $15/month. Rails seems like a fine value-add… and if there are others undercutting them, then they can figure out how to compete on price.

All of this is not to say that we shouldn’t try it make deployment easier or frameworks better. It’s just not likely to be worth making the tradeoffs required to match PHP’s deployment in an effort to shave a couple bucks a month off of low-end hosting.

Bruce Eckel is recommending that the Java language should be declared “stable” and left as is, rather than grafting on features that don’t fit well (particularly in the face of maintaining backwards compatibility). See: Java: Evolutionary Dead End. I agree with Bruce. As long as a high-degree of backwards compatibility is required, adding language features to Java will be clumsy at best. ECMAScript has the same backwards compatibility with forward movement challenges, but the language is much more flexible at its base, so I think ES4 can be a more successful evolution than Java 5 was.

Something interesting in this discussion is a comment from Vincent O’Sullivan that echoes the Scala Will Do cartoon from a couple days back. The implication there is that people in software development are always just chasing after the next shiny thing to help them with their projects… and, since it’s a joke, the implication is that people shouldn’t do that.

Software is the stuff of thoughts. It’s very malleable and new ideas can be tested out fairly cheaply. With the internet, new ideas and code can spread far and wide quickly. This is all a good thing. For me, creating software today is tremendously better than it was 5, 10, 15, 20 years ago. I watch many of the shiny things go flying by, but I don’t actually use them. But I consider the shiny things to be very important. There are all kinds of ideas wrapped up in there that can be applied in different contexts and with different tools. Some of the implementations that come out of those ideas will make it into the mainstream.

Generally speaking, people don’t switch to all of the shiny things that fly by. Some people will give them an honest try and will either be rewarded for being a successful early adopter or will suffer some pain and head back to other solutions. If enough people like the idea and follow through, it will become mainstream. Rails was the shiny new thing in 2004. There were undoubtedly people who felt the pain of being early adopters, but many more were far more productive than people using other tools that were common back in those days. Regardless of how many people ultimately adopt Rails, the fact is that many of the ideas presented there have had a significant influence on work that has happened since.

Scala is a cool language. There are many other cool languages, but the reason Scala is getting talked about is because it is more than a toy and it’s something that people can apply today. Until Scala has the kind of IDE support that Java users are used to, I’d consider the people using it to be early adopters. Regardless of whether Scala enters the mainstream, it contains interesting ideas and it’s worth looking at. I actually rather doubt that Scala will become mainstream, because the gap between its syntax and what mainstream programmers find “familiar” is too large. Steve Yegge made a similar comment about Scheme.

My point in all of this is that the way we write software is always changing and is generally evolving for the better. Look back on how you’ve created software in the past, and I’m sure you’ll find that it’s better today. If not, you probably haven’t been looking at enough of the shiny things that have flown by.

Steve Yegge attracts some attention with Code’s Worst Enemy, in which he uses many, many words to say that “big, bloated software sucks to maintain”. He talks about a 500,000 line Java-based game that he wrote that is unmanageable. Though he’s railing against code bloat in general, he does rant a bit about Java in particular. I’ll come back to Java…

First off, I agree with the basic idea that having too much code to manage in a single conceptual unit (a program) makes life very difficult.

The first defense against this is good product management. Only add features if they’re good for the product as a whole. If you strip out a lot of fringe features, you can end up with a much leaner product that is both easier for you to maintain and easier for users to get into using. Scribbles brings this to mind. The app is focused on creating drawings and provides a UI that is completely streamlined to the task. I would guess that the code for Scribbles is not out of control, because it would be similarly focused.

The next thing to consider is that we all routinely write 1 million+ LOC programs. Consider this case:

#!/usr/bin/python

print "Hello, world"

Conceptually, programs don’t get much easier than that. But, running that program and seeing its output in a Terminal.app window on my Leopard-based Mac running Python 2.5 undoubtedly touches at least 100,000 lines of code. The trick is that I have faith that Python, Terminal and Leopard are all going to do their jobs (and those big guys have faith in the libraries they use), so that I can effectively ignore 99,999+ lines of code and just focus on the one.

There were a couple of commenters on Steve’s blog post that suggested that maybe rather than having a 500,000 line program what he really should have is a 10,000 line program and a bunch of libraries. In one sense, this is the compartmentalization that Steve complains about in his post. Libraries are one effective way of compartmentalizing code that has proven to be very successful over the years.

And language features are another way to segment code. That’s part of what the recent hubbub over domain specific languages is all about. Libraries can reduce how much you have to think about as you solve a problem, and DSLs can further reduce this.

Which brings me back to Java and Steve’s original point. We can focus on reducing the amount of code we manage by using libraries, and there is certainly plenty of that going on in Javaland. We can also add higher-level features to our languages. Doing so reduces the total lines of code required to solve a problem and the cognitive load required to maintain a program. Here’s an example:

import time

class Foo(object):
    @property
    def foo(self):
        return time.time()

Python’s attribute and property handling means that you don’t need to define random setters and getters all over the place. You just define them where you need them. Think about how many lines of code in Java are wasted on getters and setters that do nothing beyond the standard behavior, and how many times you have to type bar.getFoo() rather than just bar.foo. Yes, I know that Eclipse generates the getters and setters for you, but you still have to read through all of that muck when you’re trying to figure out what a program does.

In a nutshell:

• I agree that code bloat is a huge problem
• I’m sure that part of the blame for having 500,000 lines of code to manage single handedly rests with Steve… in choices he made about features to include, and in how he decided to decompose the problem.
• But I also agree that Java carries dramatically higher cognitive load than many other popular languages and one should really think about alternatives.

There’s now a multi-author Scala Blog that provides info about this very cool language. It sounds like there is a draft of a Scala book about ready, and the lift web framework hits 0.3. Scala and lift are tools to watch, because there is a lot of interesting background going into them. Anyone who has been eyeing Erlang’s concurrency features should check out Scala, because it gives you the same concurrency features but on top of the JVM with access to Java libraries when you need them.