The magical, mystical “1.0”

It seems that many open source projects don’t want to put themselves at the magical “1.0” number until the software is “done”. In my opinion, software is “done” when it reaches its end of life and is being put out to pasture. That’s one of the great things about software: it’s malleable! Software is much more malleable that many other “products”.

One could start a project by declaring “SuperMegaWidget will be called 1.0 once it has found a cheap, renewable, clean energy source”. That may sound outlandish, and it is, but quite a few projects set the bar very high for 1.0.

Maybe this is some new breed of 1.0 that is different from the kind I grew up with in the ’80s. Do you remember Windows 1.0? It was not pretty or useful. It couldn’t even stack windows (they were all tiled). But, it was a starting point and it did what it said on the box.

These days, it seems like everyone coming out with an MP3 player starts it off at 0.1 saying “well, it only plays MP3s… I’ll call it 1.0 once it plays Ogg, MPEG video, protected WM files and can transparently connect to P2P networks to download other songs you might like”. That’s crazy! If you’ve bothered to package up a release at all, clearly you think there’s something cool and useful there. Give it a proper starting point. Call it 1.0.

One might make the argument that, with programming tools and libraries, you can’t just make changes willynilly after 1.0 because people are depending on the stability of the API for their own products. The fact is that after a project becomes useful and has a community, you’re already somewhat locked in to the API you’ve provided. If you’re not prepared to at least document API changes, don’t make a release. Just leave it in Subversion, which makes it really obvious that you’re likely to change anything at a moment’s notice.

1.0 doesn’t mean “changes are no longer allowed” or even that “changes are hard”. It does mean that anything that might break a user’s code should

  • Preserve backward compatibility where possible (and not too heinous)
  • Be documented
  • Have a suitable deprecation period for things that are going away

These requirements are not too onerous. Python and Java have grown this way. They don’t grow quickly, but I think that’s partly in the nature of the tools. Different packages will be able to change at different rates. It’s especially easy to document and assist with backwards incompatible changes if you keep track of them as you implement them.

Languages like Python that offer keyword function/method parameters and duck typing make it even easier to maintain backwards compatibility while providing new features.

For TurboGears, I started it at 0.5 knowing that such a web development package could use some more outside review and validation before the APIs could be thought of “stable”. I also knew, and said that it would quickly reach 1.0. TurboGears parts were all solid and fairly mature, and it was really only the glue that needed some review.

Even I’m somewhat guilty of 1.0-itis. The forthcoming TurboGears 0.8 release could be called 1.0, assuming the requirements on that development status page are met. I knew that the sprint was coming very soon, though, so I decided that we may as well get the sprint results merged in to the final 1.0 release.

So, I say “embrace 1.0!” And then get back to work, because 1.0 is just the beginning.

Running a code sprint

The first time I heard the term “sprint” for software, it was in the context of Zope. A ZopeMag article also credits Tres Seaver at Zope with this and talks about how Zope Sprints are run. I decided to write up how I’m planning to handle this Saturday’s TurboGears “Dilly” Sprint in Ann Arbor in hopes that I’d get some useful commentary. This will also provide tips for the people who are coming.

This is the first sprint for a “new” project. However, true to the ideas behind TurboGears, some of the people who are coming have previously worked with TurboGears parts, mostly CherryPy, before. There’s the possibility that one of the project founders will make it… fingers crossed! Everyone has some level of Python experience.

This sprint is only one day (10AM-3PM). I have no illusions that a group of new people, working together for the first time with new tools, are going to produce thousands of lines of code in a short day. I’d like for people to come there, have fun, get some ideas together for the “Dilly” (0.9) release and write some code in that direction. After the day of the sprint, we can pick up the work that was started and finish it up. By making the sprint just one day, I’m sure it was much easier for people to work into their schedules.

As in the Zope sprints, I’m planning that we’ll do pair programming. For the circumstances that I described in the last paragraph, pair programming will be a big help: two heads are often better than one at generating ideas, and people will be able to help each other find the best way to work on something. Beyond pair programming, I’m planning to use a lightweight, XP-based process that I’ve worked with before. The day will be broken into roughly 1 hour iterations, which will give people a chance to check in with the rest of the group, collectively work through any problems and validate solutions that have come up.

I’m planning to create several team logins with commit access to the svn repository so that we can do frequent integration. As long as all of the computers have Python and Subversion installed at the start of the day, we should be able to get going pretty quickly.

Comments from anyone who has run a sprint or attended a sprint would be greatly appreciated…

Weekly TurboGears URL

Weekly TurboGears URL (Sep 24, 2005)

I’ve found the Dr. Dobb’s Python-URL! mailing to be a concise way to keep up with what’s going on in Python without actually following comp.lang.python directly. I’m going to do something similar here, for those who are just following the blog or the announcements list. This first issue will go to the main TurboGears list, but I’m not sure if that makes sense to do in the future.

QOTW: “My optimism and excitement WRT TG increases daily.”
— Jeremy Jones
Link

Patches came in this week from John Miller, Bob Ippolito,
Elvelind Grandin, Fabian Neumann, kickdaddy, Kent Johnson,
and gasolin. (There were possibly others that submitted
documentation patches during the crazy early days of
the week. Sorry if I left you out!)

A TurboGears sprint is coming up on October 8th in
Ann Arbor, Michigan:
Link

TextDrive pitches in and saves me from almost certain
hosting doom.
Link

That, of course, came after Bob Ippolito saved me from
actual hosting doom (by hosting the screencast at
MochiKit.com for a day):
Link

On Thursday, with the new home at TextDrive in place,
the public Subversion repository opened for business.
http://www.turbogears.org/svn/turbogears

There were a number of interesting questions and
discussions that came up that help to define what
TurboGears is or, at least, what it isn’t. It doesn’t
involve abstracting out the individual frameworks:
Link
or specifically supporting database APIs other than
SQLObject (and the underlying DBAPI):
Link

Good news from SQLObject: Matthew Cahn has working
Oracle support for SQLObject 0.6.1 and is working on
moving the code to 0.7.
Link

More than 4,300 views of the “20 Minute Wiki” screencast.
More than 4,700 eggs served from TurboGears.org.
140 members on the TurboGears Google Group.

——–

http://www.turbogears.org

TurboGears discussion:
http://groups.google.com/group/turbogears

TurboGears announcements:
http://groups.google.com/group/turbogears-announce

A brief history of TurboGears

An email from Jimmie Houchin has convinced me to take a few minutes out and explain where TurboGears came from and why it has emerged on the scene now. One note about how I’m writing this: there are many good tools for Python that solve a variety of needs. Since I’m not out to disparage other people’s work here, I’m not going to mention tools that I had been using or tools that I decided not to use.

In December, I decided to build Zesty News in Python after considering several alternatives (Rails, Seaside, Java-based tools). Python had nothing as slick as Rails for doing webapp development, but it had a whole bunch of great tools for other parts of my application.

I started working in earnest on Zesty News in January. I had to go through the same collection of reading about and deciding on pieces of the puzzle that everyone does when they start creating a webapp. (Side note: Zesty News is a “desktop webapp”… users install it on their machine and it runs a webserver to provide the bulk of the UI.)

I chose the tools that seemed, after a little digging, to be the best combination. The only one of those tools that is part of TurboGears is SQLObject. Development proceeded fairly smoothly until May, and then a funny thing happened on the way to release. I changed gears and decided to strongly emphasize plugins within Zesty News.

That notion changed the direction of things considerably. I wanted to provide a great API for building plugins, as this would benefit me (the full version of Zesty News will be built as plugins on top of the free version) and everyone who wants to customize the program. Toward the end of May, CherryPy became my web framework of choice.

In June, I dealt with a bunch of database nastiness that I never would’ve had to deal with if Zesty News was purely web-based and not a desktop app. Once I left that behind and got back to business, I realized that I would have to get together some decent documentation on the framework I was using to show people how to build plugins. I figured that if I’m doing that work anyway, I might as well package everything up conveniently and make the whole thing one nice, tidy open source package. That was around the end of June or early July.

Around that time, I started pestering Phillip Eby on the distutils-sig list about Python Eggs. I knew that something would have to be done to make TurboGears install easily, and Phillip had a rapidly maturing answer to that problem.

Then, Bob Ippolito started teasing us with promises of a cleaner JavaScript framework. I pestered him to give me a preview and had soon replaced my other JavaScript framework with MochiKit.

Bob was extolling the virtues of attribute-based templating languages when I took another look at Kid. My first look at Kid had been much earlier in the year, and I wasn’t particularly impressed. When I looked again in July, I knew I had found the ultimate: easy and Pythonic, yet fully valid XML and viewable in the browser. It didn’t take me long to rip out my old templates and replace them with shiny new Kid ones.

After the release of the first Zesty News alpha, I turned my attention to preparing TurboGears for release. I went through some effort getting API docs together using a hacked epydoc that includes colorized source files. I cleaned up the code that creates the Blazing Things website and turned it into docgen.py that appears in TurboGears.

And, of course, I wrote, and setup servers and got domains, and figured out how to do a screencast and all that good stuff. Which is how we got to where we are today…

I’ve actively used different tools for all of the parts of TurboGears and read extensively about many more. I’m happy to have found a great set of tools that work well together, and it’s gratifying to know that I’ve helped other people to use them as well.

(Not such a brief history, huh? Unfortunately, I didn’t have time to make it shorter (paraphrasing Blaise Pascal).)

TurboGears moves to TextDrive

TurboGears.org is now running at TextDrive!

By all accounts that I’ve ever seen, the people at TextDrive are among the most clued-in you’re going to find in the web hosting business. They know how to run servers, and they understand today’s “agile” development tools.

When I described my hosting problems, Michael Koziarski, who works for TextDrive, was among those who offered assistance. The assistance offered: free hosting for TurboGears at TextDrive. How’s that for supporting open source development?

I should also note that Koz wrote the Feed Your Reader plugin for Firefox, which I make use of in Zesty News. So, I’m doubly indebted to him!

As soon as I’ve figured out some permissions issues, TurboGears will have its subversion repository up at TextDrive.

Thanks, TextDrive!

Update: I forgot to thank Jason Hoffman from TextDrive for setting me up and getting me rolling. Thanks, Jason!

TurboGears on day 4 and some Jeremy Jones commentary

TurboGears certainly got going with a bang this past Saturday. We’re almost at the 4 day mark now, and the “20 Minute Wiki” screencast has been viewed more than 2,800 times. (Consuming 225GB of bandwidth!). turbogears.org has served up more than 2,600 eggs. The discussion mailing list has 89 members.

Before Saturday, doing a Google search for TurboGears yielded a few dozen results. Now, Google says there are 23,400 results.

The mailing list is getting some good discussion going. Monday was “Installation Day” with Phillip Eby fielding questions and figuring out how installation on Linux and other systems that already have Python can be improved.

Discussions now are turning more toward building webapps. With CherryPy as a base, you can certainly do whatever you need to do to create webapps. But, that’s not why TurboGears exists. The questions that people are asking now are the features we’re going to see in TurboGears tomorrow. For any kind of common requirement, “you can integrate this other package, good luck!” is not the right answer.

Luckily, there are great solutions to many problems already built in Python. We can pick the one that looks like the most fun and productive way to get the job done, document it and bake it in. Ian Bicking has taken some time out to point out some features of Paste and how they can work with TurboGears.

If there’s something that’s been bothering you about writing webapps currently, drop by the Michigan Python Users Group mailing list.

In other news, I’m expecting to have a public SVN repository soon. I’ll keep you posted on that one. I’m planning to meet the documentation schedule listed on the site, so expect to see the Getting Started Guide later today.

One thing I’ve had no time to do is comment on a lot of the interesting things being said about TurboGears on the blogs that have been talking about it.

Since Jeremy Jones has had three TurboGears related postings and sent a nice email to boot, I wanted to comment a little on what he’s written.

In his first posting, TurboGears: a slap-it-together web framework for Python, Jeremy says that he was impressed by the video, but…

I’m still a little skeptical, however, of what appear to be “me, too” Ruby on Rails projects. I guess maybe it’s a knee-jerk reaction against the massive amount of publicity they’ve received and wondering if they really are as good as they claim to be or if it’s just hype from “the analysts” and the masses.

In my opinion, neither TurboGears nor Django are “me, too” Ruby on Rails wannabees. Django has a history of being used to manage content on a variety of sites going back in time before Rails was in vogue.

In TurboGears’ case, I had to pull together the tools necessary to build my product. When I decided that plugins are an important part of the overall product I’m creating, I knew that I would have to provide good documentation on using those tools. I could have just done documentation completely in terms of Zesty News but, to me, that would be a waste when these projects can all really benefit from some more docs. That, and I found that this collection of tools made for an exceptional overall way to work on a webapp.

At the end of the day, TurboGears, Django and Rails all exist to take drudgery away from building a webapp, so that you can focus on the fun parts that are actually useful for people. That’s what has people talking. They’re better for programmers and better for the customers, too.

In his second post, Jeremy highlights CherryPy:

This is an easy to use web framework which, even by itself (meaning without TurboGears), allows you to put together a web site in little time. This project has done well standing on its own and has gained momentum and appears to be gaining market share in the Python web frameworks arena.

CherryPy is a fabulous tool and can reasonably be said to sit at the heart of TurboGears. There was some recent discussion on the CP list about a post-2.1 wish list. While there are some small things that I think will be good to move from TurboGears to CherryPy, I didn’t have anything at the time to add to a CP wish list. It handles its layer of the stack really well.

In Jeremy’s third posting, It’s all in the marketing – TurboGears’ cool demo video, he says:

I think what TurboGears has done with its marketing is going to get people excited about it, gain it market share, and will help drive its credibility. Credibility? Yes, credibility. I’m not saying that it makes it any better, but when people see with their own eyes how someone can create a wiki in 20 minutes, it will make them believe that it is a capable web framework. It will make them believe that it’s easy to use. It will make them believe that they can make a website with it with less work than an alternative.

Months before I had assembled TurboGears and decided to package it and release it, I wrote about how Ruby on Rails wins the marketing war. This has been one of my most popular posts of the year. When you’re in a small business, you have to think about marketing. In a world overflowing with information, you have to have a story to tell people so that they know why they should care about what you’ve got. David Heinemeier Hansson told his story very well indeed, and a thriving community has grown around his dream.

For my part, I had found this amazing set of tools that worked well together, and I wanted to tell people about it. I wanted to yell out “hey! look how easy it can be, even in Python!”. Maybe that should be “especially in Python”, because Python has been my language of choice for a decade.

There is more to say about this, though: once you’ve gotten the story out there, you’ve got to follow through on it. A significant part of the TurboGears appeal is the documentation and the promise of more, and we’ll keep that going. TurboGears doesn’t have all of the answers to common webapp needs yet, but it will. I was comfortable with this picture before, because TurboGears already had a community of sorts even before day 1: all of the users who have been building cool things with CherryPy, SQLObject, MochiKit and Kid.

Now that there are 89 people with enough interest to join the mailing list, I’m that much more certain that TurboGears will become far cooler than the humble 0.5 release.

Putting a hosting company to the test

Pricing is an interesting thing. As a general rule, unless you’re in a completely commodity business, price is not related to the cost of something. Price is based on the value of something and what people will pay. What this means is that the cost structure of a product or service is not necessarily in line with the price.

What does that have to do with hosting? Well, hosting companies typically charge for packages based on how much data you transfer in a month. However, the way they pay for bandwidth is not based on how much is transferred per month, but rather how much capacity they have for instantaneous transfer.

So, let’s say I have a 2,000GB per month allotment. And then I put an 80MB file online that gets downloaded 1,800 times in a couple of days. From my perspective, I’ve only used 144GB of my quota and have plenty of room to spare. My hosting company, however, looks at the fact that I was, at times, using almost all of their 100Mbit/second connection to the Net.

So, what happens? They disconnected my server. Their original rationale for doing so was a reasonable one: my traffic had spiked so much over the previous 5 months of traffic statistics that they thought I had been hit by a worm. Of course, when I told them that my server was just really popular right now, they brought up the fact that my bandwidth usage was harming connectivity for other customers.

One option was mod_throttle to prevent the “20 Minute Wiki” screencast from using too much bandwidth. That’s not a good option, though, because I *want* people to have a good experience downloading things.

Another option was to switch to a larger hosting company that has plenty of capacity to deal with the good times when they occur. That’s the route I’m taking, though it meant taking the screencast down just when there were lots of people watching.

TurboGears.org was completely unavailable from about 12:25 to 1:10 today. From 1:10 to 3:45, the site was up but the screencast was down.

Bob Ippolito put up a mirror at MochiKit.com, which brought the screencast back online. Bob has served up about 225 views already. Scott Oertel also offered to mirror the file. And, impressively enough, I got email a short time ago that my new Go Daddy server is up. So, the bandwidth issue has been dealt with…

When you sign up for a hosting plan, keep the difference between price and cost structure in mind. A small hosting company doesn’t have the instantaneous bandwidth to deal with something that’s very popular, even if your plan calls for “uncapped” bandwidth and huge data transfer limits.

Update: I didn’t mean for this to sound so negative toward my hosting company. My philosophy in business is that every deal should be a win/win, and it’s not about squeezing the last dime out of the other party. I completely understand that one customer spending $99 a month on hosting is not a good tradeoff against all of your other customers.

I’ll be spending 60% more at Go Daddy, but I also have the confidence that my server is not going to overrun their network. I also get some other services that are simply not available to a small outfit like the one I’ve been with. I haven’t been mentioning them by name, because I have no ill will at all toward them and don’t want to dissuade people from using them, since their service will likely suit some people just fine.

I’ve also been getting more interesting email regarding this topic, so I may have a little more to write on the hosting picture later today.

A day after TurboGears’ release

TurboGears had a quiet launch on Wednesday, when I presented the “20 Minute Wiki” live at the first MichiPUG meeting. It was listed as just “CherryPy+SQLObject” on the agenda, since TurboGears hadn’t been announced in any form at that point.

Yesterday, I made the more public announcement here on Blue Sky On Mars and on python-announce. From previous experience, it seemed like things tended to be a bit quieter on the weekends. It turns out that Python people were watching, even on Saturday. And, it also turns out that Python people are still very interested in web toolkits!

In a little more than a day, the “20 Minute Wiki” screencast has been viewed more than 1,000 times. That’s rather startling for my first screencast. By the way, I should take this opportunity to correct a problem with my screencast: I pronounced “MochiKit” wrong. Bob Ippolito pointed out that “mochi” is a Japanese word that is pronounced MOH-chee.

Bob, in addition to creating the fabulous MochiKit, also gets credit as the first person to submit code to TurboGears: a nifty snippet for doing Bonjour (formerly Rendezvous aka zeroconf) advertisements of the project you’re working on. Phillip Eby gets credit as the first person other than me to respond to a question on the TurboGears mailing list. I’m glad he did, because he was able to go into some detail about installing Python Eggs when you already have some of the packages installed.

On the TurboGears mailing list, it sounds like Windows and Mac users have been having good success with the Easy Install, but that Linux users have been having a harder time. ASAP, I’m going to write another install doc to deal with some of the complexities that can come up. Eggs are a great tool for Python and the solve a number of problems, so they’re well worth the little bit of extra trouble involved right now until they become the norm.

Out on the web, there were several blog posts talking about TurboGears. Bob said:

There’s a lot of good first impressions to be had with the technology

And he enumerated some of the reasons why I like the projects that I chose for TurboGears. As a Twisted fan (and contributor), he’d like to see it running behind Twisted. That would be difficult with SQLObject, which is synchronous, because Twisted is asynchronous.

Phillip said:

Okay, so TurboGears is cool. Not because it’s another Python web framework, but because it’s a… megaframework? Okay, so it’s not a framework, but what the heck is a megaframework?

Not that I have a better name for it, mind you. It is definitely a new thing in the Python world – a compelling project built almost entirely from separately-packaged Python components, powered by setuptools and EasyInstall.

EasyInstall is a key piece of technology to make something like TurboGears fly. If Phillip hadn’t created it (and done such a fine job of it!), I would have had to create some far less pleasant and less powerful mechanism to make TurboGears installable for people who don’t have an afternoon to waste tracking down dependencies.

It’s late for me now, so I’m going to have to continue checking out and commenting on the responses another time. I appreciate the early interest and I’m really excited about the parts that are yet to come!

Announcing TurboGears web megaframework (Python)

TurboGears brings together four major pieces to create an easy to install, easy to use web megaframework. It covers everything from front end (MochiKit JavaScript for the browser, Kid for templates in Python) to the controllers (CherryPy) to
the back end (SQLObject).

The TurboGears project is focused on providing documentation and integration with these tools without losing touch with the communities that already exist around those tools.

TurboGears is easy to use for a wide range of web applications.

To get a quick idea of what TurboGears is like, take a look at the 20 Minute Wiki tutorial and screencast. The total screencast with brief intro and conclusion did take a little over 23 minutes, but it’s close to 20 minutes 🙂

A big thanks to Remi Delon, Ian Bicking, Bob Ippolito, Ryan Tomayko, Phillip Eby and the many other contributors who have provided all of these great tools.