"Dreaming in Code" | Programmers strive in vain to get ahead of the curve

"Dreaming in Code: Two Dozen Programmers, Three Years, 4,732 Bugs, and One Quest for Transcendent Software"
by Scott Rosenberg
Crown, 400 pp., $25.95

Scott Rosenberg could be forgiven for humming the theme to "Gilligan's Island." Setting out to document a two-year course of development on a software project named Chandler, the author and Salon.com co-founder found himself adrift with an intrepid crew, a host of dogs and an ample food supply. After three years, no land was in sight.

Chandler is an attempt to redefine how information is stored. Rather than put appointments, people, e-mail and notes in "silos," or highly compartmentalized structures that have little to do with one another, the Chandler developers proposed breaking down categories. They wanted the equivalent of Freud's polymorphous perversity for personal information: a calendar entry for an appointment could be turned into an e-mail message by checking a box, without requiring the use of a different software program, or re-entering information.

Rosenberg became their chronicler in "Dreaming in Code," a book that covers three years of meetings at the nonprofit Open Source Applications Foundation (OSAF) in San Francisco, marking missed deadlines, the introduction of more and more pets at the office, and a remarkably small number of raised voices. OSAF developed Chandler as open source, in which a program and its underlying code can be freely used, modified and distributed with few constraints.

"In the annals of software history," Rosenberg writes, "Chandler's disappointing pace is not the exception but the norm. In this field, the record suggests that each driver finds a different way to run off the road, but sooner or later nearly all of them end up in a ditch."

Rosenberg makes the case through the book that despite decades of improvement in how programmers work, and actual increases in productivity, the demands on software have increased at a pace that far outstrips the best programmers' abilities to keep up. For instance, he cites a talk given in 2004 by a top Air Force director, noting that since the 1960s, the software code that runs a jet fighter has grown from 50,000 lines to 5 million lines. Programmers, meanwhile, have only doubled their productivity.

Drawing on several classic information-technology texts in writing this approachable and cautionary tale about progress, Rosenberg leans heavily on "The Mythical Man-Month," a 1975 book by IBM manager Frederick P. Brooks. The core maxim: "Adding manpower to a late software project makes it later." The time it takes to bring new programmers up to speed negates any short-term advantage in additional work hours.

Rosenberg glides smoothly between the close-up view of Chandler and a comprehensive look at software development and its lack of more than incremental progress in providing predictable results. He never finds blame in the OSAF team, although it's clear that the lack of time pressure — the founder, Lotus 1-2-3 inventor Mitch Kapor, largely funded the nonprofit endeavor — and a lack of more than a sketchy vision at the outset hampered the outcome.

Rosenberg ultimately had to cut bait and write his book, otherwise he'd still be sitting there today in a kind of Sartrean author's hell. But his choice illustrates one of the great paradoxes in creating software: the halting problem.

Legendary computer theorist Alan Turing proved that there is no way to predict that when presented with a certain set of data any given computer program would finish within our lifetimes — or ever.

"It seemed that ever since I had started visiting OSAF's programmers, their target held at a constant distance of roughly two years; as real time passed, the goal receded like a fugitive horizon," Rosenberg wrote.

They started work on Chandler in a limited form in October 2002, and Rosenberg started sitting in on meetings in July 2003. At that point, they expected to offer a reasonably complete piece of software by mid-2005, possibly a little later. Rosenberg moved on to write his book in December 2005 — the time remaining? Based on their previous incremental releases — about two years.