Fishpool

Having recently witnessed the powerful effect establishing a robust vocabulary has on the process of design, and seeing today the announcement of the oft-delayed Nokia N9 finally hit TechMeme front page, I again thought about the common misconceptions of creating software products. It's been a while since I posted anything here, and this is as good a time as any to do a basics refresher.

A typical axis of argument sets software engineering somewhere between manufacturing and design. I, among many others, have for years argued that the relationship of software to physical manufacturing almost non-existent. This is because while the development process for a new physical product, like any involving new creation, starts with a design phase, the creation of a specification (typically in the hundreds of pages) is where the manufacturing really only begins. The job of the spec is to outline how to make the fully-designed product in volume. In comparison, by the time a software product is fully-designed and ready to start volume production, there is no work left - computers can copy the final bits forever without a spec. There's more to that argument, but that's the short version. Creating software is the design part of a product development process.

So, goes the line of thinking, if software is design, then it must be right to always begin a software project from zero. After all, all designs start from a blank sheet of paper, right? At least, all visual designs do... No good comes from drawing on top of something else.

If this truly was the case, what do you think they teach in art schools, architecture departments, and so on? Technique? For sure, but if that was all there was, we'd still be in the artesan phase of creation. History? Yes, but not only that. An important part of the history and theory of design is establishing lineage, schools of thought, and vocabularies which can serve as a reference for things to come. All truly new, truly great things build on prior art, and not just on the surface, but by having been deeply affected by the learning collected while creating all which came before them.

Not having actually studied art, I have only a vague idea of how complex these vocabularies are, and this is an area where a Google search isn't going to be helpful, as it only brings up the glossaries of a few dozen to at most a hundred basic terms of any design profession. This is not even the beginning for a real vocabulary, since those describe to a great detail the relationships of the concepts, ways of using them together, examples of prior use, and so on. However, even from this rather precarious position, I will hazard a statement which might offend some:

Software design, in terms of the vocabulary required for state of the art, is more complex than any other field of design by an order of magnitude or more. The practical implication of this is that no new software of value can be created from a "blank sheet of paper".

This will require some explanation. Let's tackle that magnitude thing first.

Any complete software system, such as that running within the smart phone in your pocket, measures in the tens, if not hundreds of millions of lines of code. LOC is not a great measurement of software complexity, but there you have it. In terms of other, more vocabulary related measurements, the same system will consist of hundreds of thousands of classes, function points, API calls, or other externally-referable items. Their relationships and dependencies to each other typically grow super-linearly, or faster than the total number of items.

By comparison, the most complex designs in any other field are dwarfed. Yes, a modern fighter jet may have design specs of hundreds of thousands of pages, and individual parts where the specs for the part only are as complex as any you've seen. Yes, a cruise ship, when accounting for all the mechanical, logistic and customer facing functions together may be of similar complexity. And yes, a skyscraper design blueprints are of immense complexity, where no one person really can understand all of it. However, a huge part of these specs, too, is software! Counting software out of those designs, a completely different picture emerges.

None of these designs would be possible without reusing prior work, components, designs, mechanisms and customs created for their predecessors. Such is the case for software, too. The components of software design are the immense collections of libraries and subsystems already tested in the field by other software products. Why, then, do we so often approach software product development as if we could start from scratch?

Why was it that the N9 reminded me of this? Well, if stories and personal experiences are to be trusted, it seems that during the process of creating it, Nokia appears to have "started over" at least two or three times. And that just during the creation of one product.. As a result, it's completely different, both from a user as well as a developer standpoint to the four devices which preceded it in the same product line, and two (three?) years late from it original schedule. Of course, they did not scratch everything every time, otherwise it would never have finished at all. But this, and Nokia's recent history, should serve as a powerful lesson to us all: ignoring what has already been created, building from a blank sheet instead, is a recipe for delay and financial disaster.

Software is design. Design needs robust vocabulary and the processes to use them well, if it is to create something successful.

Picked up a piece of analysis today from my newsfeed regarding Twitter audience. Nielsen has posted information about Twitter's month-to-month retention (40%) and compared that to Facebook's and MySpace's. Pete Cashmore over at Mashable promptly misread the basic information and came to an entirely wrong conclusion about the stats, titling his post about it as "60% quit Twitter in the first month". A simple misunderstanding of basic audience analysis like this is the crucial difference between explosively growing traffic and a failure. That's a fail for you, Pete.

What's wrong? Well, retention is a separate matter from conversion. 40% conversion from a trial registration to being a continuing active user to the second month would not be a bad conversion rate. It's not stratospherically great, I've seen better, but I wouldn't be terribly unhappy about such a figure. However, Nielsen didn't say anything at all about first-to-second month conversion. This is what they DID say: "Twitter’s audience retention rate, or the percentage of a given month’s users who come back the following month, is currently about 40 percent."

That's pretty plain English when you take the time to read it. Month to month, regardless of visitor lifetime, not first to second month. On this metric, 40% retention is not good at all, and will definitely be a limiting factor to Twitter's traffic and audience size over time, just the Nielsen article points out (and shows the math for). For any given retention rate, there just is a certain maximum audience reach beyond which any new traffic can't overcome the leaving base, since new traffic is not an inexhaustible supply.

And since today is a busy day, that concludes the free startup advice. Take the time to understand the difference between these metrics, you'll thank yourself for it later.

I've been talking recently about a few particularities in the business models based on end-user micropayments that have created lots of followup discussion and questions. So much, in fact, that I decided it's time to try to explain one crucial and somewhat counter-intuitive detail in writing for later reference.

First, a bit of background: this information is based on my work with Habbo over the last 5 years, and is half learned from experience, half based on theoretical models built from that experience. I'm sharing this with the world because while it's been an interesting ride to build an online social game with an end-user business model, breaking pretty much every conventional rule in the process ("games have to have objectives", "there is no profit in micropayments", and so on), it's still better for our business if people understand why it works. If this allows a competitor to fix a problem in their product and get off the ground, so be it - there's plenty of growth to go around here, and failures don't help anyone. As a disclaimer, the numbers I'm discussing here have no relation to Habbo, though the basic observations certainly apply.

Let's start with an obvious statement and follow it up with something less obvious: Everyone wants to maximize revenue per player. However, in a free-to-play environment, where the majority of players do not contribute direct revenue, the right tool for the job is not to try to extract the maximum amount of money from those who do pay - rather, to increase the number of players buying anything at all - even if it's just $1 over their entire lifetime. In other words, it's good to have a lot of very low individual value players.

To explain it in detail, lets look at two assumptions behind a flexible pricing business model: first, that the number of customers grows as the cost of goods drops, and second, that the maximum consumption is unrelated to the minimum. There is no average customer who would spend more than half of others, and less than half of the rest. If there were, the picture of that customer base would look something like the image here, and it's pretty strange looking, wouldn't you say? You've probably seen pictures resembling this one where they don't start from the dominating $0 value point - that's the normal distribution.

The first assumption really is very simple: more people are willing to buy a product at a lower price. This is true for most goods, with some notable exceptions in the luxury goods market, where the perception and desirability of a product goes up with its price. However, it is difficult to create a mass-market luxury item, and those do tend to be cheap (and small).

The second is perhaps slightly more involved especially if one is used to thinking of fixed-price models such as one-time purchase of a boxed product or monthly subscriptions, both of which are difficult to scale up on a revenue per customer basis, so scaling them down is highly undesirable as well. However, it's more clear, if not obvious, by looking at other consumer goods - whether tangible such as drink- and foodstuff or intangible like movies, music and other entertainment. Buying these once certainly does not exclude further sales of the same product to the same customer - rather, it's a strong indicator of sales potential!

The free-to-play, pay-for-stuff model follows both of these assumptions. Cheap purchase price attracts more customers out of the existing free users, and transactional item-based sales allows repeat purchases of theoretically unlimited amount. Those who are willing to buy more will do so, up to some practical maximum of consumable goods and discretionary spending.

In this environment, focusing on higher-paying customers makes sense only if the number of customers drops by less than half when the revenue per customer doubles. Again, with the exception of some luxury goods segments, this rarely happens. Think about it: how many chocolate bars of standard quality would you expect to sell for $1? How about for $2? More or less than half? How about for $10 for the exact same package? I'd wager chocolate bars sell at least 10x better at the price of $1 than at the price of $10 each, and the increase of customer base more than covers the lower per-unit revenue.

This is a simple exhibit of power-law market dynamics, and is easiest observed when looked at through a logarithmic chart. Readers of books like The Long Tail or Critical Mass should not be surprised. There's a twist through - because this starts from zero gains (at the free players), the exponential behaviour follows a different path in the beginning. This model also turns Pareto's Law on its head - due to the (in my experience) relatively high exponent, the highest total value is at the lowest end of the spending.

Now, of course there is a minimum profitable price for a bar of chocolate that does not become near-$0 even at very high volumes, unlike purely digital products, so increasing chocolate-sales revenue by dropping prices does not necessarily increase profits, and I'm completely ignoring the effects of packaging and marketing on the perceived value of items. For digital sales, where packaging is more flexible and material costs are effectively non-existent, we still have to consider not-unsubstantial fixed development costs, a certain amount of costs associated to servers and bandwidth, some transaction-related pricing friction, and so forth, but certainly the minimum value (and price) of one unit of digital sales can be driven much lower than a bar of chocolate.