Genius is 1% inspiration and 99% perspiration

Thomas Edison is credited with the saying that “Genius is 1% inspiration and 99% perspiration”.

Many of the most successful web 2.0 companies understand this intuitively and it is reflected in their product management. Although they all have a general vision for their product, it does not spring full formed from their minds. Rather, they build A:B test harnesses to explicitly test their hypotheses on live users. They don’t ask their users what they want, but rather they watch what they do. They try multiple versions of everything (title text, call to action copy, buttons versus links, number of screens in signup etc) and they let the data decide the direction of the product. They’re not driven by philosophy, but by the scientific method. Examples of companies that take this approach include many of the standout viral growth companies of the current generation, including RockYou, Slide, Plaxo, LinkedIn, Facebook, Tagged, Flixster and many more. (Disclaimer: Lightspeed is an investor in both Rockyou and Flixster)

For game design, the equivalent would be the trend towards metrics driven development. Raph Koster wrote up the Master Metrics presentation given at GDC by Dan Arey and Chris Swain from USC. They talk in part about Microsoft’s approach to metric driven design:

MS User research group… using heatmaps. When a project goes thru MS, 3 people from the user research group assess the gameplay experience. They are a real thought leader in this area.

1. usability testing – can user operate software
2. playability, does user have a good play experience
3. instrumentation, how exactly is the user playing, using tracking software

This is the first year that they are talking about this stuff publicly, the Wired article (Ed Note: Halo 3: How Microsoft Labs invented a new science of play), etc. Here’s a picture showing black dots on the Halo map. So dense on deaths that there is no info. So let’s tie it to color intensity. Then patterns emerge, you can see a pattern of where people tend to die.

In single player:

– tracking time on task, red zone indicates usability problem
– comparing if designer intent matches what players do… designer maybe wants intense “speed through gauntlet” feel, but heatmap shows players moving slowly…

In multiplayer:

– tracking deaths by weapons lets designers read exactly how players use items, more useful than written reports or lists of data. Designers collectively tend to be visual thinkers.
– Designer tuned placement of items and terrain to achieve most satisfying play experience.

User researchers independent from developers. Researchers help quantify into something measurable. Designers say “We want feeling of chaos” — researchers help pin that down.

Researcher are passionate about good game experience, but dispassionate about design specifics. Developers tend to fall in love with their designs.

Danc had a nice summary of metrics driven game design a couple of years ago that is worth re-reading.

As we see more games move to the web, allowing for much better real time data, true A:B testing against live users (not just beta testers), and shortening development cycle times, I would expect to see even more of this metrics driven approach to game design emerge.