Rich Skrenta shows an analysis of AOL Search terms broken down by category:

Entertainment 12.60%
Shopping 10.21%
Porn 7.19%
URL 6.78%
Research 6.77%
Misspellings 6.53%
Places 6.13%
Business 6.07%
Health 5.99%
News&Society 5.85%
Computing 5.38%
Orgs&Inst 4.46%
Home&Garden 3.82%
Autos 3.46%
Sports 3.30%
Travel 3.09%
Games 2.38%
Personal Fin 1.63%
Holidays 1.63%
Other 15.69%

Interestingly enough, both Shopping and Entertainment show significantly more search volume than porn. I guess we want our ipods and Britney gossip more often than dirty pictures.

The numbers add up to 118.96%, presumably because some search terms fall into more than one category. Representing each category on a percentile basis out of 118.96%, and organizing them into broader groups, we see that a little over a quarter of searches have some commercial component (in red below), a little under a quarter are entertainment (blue) or information (pink) related, with navigation and other making up the rest.

I’ve posted in the past about the difference between internet users who are time rich and time poor.

Time Rich people use the internet to kill some time. They are bored. They are willing to be diverted and entertained.

Web services based on discovery are often useful to the time rich. Last Sunday’s NY Times has a good article on one of the leading discovery services, Stumbleupon. Since its acquisition by Ebay, Stumble has continued to add functionality and grow:

In recent months, StumbleUpon has added the ability to stumble through specific sites, including Wikipedia, Flickr, YouTube, TheOnion.com, CNN.com and PBS.org.

It is when you are stumbling through YouTube or through Web videos in general that the StumbleUpon experience most resembles the TV remote — though one that tries to serve up programming to match your interests and whose suggestions get better with time.

That is one reason Mr. Camp is confident that StumbleUpon, or some other discovery service, will become a Web-wide hit over the next few years, as people increasingly shift their consumption of media to online from offline. “People aren’t going to stop channel surfing just because they don’t have a TV and they have laptops instead,” he said.

My hypothesis is that discovery works best when the cost of being wrong is very low. With Stumble, you get presented new websites (or videos, or news stories) and can almost instantly figure out if they are of interest to you. Channel surfing works similarly - you can often quickly identify if a show is of interest, especially if its a show that you’re somewhat familiar with. Browsing through Flickr’s interesting pictures works that way as well.

But some other forms of content (e.g. music, audiobooks, novels, movies, video games) can take you a little longer to tell if you like them or not. Songs often take more than one listen to develop an appreciation. Audiobooks and novels require a commitment of at least 15-30 minutes before you get drawn in. Completely new movies are the same way. If they are not interesting, then you’ve wasted a meaningful amount of time - the cost of being wrong is higher. This makes you less willing to keep on “discovering” more content at random - you want more data (e.g. reviews, plot summaries, information on the actors/bands etc) before you’re willing to try something new.

Do readers have any thoughts on this?

Tod Sacerdoti, CEO of the online video ad network Brightroll, notes that video impressions will soon pass number of searches.

1. U.S. video impressions will pass core search impressions in the next three months
2. U.S. video impressions will pass expanded search (meaning including Amazon, eBay, etc.) in the next twelve months.
3. Video advertising spend is being underestimated by analysts (eMarketer currently estimates video will grow from 10% to 25% of search revenue, and from 5% to 12% of total online ad spend, over the next five years)

As with all audience shifts, such as network television to cable television or television to the Internet, ad dollars will follow the audience. However, it does take time, as the network to cable transition took 5+ years and we are still in the midst of the spend movement from television to the Internet.

My bet? I estimate that video advertising will be 50% of search revenue within the next five years and will be larger than the entire search advertising business in the next ten years.

A couple of years ago, when I was running Netscape, the average revenue per search was about 2.5 cents when factoring in sponsored link click through rates and average CPCs. That translates to about a $25 CPM. Web video eCPMs may end up in a lower range than that, although premium video advertising inventory is certainly in that range today. This bodes well for Todd’s projections.

Liz Gaines at New Tee Vee weighs in with her opinion here

Yahoo’s new search product has just launched to positive reviews. I think it’s pretty great as well.

The Yahoo search blog outlines most of the new changes, but they can be summarized in one phrase; the new Yahoo search understands queries better. This gets instantiated in a two different ways:

1. Yahoo recognizes certain classes of queries and brings relevant vertical search results (e.g. video, photos, news, local, music etc) back into the body of the search results. Both Ask and to a lesser extent Google (with its Onebox) also do this. e.g. try searching for evolution of dance and get the video at the top, or Justin Timberlake and get a capsule of information on the singer at the top.

2. Yahoo aggressively helps users to refine their queries. Although average query length has been creeping upwards, search is still an iterative process for many users. People will type a search string, review the results, realize that it isn’t what they wanted, and improve the query string. Yahoo doesn’t just correct typos and suggest longer search terms that start with the same words, but it also recognizes concepts that are similar. (Ask does something similar in the left rail). E.g. try searching for King Henry VIII and get a suggestion for Catherine of Aragon. It also watches for hesitation as users type and auto-magically makes suggestions when they are needed.

As John McKinley (the ex CTO of AOL) pointed out a few weeks ago, AOL Search had this vision nailed over a year ago, but when new management came in they instituted a much sparser, more “Google-like” model instead. Henry Blodget thinks that this change tanked AOL’s search revenues in Q2, causing it to miss the quarter.

Yahoo has been rolling some of these search improvements out incrementally, and as a result, Compete found that they have both higher search fulfillment rates than Google. More of Yahoo’s search queries turn into clicks than Google’s, suggesting that users are finding what they want more often.

These improvements involve some deep technical problem solving. Finding a way to identify when to show results from a vertical search engine, and understanding related search concepts at scale are both difficult problems. You can cheat and do this with a big list, but that rapidly runs out of steam given the long tail of search queries. This isn’t just a lick of UI paint over the same old search engine.

The question will be whether this is enough. Yahoo has put a lot of time and effort into product innovation. I believe that there are three phases of competition in a consumer technology market; first distribution matters most, then product, and finally brand. Search may have already passed into the “brand” phase of competition. If people believe that Google has the best search, then they may not even try Yahoo’s search to be proved wrong. Yahoo will need to do more than ship great product (as it has done with this product), it needs to also find a way to drive trial from users who have Google as their default.

DavidZHawk asks, “What if Google Declared War on Comparison Shopping Engines and No One Noticed?” and points to an Inside Adwords blog post (my bolding):

The following types of websites are likely to merit low landing page quality scores and may be difficult to advertise affordably. In addition, it’s important for advertisers of these types of websites to adhere to our landing page quality guidelines regarding unique content.

* eBook sites that show frequent ads
* ‘Get rich quick’ sites
* Comparison shopping sites
* Travel aggregators
* Affiliates that don’t comply with our affiliate guidelines

Comparison shopping sites and travel aggregators are just two classes of the many flavors of vertical search engine, although they monetize better than most because of the high proportion of transactional search queries. As a result they have been able to afford to buy traffic through Seach Engine Marketing (SEM) where other vertical search engines have not been able to afford to due to lower monetization rates.

When you combine this move to send less traffic to vertical search engines with Google’s more aggressive inclusion of “One Box” search results from Froogle and their other owned vertical search efforts, you start to wonder if Google is looking to keep more of its traffic recirculating within its own properties. iGoogle and Gmail were the first signs that Google might aspire to keep control of more of the traffic that starts there.

Facebook is now making (very) limited public search listings available to people who are not logged in to Facebook on an opt out basis.

Additionally, they will soon be allowing search engines to crawl these Public Search listings. Presumably the business rationale it to increase traffic from search engines.

Myspace has allowed search engines to crawl profile pages for some time.

Oddly enough though, according to Comscore, in July 5% of Facebook’s visits came from Google but only 4% of MySpace’s visits came from Google. It may be that there won’t be much of a lift in traffic from this move after all.

Before I joined Lightspeed I was General Manager of Netscape, where I was responsible for the portal and the browser. Search drove about half of Netscape’s revenues and so I spent a fair amount of time trying to understand how to best monetize search traffic.

One thing that initially surprised was that the top two search terms on Netscape.com were “Google” and “Yahoo“.

In fact, around 20% of searches are “navigational” in nature - users looking for a particular website. Another 50% of searches are “informational” in nature (e.g. “capital of Taiwan”, “top social networks”) and the remaining 30% are “transactional” in nature (e.g. “cheap flights to Orlando”, “flat screen TV”. These stats come from an IBM research paper from 2002 that defines a taxonomy of web search, but the ratios were still roughly accurate as of 2006 when Gina Winkler, the outstanding woman who ran Netscape’s search team, left the company. [NB Netscape's search is now largely a re-skinned version of Google, a very different product to what it used to be]

It is relatively difficult to monetize navigational and informational searches. Try searches for “amazon” or “specific gravity of lead” and you won’t see any sponsored links. All the monetization comes from the transactional searches. Look at the huge number of sponsored links for searches on “ipod“, “rowing machines” or “disneyland hotels” in comparison.

So a new search company’s ability to monetizing search depends largely on what percentage of its search volume is transactional. For some of the new vertical search sites, this percentage can vary dramatically.

Take people search for example. A search on “jeremy liew” in Google yields no sponsored links (although before Ebay cut back its spending on Google there used to be an ad for “Great deals on jeremy liew at Ebay”!). In general, people search is informational. The proportion of transactional searches will likely be lower than general search. This is something that companies like Wink and Spock will need to take into account as they develop their business models.

Conversely, sites focused on shopping search will have a very high proportion of transactional queries. The first generation of comparison shopping engines such as Shopping.com, Shopzilla built valuable businesses on much lower traffic than the big general search engines because almost every query is monetizable. This bodes well for the next generation of shopping search engines including companies such as Shopwiki, The Find (a Lightspeed portfolio company), and Krillion.

Similar analyses can be conducted on other vertical search engines in areas such as local, travel, video and health - some of these will have a much higher proportion of transactional searches than others.

Semantic search startups propose to do a better job on informational search than the current search engines. If they see a greater proportion of informational searches because of this, then they may in fact monetize at a lower rate than today’s search engines.

Search is a tough business because of the need to change customer habits and pull search share away from today’s big branded search engines. If a new search engine does not monetize well because of its mix of queries, it has even more work to do.

(with apologies to William Shakespeare)

Recently Ben Elowitz, CEO of Wetpaint, wondered why the term “wiki” was not better understood. Wetpaint (a wiki company), prompted by wiki being listed as one of the top 10 most hated internet words, commissioned a survey to ask online users about their awareness of wikis, as compared to blogs, social networks, forums and search engines.

At the top level, the awareness levels were as follows:

Since these were online users (not the general population) this could be construed as discouraging; many don’t seem familiar with the basic technologies behind the modern web. However, I think that the data is misleading - while many people may not know about the technology, they do know specific examples of these technologies. As always, people focus on how their problems are being solved, not on what technologies are being used to solve those problems.

Take search as an example. Although only 76% of internet users were familiar with the term “search engine”, Google was recently announced to have the most powerful brand in the world. It beat household names like Coca Cola, Marlboro and Toyota. Its hard to imagine that there are ANY internet users who don’t use a search engine an a regular basis, whether they know the term of not.

Similarly, although only 28% of the surveyed audience were aware of the term “social networking site”, according to Comscore 64% of US internet users visited a social network in June 2007, with 39% visiting MySpace alone. Awareness does not appear to be a barrier to usage.

The same is also true of wikis. Although only 16% of internet users were aware of the term wiki, Comscore says that 26% of US internet users visited Wikipedia in June. If people are using Wikipedia, it doesn’t matter if it sits in the “encyclopedia” category or the “wiki” category in their minds

In the most successful consumer technologies, the technology becomes transparent to the user. Apple has sold over 100m iPods but I’m sure that many iPod users will not be familiar with terms like MP3, AAC or DRM. Users of cordless drills may not be familiar with the term Lithium Ion, even though that is hoe their drill became cordless.

For consumer facing internet startups the lesson is to view the world through your users’ eyes. Talk about the problems you’re solving, not about the technologies you use to solve those problems. That means more about “music” and less about “ACC”, more about “writing” and less about “blogs”, more about “collaboration” and less about “wikis”. After all, as Juliet tells Romeo:

That which we call a rose by any other name would smell as sweet

Last week Kosmix announced that Jon Miller has joined their board of directors. I’m very excited.

Jon is one of the most visionary thinkers about the internet that I know. I was his chief of staff while he was CEO of AOL and was privilidged enough to have a front row seat as he took AOL from a shrinking dial-up-ISP centric business into an online media business driving 40-50% advertising growth year on year. I came with him from USA Networks/IAC when he took the CEO job at AOL in 2002. He is a great big picture thinker who can see several years and several steps ahead as the industry evolves.

My partner Ravi Mhatre led our investment in Kosmix in the first institutional round (Jeff Bezos joined us as an investor in that round, and Accel joined us as an investor in a subsequent round). We introduced Kosmix to AOL’s Search team in mid 2006, and when the deal wound its way up the AOL chain, Jon eventually met the company at the Web 2.0 conference last year. He immediately understood the huge opportunity in what Kosmix was capable of doing; using a search interface to automatically aggregate relevant web content and presents it in a familiar portal like page format. When faced with “research” type searches and topics where the user may not be familiar enough with a topic to refine search queries without some help (such as in healthcare), this is a truly better experience.

After Jon left AOL earlier this year, we were able to convince him to continue talking to Kosmix as an independant agent, and eventually brought him on to the Board. Its great to be able to work with him again.

Om Malik is on to an important trend in his recent post  regarding the marginalization of Ebay, Amazon and other legacy ecommerce marketplaces with the advent of e-commerce 2.0.  Given the emergence of new and better merchandising technologies, more intuitive and comprehensive product search services, and the proliferation of contextual and performance-based advertising channels,  small and mid-sized merchants are able to establish rapidly growing web outlets more easily than ever before. 

In the first generation of ecommerce,  marketplaces with recognizable consumer brands (like Ebay and Amazon)  could offer small and mid-sized merchants access to large pools of customers.   However, there was a significant premium charged for this access - usually 10 or more percent of the transaction price. Bear in mind that the typical merchant will have total gross margins of no more than 20-30 percent. 

Like many net-based ecosystems we’re now witnessing the emergence of an open environment to replace first generation “closed”  marketplaces or communities.  Instead of listing on Ebay or Amazon and relying on their brand to attract customers and their standardized merchandising and search to drive purchases, a merchant can now easily build a product website that will drive organic traffic from vertical and horizontal search engines picking up their unique product content and also utilize a variety of performance based advertising channels including comparision shopping lead-gen sites (the top 10 sites delivered over 100 Million shopping leads to merchants in January 2007)  as well as  search engine keyword marketing to acquire new customers.  These channels are less expensive and drive significantly more customers and purchases at higher margins than legacy marketplaces.  

From a VC perspective, we believe a key requirement to making this work is the emergence of next-generation product search services that tame the Internet’s infinite shelf-space and provide consumers with truly comprehensive product search results through an interface that is highly intuitive and digestable.  Several start-ups are intensely working to solve this problem such as TheFind (LSVP portfolio company) Become, and ShopWiki.  Let us know what you think of their services.