Weidman Consulting, Inc. logo

HEADING



WC Journal logo

June 19, 2003    Issue VIII

Search Engine Marketing

Will Weidman
Weidman Consulting, Inc.
will@weidmanconsulting.com
(585) 787-7860

Also available: PDF version of this article.


In past WC Journal articles I have discussed some aspects of Search Engine Marketing (SEM), and even devoted an article to designing Web sites for better search engine performance. But I have wanted to write a good, general-purpose marketing article covering the different types of SEM and what to consider when choosing among them.

***

Search Engine Marketing Defined

Search Engine Marketing (SEM) refers to the strategy, planning, implementation, and ongoing maintenance activities related to driving targeted and profitable Web site traffic from search engine sites.

There was a time (in the mid-nineties) when there was Yahoo! and just a few other wannabe players in the search engine world. Today, there are still only a few key players in the marketplace, but the number of ways that a site can become visible in the search results pages has grown significantly. I'll discuss each of those, but first I want to examine the fundamental difference between a search directory and a search engine.

Search Engines and Directories

There is a common misconception that Yahoo! is a search engine. In truth, it is more properly classified as a Web directory. But we all use it just like a search engine, so the difference is more technical than practical. A directory is a hierarchically organized group of Web sites. It used to be that when you went to the Yahoo! home page, you were presented immediately with the familiar directory categories, "Business & Economy," "Computers & Internet," and so on - just like the Yellow Pages. These days, you have to scroll down past the animated Incredible Hulk ads to get to the directory listings, but they're still there. Clicking on any of the categories presents you with a list of sub-categories, allowing you to browse deeper and deeper into the directory. Yahoo! uses reviewers (actual human beings, imagine that) to review each Web site submitted and place it in the category that best fits it. You have to pay $299 per year to be listed in the directory (it used to be free). When you submit your site you can suggest which category you'd like to be placed in, but the reviewer gets the final say. Although the directory allows you to click down into the various categories to find a site, many of us just use the search box to find what we're looking for. That's where Yahoo! takes a turn and becomes more of a search engine. What many people don't realize is that many of the results returned from that search come from - of all places - Google.

Google is a search engine. It uses automated programs (crawlers) to visit Web sites, extract information, and copy that information into its database. There are no people reviewing what gets put into the database like there are with Yahoo! When you go to Google's home page and enter a search phrase, its complex algorithm goes to work finding the best pages to show you on its results page. So why does Yahoo! show you Google search results? It's all rooted in the complicated relationships that have been established between the major search providers. Because Yahoo! had built an exceptionally useful directory, but didn't have the world-class Web searching technology to go with it, they went looking for someone who did. Yahoo! has had several search technology partners in its history, but Google is the only one to win two consecutive contracts with them.

So the first thing to keep in mind is the distinction between a search engine and a directory.

Web Directory - a site that lists hierarchically structured groups of Web sites that are categorized by human reviewers.

Web Search Engine - a site that employs automated agents to find, visit, extract, and index data from Web sites in order to help its users find the Web pages most relevant to their search query.

The fact that all of the directories not only offer the ability to search their database of sites, but also integrate the search results of partner search engines into their results, blurs the distinction significantly. For now, let's call them all search engines. See Figure 1 for a listing of popular search engine and directory technology companies.

Search Engine URL Type
AltaVista www.altavista.com Search Engine
DMOZ (Open Directory) www.dmoz.org Directory
FAST www.alltheweb.com Search Engine
Google www.google.com Search Engine
Inktomi www.inktomi.com (generally accessed through its partner sites) Search Engine
LookSmart www.looksmart.com Directory
Teoma www.askjeeves.com Search Engine
Yahoo! www.yahoo.com Directory

Figure 1: Popular Search Engine and Directory Technology Companies

Organic Search Engine Marketing

Search Engine Optimization (SEO) is the most traditional method of driving traffic from search sites. The SEO process involves selecting a set of keywords to target, making content changes to the site that include those keywords (optimizing the pages), and submitting the pages to the search engines so that they'll get included in the database. In the early days of optimization, there were many tricks that could be played in order to get pages to rank well. Over time, those methods have become ineffective, as search engine algorithms have changed to detect them and penalize sites that use them.

SEO is also called search engine positioning (SEP) in some circles, and more recently the term organic search engine marketing has become popular. An organic campaign takes time to develop, as it requires keyword research, strategy development, and then modifications to the site. Also, because the search engine crawlers visit each site only periodically, it may take a couple of months before the changes appear in the results. The advantage of an organic campaign are that the cost is fixed (you may pay an optimization firm fees to implement and maintain the campaign, but you don't pay any more for 1,000 clicks-through than you do for 100).

Paid Inclusion

Paid inclusion solves one of the problems experienced in a pure organic campaign, which is the lengthy wait to get your pages found by the crawlers and indexed. Paid inclusion programs let you pay an annual fee for each page you want indexed by the search engine. For that fee you get automatic inclusion in their database (usually within a couple of days) and the guarantee that the crawler will re-visit the page every few days to look for changes and updates. Paid inclusion programs do not guarantee any sort of high rankings, though. Once your pages are in the index, they compete on equal footing against all of the other pages there.

Not all search engines and directories offer paid inclusion. Google, for instance, has no such program; submission to Google is always free. Other search engines offer paid inclusion only - there is no free option. Directories are a mix of free and fee-based. As noted above, Yahoo! charges $299 annually to be included in their directory. (But remember, your site can still come up in the Yahoo! search results that come from Google - tricky stuff!) The Open Directory Project (www.dmoz.org) is a directory that is not all that well known, but one that is extremely important to be included in because a listing there will help your rankings on other search engines such as Google. Its directory listings are maintained by a large corps of volunteer editors, and submission is also free.

The benefits of paid inclusion are that it guarantees that the pages you pay for make it into the search engine index, and it shortens the lag time between submission and seeing results. But you have to pay again each year to keep your pages in the index, and poorly optimized pages will still not rank well regardless of whether you pay to get them there or not. Figure 2 shows some common paid inclusion costs.

Search Engine AltaVista FAST Google Inktomi Teoma(Ask Jeeves)
Paid Inclusion? Yes Yes No Yes Yes
Cost per year - 1st page $78 $18 N/A $39 $30
Cost per year - each additional page $58 (2-10)
$38 (11-500)
$12 N/A $25 $18

Figure 2: Paid Inclusion Costs

Pay-Per-Click

Everybody wants to be Number One in the search engine listings. Don't believe any SEM company that tells you that they can guarantee to get you there - unless they plan to use a paid listing to do it. In recent years, the popularity of pay-per-click (PPC) listings on search engine results pages has grown significantly. Consider that Overture, one of the pioneers of this advertising technique, had revenues of $225M in the first quarter of this year. While most dot-coms failed, they are on the way to becoming a billion-dollar company.

You may not have heard of Overture because they primarily sell their listings to other search engines like Yahoo! and MSN. But their explosive growth and strong partner base have established them as a bright spot in the technology arena. Not to be outdone, Google has their own PPC program called AdWords, whose results are shown not only on Google but also on their partner sites - most significantly AOL.

The basics of PPC are straightforward: text advertisements are shown on the search results page in an area separate from the organic search results. Which advertisements get shown depends on the search phrase entered; the ads are tied to specific keywords and phrases. Advertisers pay a fee each time a search user clicks on their ad, and it doesn't matter how many times the ad is seen, just how many times it's clicked on. Whether or not the advertiser's ad gets shown on the page depends on how much they're willing to pay for each click, and this is managed in an auction format. Each advertiser bids against the others who want their ads to be shown for the same keywords and phrases. These bids start at just a few cents per click, and go as high as $10. See Figure 3 for an example of Google's PPC ads.

Figure 3, Google Results Dissected.

The concept of PPC elicits skepticism from many when they are first introduced to it. "What if my ad is so good that thousands of people click on it and I end up with a huge bill at the end of the month?" Both programs allow you to set monthly caps on the amount you spend to prevent this. "What if my competitors see my ad and click on it over an over?" Again, both programs have sophisticated fraud protection schemes to detect this sort of activity.

PPC is great for quick traffic. A campaign can be set up quickly (if you have already completed the keyword research) and one at Google will start running immediately. Overture has a 3 - 5 business day lead-time. Campaigns can be modified many times a day if desired. You can change your bids, experiment with different ad text, add keywords, and remove keywords - all in real time. If you sell baseball bobble head dolls and the Sammy Sosa model is suddenly hot you can have a special Sammy ad up and running in no time. The downside of PPC is that the cost is not fixed as it is with both pure organic and paid inclusion campaigns. In the PPC world, you must carefully track your return on investment (ROI) to make sure that you're not overspending each month.

Trusted Feed

If you run a very large Web site (500, 1000, or more pages) then trusted feed programs may be applicable to you. For a 1000-page site, the per-page fees charged by paid inclusion engines can add up a significant amount. Submitting 1000 pages, even using the bulk submission capabilities of some search engines, can be a hassle. Trusted feed programs allow a site's webmaster to provide a formatted data feed directly to the search engine. The data feed includes information about each page, including the URL, title, description, and body text. The "trusted" nature of this transaction lies in the assumption that the data being fed in is truly representative of the pages' content, and not intended to mislead searchers. Instead of charging per-URL (or per-page), trusted feed programs use a per-click fee model. How it differs from PPC is that the per-click fee is fixed; there is no bidding against other advertisers. It is similar to paid inclusion because your page still needs to rank well on its own merits in order to get seen in the results.

Which To Choose?

So now you now have an appreciation of the fact that search engine marketing is not a one-dimensional space. There are several alternatives for getting traffic to your site, and there is no one-size-fits-all solution. In general, many sites can benefit from a combination of organic, paid inclusion, and PPC strategies. You may spend more heavily on PPC early on in order to quickly drive traffic, and then downsize that campaign as your organic results improve to the point where they're competing against the PPC ads on the same pages. You might also defer paid inclusion strategies until you see if those crawlers find your pages naturally (because even the paid inclusion-only engines are crawling to discover new pages or have feeds from other engines). If you find that they're not being included, you can add paid inclusion to the campaign.

Measure It

It all boils down to ROI. The strategy that makes the most sense is the strategy with the highest ROI. Will you know what that strategy is at the beginning of your campaign? No way. That insight comes only after you've experimented with, implemented, measured, analyzed, and modified your SEM campaign. Measure It is my new mantra. An SEM campaign can be very accurately measured, yielding data from which the ROI can be calculated. For that matter, every Internet Marketing initiative can be measured and should follow the same process:

  1. Analyze the current and historical data and identify changes that can generate improved results
  2. Implement changes that you expect will improve your results, and get rid of those parts of the campaign that are underperforming.
  3. Measure the results of those changes, and repeat the process again starting at Step 1.

The Internet is the most measurable medium out there - take advantage of it.

***

I hope that you continue to find insight and value in the WC Journal, and welcome your comments, which can be addressed to me at will@weidmanconsulting.com.

I encourage you to subscribe to the WC Journal so that you can receive articles via e-mail as soon as they're published.

Full list of WC Journal articles


Join the WC Journal mailing list
Email:
We value your privacy.