SEO (Search Engine Optimization) for Web Development: Ecommerce Websites & Dynamic Database Driven Websites

One of the most valuable search rankings a site can have is in the Google organic listings.  Imagine being ranked for "SEO", or "Organic SEO" or "SEO Services", or a retailer being ranked for its leading terms, such as "dinnerware", "dining room tables" or "pet supplies" or "massage tables".  Most people trust organic search engine results over paid search results, or sponsored links.  Both sets of links work, however, organic represents more search value than paid in many cases, according to eyetracking research.

So how do retailers and corporate database driven websites get ranked in the organic search results?  Often the biggest challenge for database driven websites is getting search engines to index its content stored deep within its database structure or heirarchy.  Search engine optimization (SEO) can be a challenge for large corporate sites, especially retail sites, however, both can be accomplished within time.

In the beginning, a webmaster, marketing manager, or search marketing account manager needs to determine whether or not Google, Yahoo or MSN has even indexed his or her website and the majority of its pages.

For example, if one wanted to determine how many pages Google had indexed, it would perform the following search query:

"site:www.yourdomainname.com" - Google

For example, a popular home decor site, such as Pier1.com, is database driven as are many ecommerce websites.  As of early November, 2006, it currently had 6,180 pages indexed in Google.  This number may fluctuate, so see the current Google search query here.

Many database driven pages are not indexed by search engines, for various reasons, including but not limited to:

• too many characters in the URL (3 or less variables is best);
• unacceptable characters in the URLs
• duplicate or similar content within one site, or across multiple domains (think recipes, real estate, ecommerce sites)

For example, as of November 2006, one particular page on Pier1.com's website was not indexed.  In order for the site to show up for "dining room tables" or "dinnerware", it's crucial for the site and its relevant pages to get indexed in Google.  In working on the project for four months, it's been interesting to note that Google does not seem willing to pick up more Pier 1 web pages.  For example, we created this Bizresearch page in early November.  Google picked it up within a few days, indexed it, and ranked it for various related keyword phrases within the copy.  Ironically, it is only linked from top drop-down menu navigation, and not a text link from anywhere else on the site.  Google still found this newly created page on Bizresearch's site, and in fact ranked it in the top ten search results for "dynamic database driven websites".

Our challenge is to find out why Google does not like the pages mentioned below.  We'll be updating the site over the next two months to report back on our progress in getting Google to like the pages below.

The retailer below has two different types of pages, at a minimum.  It uses the FAST site search engine, which generates its web page URLs.  We did note, however, that the breadcrumbs generate a friendly version of its URLs.  In fact, Google is indexing some of them, or slight variances of these URLs. 

http://www.pier1.com/catalog/collections.aspx?fh_location=//pier1direct/en_US/categories<{110297}/categories<{110323}

The page above is a breadcrumb version of the page shown below:

http://www.pier1.com/catalog/collections.aspx?fh_location=%2f%2fpier1direct%2fen_US%2fcategories%3c%7b110297%7d%2fcategories%3c%7b110323%7d&fh_refpath=facet_59432939&fh_start_index=0&fh_view_size=8&fh_view=lister

Note the first URL is shorter, and has less variables.  In fact, it has only one variable past the question mark.  The second URL has five variables (count the number of & (ampersands) in the URL past the question mark (?).  Google is more likely to see both pages and index the shorter version, which works just the same as the longer URL.  Strangely though, despite both URLs being listed here, despite Google picking up this page within a week or so, Google chose not to index any of the Pier 1 links listed on this page.  So, I'm going to vary the links frequently over the next month to see what happens.

Google's Vanessa Fox has indicated that the %7b URLs could be excluded in the robots.txt file, which we initially suggested but then we noticed that some of the breadcrumb URLs are resolving to %7b...%7d so that will not work.  We're also recommending that any of the refpath URLs be excluded, which may be one and the same.  We're also talking to FAST, to see if they can help us better understand their URL structure. 

Search marketers and webmasters have to help Google by providing a roadmap to locate its hard-to-find content.  Footer and navigational text links help, as do content links such as the links above.  Marketers can also have their webmasters rewrite the URLs and reduce the number of variables to three or less.   However, it's often not easy for retailers to simply rewrite their website.  It has potential to adversely affect their website marketing and production strategy.  We challenge the site search engine to be aware of its affect on a company's ability to get indexed and get ranked.  To imagine that a company could not do well organically in a search engine due to its inability to get indexed, strangely enough, due to its site search engine. 

We're listing several new URLs below, breadcrumb version only, to see if any of them get picked up in the next couple of weeks. 

Pier 1 Dinnerware

Pier 1 Bar Accessories (Coasters, Martini Cocktail Shakers)

Pier 1 Bottles & Accessories (Canisters, Cookie Jars)

None of these pages above are indexed yet.  It is November 26th.  We'll review in a week or so to determine if Google picked up any of the pages above.  We'll also note whether or not Google resolved the URLs above to a slightly different version.  We're waiting to hear back from Vanessa Fox on why this happens, is it Google, the browser or something on the retailer's website that does this.

Another thing we've noticed with Google, is that it puts the search results pages in its supplemental results.  For example, type in the following search query at Google, or click to view the query here.

Google search:  allintitle: bar accessories site:www.pier1.com

Notice the search results showing up in Google - they are labeled as supplemental results. 

If Google does not index these pages, or like these pages, all due to the site search engine's resolution of the URLs, which is truly not easy to fix, what can you change on a database driven website in this case?  You can change the home page title, which we did.  It resulted in first page and second page rankings for two out of the six keyword phrases indicated in their home page title in Google. 

The power of the home page is often missed when optimizing database driven websites.  Site webmasters and marketing managers often do not realize its power.  Thankfully Pier 1 did.  But the ultimate goal of this project with them is to get Google to like its site search engine URLs and provide a roadmap to this extremely valuable content.  It can work very well in an overall search marketing program, by working with the paid search program for the same or similar keyword phrases.

Goals for Getting a Database Driven Website Listed in the Search Engines:

• Get indexed - home page, category, subcategory and product detail level pages
• Use Wordtracker.com's Keyword Universe - how are people searching for what you sell or do?
• Optimize your page titles (home page, category/subcategory/product detail level pages)
• Optimize your page headings in similar fashion, if possible (this is not always necessary)
• Optimize your leading introductory paragraph on each indexed page
• Optimize other content on the page, if possible
• Links, text links, incoming, outgoing, link, link, link (don't overdo it, no link farms please)
• Consider Yahoo Paid Inclusion Data Feeds, really - they work in some cases
• Paid Search - link here as well