I’ve been seeing great commissions since switching my eBay affiliate links to ePN (eBay Partner Network), but unfortunately eBay’s reporting isn’t much better than Commission Junction’s. If you want to see what item sold on eBay for any particular transaction, you still have to download the report, open it, scroll over to the column with the item numbers, copy the item number, go to ebay.com, paste the item number in the search field, and hit “Go.”

What a PITA! I’m pretty good with php, so I decided to roll my own script. I started out with just the basic info that eBay provides in the downloadable report, and wrote a script that outputs the report contents with the item id as a clickable link so that I could go directly to that item on eBay.

Then I added code to use eBay’s developer API to pull in the item name, thumbnail image, number of bids, buyer name, seller name, and category name.

Now I have a really cool script that shows me exactly what items sold in eBay, their prices, pictures, and extra info.

I’m such a nice person, I decided to give this script away to any eBay affiliate who wants to use it. I’m not forcing you to upload your transaction report to my server; oh no…. I’m offering the script as a free download that you can download, put on your own server, and run from within your own hosting account.

I’m doing this because I’m such a nice person. And because I think it just might get me a few nice backlinks.

You can get the script here:

TWW’s eBay Report Tracker

Check out a screenshot: (click to enlarge)

But it’s also surprisingly satisfying to spend a few hours optimizing a small site for a new client, and then to see, almost immediately, improvements in that site’s performance in the search engines.

I recently spent less than a half day working a small site (about 5 pages) for a client. I didn’t build the site; it already existed when the client came to me, looking for help to get better search engine placement.

I did minor, basic, SEO 101 type things: I wrote custom title tags and description meta tags for each page. I added appropriate alt text to all the images. I cleaned up the html a little, to use proper h1 and h2 headings. None of this was rocket science, nor even high-level SEO.

Almost right away, the site started performing better in the search engines. From last September through March 10, the site had been found for a grand total of 7 different search phrases, most of which were some variation of the company’s name. I worked my “SEO magic” on March. 10. From March 11 to today, March 28, the site has been found for 46 different search phrases, many of which are excellent keyphrases for this company, and traffic has increased tenfold.

This site has only 6 pages, virtually no backlinks, and no marketing or advertising budget to speak of. The company’s service is a very small, narrow, geographically limited niche which will never draw hundreds or thousands of visitors per month. But within days of the small changes I made to the site, the site was drawing targeted organic search traffic at levels previously undreamed of.

I’m currently participating in a discussion on the SitePoint forums in which the topic of discussion is a web site developer who offers “SEO friendly CMS” (content management system) — but this developer builds his clients’ sites in frames. Not only does he use frames, but the framed content is actually hosted on his domain, not his clients.

There are many, many reasons this is a terrible idea. I won’t go into those here.

What I want to talk about is the statement by one forum member who wrote, “When the pages are indexed, yes they’ll show his domain but unless the client has specifically asked for that not to happen it’s not unethical.”

That’s completely wrongheaded thinking, in my opinion. Clients should not have to know enough about the website building process, the architecture of html pages and framesets, and the inner workings of search engines to request that the developer avoid certain practices that would be bad for the client’s site.

The client hires the web developer because the web developer allegedly has expertise in this area. Any monkey can peck at keys on a keyboard; it’s not the web developer’s time-on-keyboard the client is in need of. It’s the web developer’s expertise.

When I hire professionals or specialists to perform a service for me, I rely on their expertise. I don’t think I should have to study up so that I can tell my mechanic how much torque to use when tightening the tires on my car, or tell my plumber what not to do in order to avoid damaging my plumbing system, or tell my doctor which medicines are contraindicated, or provide my lawyer with the exact wording necessary for a business contract. I hire those people because they supposedly know what they’re doing. They’re supposed to know how to do the things they’re hired to do, and they’re supposed to know what techniques and processes to avoid due to potential damage.

In a similar vein, I believe that anyone who holds him or herself out as a professional website developer — and particularly one who claims “SEO friendly” anything — has a responsibility to perform the promised service using industry-accepted “best practices” without regard to whether the client has sufficient knowledge to request those best practices specifically.

I believe, in fact, that it is unethical for a website developer to engage in the types of practices discussed in the SitePoint forum, and in many other practices as well.

But unfortunately, from the client’s point of view, I don’t have a solution to offer. The client, who is usually not an expert in these things, doesn’t even know the right questions to ask, much less have any way of evaluating the answers. It’s all too easy for the unethical developers to give impressive-sounding, baffle-’em-with-bullshit, high-falutin’ answers. It’s easy to talk the talk; but how does the client assess whether the developer walks the walk?

A lot of the things the client should be assessing aren’t obvious from looking at the developer’s portfolio, unless the client himself becomes an expert on web technology. It’s not rocket science, but in my experience, most clients just don’t have the time or the inclination to do that. They’re too busy running their businesses.

But the web is now about 15 years old, and I wonder why so many newspapers still don’t manage to get some of the basic things right.

When?

This morning I was reading an article about a replica of the Nina coming to town. The Nina is coming to town on Monday. The article is listed in this morning’s edition (Wednesday, March 12) but under the headline there’s a line that says “Last updated March 11.” This suggests the article was first written and published sometime prior to March 11.

So when the article says the Nina is coming to town Monday, does it mean Monday two days ago, or Monday of next week, or some other Monday? If I go down to Fisherman’s Village today, will the Nina be there or not?

Perhaps “on Monday” was a sufficient identifier when newspapers were printed and distributed once a day. But “on Monday” is completely insufficient when articles live forever on the Internet, and when articles might be updated after their original publication date.

Please, sir, may I have a link?

Then I read another online newspaper. I was interested in the reader comments on the “most commented story.” The comments were interesting, but I really would have liked to read the actual story. Unfortunately, there was no link to the original story. Not even a publication date or a headline, which would at least have allowed me to search the archive to find the story.

Really, how hard is it to link to the original story from the comments? I figured out to do basic things like that back in 1996, within days of writing my first HTML tag.

Next up: Location, Location, Location

Then I was looking for some information on a certain event, and I found a newspaper’s web site which contained a story on the topic I was looking for. What I really needed to know was where this event was taking place. The newspaper article helpfully told me it was in the Marshall Civic Center, but there was no clue, anywhere on the page, as to what state the newspaper is in. Heck, I’m not even positive it’s in the U.S. I know the newspaper is the Marshall News Messenger. I can guess that the city is Marshall. The Marshall News Messenger tells me that Marshall had two fatal wrecks, and that Marshall is planning its inaugural “Third Saturday” event. But where is Marshall? Michigan? California? Florida? Maine? I have to go digging through the site to find out.

It’s common sense, not rocket science

The way some newspapers handle their online content, they might just as well make a giant pdf of each issue and upload that. What they do is no more usable than a giant pdf would be.

Web site usability is a growing and important topic of discussion among web developers. Usability can involve some finely nuanced tweaks, and also some very advanced technical things, and I can forgive any web site that doesn’t get some of the more advanced stuff right. But come on! Clearly identifying dates, and linking from article comments to the original article, are neither technically advanced nor finely nuanced. A reasonably intelligent sixth-grader should be able to figure out to do things like that.

Read my explanation here.

I generally recommend that they use WordPress and host it on their own site — usually in a subdirectory on their site’s domain, e.g., example.com/blog, or perhaps on a subdomain, e.g., blog.example.com.

There are several reasons I lean toward hosting your own blog on your own site:

1. Reliability and Safety: I dislike, in general, the idea of having your material dependent on a third-party service where the security and reliability of your blog is at the mercy of that third party. A few years ago one of the major hosted blogging services closed up shop — unexpectedly and unnanounced — and stole away in the night. All of the bloggers who had been using that service found their blogs gone with no backups available and no way to re-create their work elsewhere. With your own copy of WordPress running on your own site, your blog is totally portable — you can host it anywhere you wish, and you can easily set it up elsewhere if your hosting disapppear That’s provided you have a backup, of course — when my clients host with me, there’s always a backup available, because I run a cron job that backs up all the databases nightly, and my hosting provider runs daily backups, as well. So none of my clients will ever find their site, or their blog, gone with no backup. All of the remotely hosted blogging platforms have a terms of service (TOS) that disclaims any responsibility for your material.
2. Link Juice: Assuming you do a good job writing your blog, and it attracts inbound links from other sites or blogs, links to your blog at example.com/blog are likely to help the site as whole rank well in the search engines, whereas a link to yourblog.blogspot.com isn’t going to have the same effect.
3. Branding: When your blog is hosted on your own domain, people who visit and read your blog can’t help but be aware of the rest of your site. But when your blog is at yourblog.blogspot.com, visitors might have no idea that you even have a site of your own, even if you add links to your site in the blog sidebar.
4. Flexibility: WordPress has a lot of functionality “out of the box,” and much more available through a variety of free plug-ins. With the remotely hosted services, you can only do what they allow.

With that said, I’ve recently been exploring Blogger, the blogging platform now owned by Google, and I have to say I’m pretty impressed. There’s more flexibility and functionality available than I was aware of, and the set-up is even faster and easier than WordPress, which is famous for its 5-minute installation. The WYSIWYG editor in Blogger for writing posts seems to be a little easier to use than WordPress’s. And Google loves it own blog service — A blog that I set up there had its first post found and crawled by GoogleBot the very same day I posted it.

Nevertheless, I shall continue recommending to clients that they host their own blog on their site. I think the advantages of self-hosting far outweigh the advantages of hosting on Blogger or any other blogging service.

That was basically the response of Network Solutions when it was caught with its hand in the cookie jar, registering domains for themselves that people had looked up in their whois registry. NetSol basically defended their contemptible practice by saying, “In order to prevent domain registration abuse, we’re committing domain registration abuse.”WARNING: If you do a WhoIs search on a .com or .net domain at the Network Solutions website, NetSol will immediately register that domain. They will then kindly allow you to register it at NetSol — but they’ll also allow anyone else to register it, too.

Their claim that they’re “protecting” the domain “on your behalf” just doesn’t hold up. What they’re doing is ensuring that you cannot register the domain at some other registrar, such as GoDaddy, NameCheap, DotRegistrar, or any other registrar that actually charges a reasonable fee for domain registration. The only place you can register the domain is with Network Solutions, at their highly inflated registration fee of $35 (compare that to GoDaddy’s fee of about $10).

This practice costs Network Solutions nothing — by dropping the domain within the 5-day grace period, they don’t even have to pay the registration fee. Yet — yet — when they drop it, that domain goes on a list of dropped domains, and the domain tasters will snap it up. So, if you simply decide to wait it out until NetSol drops it, then register it at the registrar of your choice, you’ll likely miss out.

ICANN should stop with its hands-off posture toward registrars abusing their position of trust, and take action to prevent this sort of thing. NetSol should start behaving ethically, and should start charging fair prices for domain registration. Everyone should avoid ever using Network Solutions, for anything, ever again.

What’s next, Network Solutions? “We had to destroy the internet in order to save it, sir”?

PayPal Impersonators

Anyway, a client sent me a copy of an e-mail they had received, allegedly from PayPal, which contained “confirmation” of a purchase by the client using their PayPal account. The e-mail included a prominent link to “Dispute Transaction,” and the surrounding text instructed the recipient, “If you haven’t authorized this charge, click the link below to cancel the payment and get a full refund.”

Now, I understand that PayPal is good about chargebacks for fraudulant transactions, but I’ve never heard of them so openly inviting people to dispute a transaction. So that should have been a clue. Fortunately, when the client clicked the link to dispute the transaction, their antivirus program popped up with a warning message about it being a scam. The client then promptly contacted me to ask what they should do.

Undoubtedly, the link would have taken the client to a site that looked exactly like the PayPal site. There would have been instructions to log in to dispute the transaction. The client would have entered his PayPal account name and password, with the intention of disputing a fraudulent charge. Bingo! The scammer would have just got hold of the client’s PayPal login information — and there’s no telling what havoc would have been wreaked. Disaster averted — thank heaven the client had a security program installed and running on his computer.

Microsoft’s False Positive Phishing Warnings

And then there’s Microsoft. Internet Explorer 7, to be precise. With it’s much-vaunted “anti-phishing filter.” Bah, I say!

I recently started using an RSS feed from eBay to display live listings from eBay on an informational site. When I was testing the site in IE7, IE was giving me security warnings that this was a “suspicious site” and might be a “phishing site.” I know darn well it’s not — the site is clearly not eBay, it doesn’t pretend to be eBay, it’s clearly a separate site that doesn’t look anything like eBay — it merely displayed listing from eBay, with affiliate links to those listings on eBay. (For the record, it was my Dog Breeds Site, but I’m using the eBay feed on several sites.)

IE7 offered me the option to submit a report to Microsoft, stating that I was the site’s owner and could verify that it wasn’t a phishing site. I did this, and the next day I got an e-mail from Microsoft that they had inspected the submitted URL, verified that it was not “phishing,” and removed the warning.

Then I discovered they had only removed the warning from one page of the site. In order to remove the warning from every page that used these listings, I would have to report every page individually.

This was first of all, much too time consuming, and second, far too annoying. I would have to submit that report for every single page on every single site where I wanted to use the eBay feed. So I set about looking for what the code might contain that caused IE to pop up the phishing warning.

Here’s what I learned:

• Text links to the eBay listings didn’t trigger the warning.
• The images, pulled in directly from eBay and displayed on my site, didn’t trigger the warning.
• But the images, when linked to the eBay listings, did trigger the warning.

Hmmm ….. I didn’t want to remove the links from the images. People are naturally inclined to click on the images. After some trial and error, I discovered that if I sent those links through a redirection script, it stopped the phishing warnings cold.

Yay me!

The result is the Orangutan Island Forum. Almost immediately, the site jumped to 3rd place in Google for a search on Orangutan Island — just behind the official Animal Planet pages. I was pleasantly surprised when people started registering on the forum and talking about the show, and I’m having fun participating in the discussions and learning the ins and outs of the phpBB software.

But I wasn’t sure about my choice of forum software. I didn’t want to pay for the well-loved and very robust vBulletin — after all, this whole thing was just a whim, and my budget was non-existent. I knew that phpBB is a well-known and popular open source forum package, but development is in flux just now: They’re working on getting phpBB 3 into final release, and I dithered about whether I should install the “stable” version 2 or the “unstable” RC (release candidate) version 3. I went with version 2 and I’ve been second-guessing my choice every day.

Then I got to looking more deeply into forum packages, and I discovered the Simple Machines Forum — not open source in the conventional sense, but free to use, so long as you keep the copyright in place and follow a few other minimal restrictions. From everything I read about it, SMF sounded robust and full featured, and I wanted to try it out.

I started casting about for another forum idea, looking for some reason to set up another forum somewhere so that I could give SMF a whirl, and I had a brainstorm: I use a software program called Studiometry for my client and project management and invoicing. The company that makes Studiometry used to have a very useful user forum, but they shut it down sometime back with no explanation.

So I registered studiometryforum.com and set up the Studiometry Forum.

The SMF software was reasonably easy to install, although it was more difficult than I had hoped to find a theme that suited me. But everything worked as advertised out of the box (which isn’t always the case), and the theme I finally settled on was easy to install. I wanted to make a few minor changes to the html/css of that skin, and that was also easy.

I have only a handful of users so far on this new forum, but I’m hopeful it will gain traction and attract the Studiometry users who used to frequent the official Studiometry forums.

And even if it doesn’t, I gained valuable experience learning my way around a fine forum software package. It’s always beneficial to know what’s available and how easy or difficult it is to install and customize.

I needed to set up a MySQL database for a client recently. The client hosts their website with a large and well-known hosting company, which advertises MySQL available with all hosting packages. It should have taken just a few minutes to create the database and assign the db username and password, and then I could get on with the backend programming for the site. Several days and several billable hours later, I was just getting started.

First, I logged into the client’s hosting control panel and looked for the “MySQL” or “Database” option. No problem, that was easy enough to find. There was a link to “Activate MySQL” for the account. I thought that a little odd — most of the time, hosting accounts that have MySQL available simply have it available, no “activation” required. But okay…. I clicked the “Activate MySQL” link, then received a message that MySQL wasn’t available and the account would have to upgraded to a package that included MySQL. E-mail the client, explain the situation, tell them they need to upgrade their hosting account. It turns out that this host’s current packages all include MySQL, but older legacy accounts never had MySQL added to the available options.

Some days later, the client has upgraded their account to one of the newer hosting packages, which does include MySQL. I log back in to the control panel, activate MySQL, make note of the host, username, and password to use for database connections, then look for a link to phpMyAdmin to access the database. No phpMyAdmin anywhere, but the knowledgebase included instructions for downloading and installing phpMyAdmin. Wow, even the low-end $4/month hosting accounts at GoDaddy include phpMyAdmin already installed and ready to use.

I wouldn’t have to waste time installing phpMyAdmin if I could access the MySQL server through a secure shell, so I checked the account’s features for connecting via telnet or ssh. I didn’t find any information on that, so I tried a few random-but-likely ssh connections, but no go. Apparently the account has no telnet or ssh ability. So off I go to download phpMyAdmin.

I downloaded phpMyAdmin, uploaded it to the server, configured the connection settings, and tried to load up phpMyAdmin. I got a variety of “access denied” messages, all pretty worthless.

I’ve only installed phpMyAdmin myself a couple of times, and I wasn’t positive I had it configured correctly, so I decided to just use the connection string and try a database connection in a test page on the client’s site. I still got “access denied” errors, but this time the error message was more useful: The hostname for the database in the error messages was not the hostname specified in my db connection string. I double-checked and triple-checked the hostname specified in the hosting control panel and made sure I had it correct in the connection string, but I kept getting this same error. So I finally decided I needed to contact the host for technical support.

I hate calling tech support on the phone, because it invariably involves being put on hold for anywhere from 30 minutes to 2 hours, so I looked for a tech support contact form or e-mail link. No e-mail link, but I did find a form. Unfortunately, the form required that I enter the last 4 digits of the credit card number associated with the account. This is a client’s account, not my own, and I don’t happen to have the client’s credit card number, so I couldn’t use the form — even though I’m logged into the account’s control panel using the username and password associated with the account.

Looks like I have to resort to the telephone after all. I dial the tech support number and I get put on hold for about 45 minutes. I put my phone on speaker and tried to get some other work done while waiting, but every couple of minutes a robotic voice would issue from the speaker telling me how important my call was — repeatedly interrupting my concentration and making it virtually impossible to get any work done while waiting on hold. This is exactly why I hate calling tech support.

Finally, after 45 minutes of this, I get a live person who takes down the necessary information, puts me on hold while he checks into the situation, and then tells me that the database was activated on the wrong server and he would fix that but it would take 6 hours for the correction to “take.” I don’t get this at all — if he fixed it, why is there a six-hour delay? Nevermind, I was glad that the problem was identified, so I thanked him and hung up.

By now it’s well past my normal working hours, and the database is supposed to take 6 hours to be fixed, so I closed up shop for the day. The next morning, my connection script is working, and phpMyAdmin is working, and I’m finally able to start work on the client’s database.

Fortunately for me, I charge this client on an hourly basis, and you better believe they were billed for every minute I spent attempting to get their database set up and working, including the 45 minutes I spent on hold listening to that irritating robotic voice telling me how important my call was. If they had been with a better hosting company, they would have saved about 4 billable hours on their last invoice.

A provider of real estate web sites wrote:

“We do not use a system that allows you to make your own major content and format changes through your web browser (except you can add, edit, and delete your own listings and links) because that type of system requires a setup that does not allow search engines to index those pages.”

Malarkey! It is entirely possible to provide a system that permits the site owner to edit their site without erecting a barrier to search engine crawlability. The above statement really means that the site provider who wrote those words doesn’t know how to develop such a system.

A so-called “expert” for a major real estate portal wrote:

“Your site will never place well in search engines for competitive keywords (those actually used by searchers) if you are using a provider whose system allows you to make your own changes to your site through a web based administration area.

This is because the areas that you can access - search engines can’t.”

At the risk of repeating myself: Malarkey! Any website developer who knows a little bit about how search engines work, and a little bit about back-end server-side programming, should be perfectly capable of developing such a system. The search engines don’t need to access the admin area that the site owner uses to edit the site; the search engines only need to be able to access the public-facing pages. It takes the right knowledge and expertise to set the site up properly to make it easily crawlable by the search engines, but it’s not only possible, it’s common. Tropical Web Works has developed such sites for many clients.

This same “expert” wrote:

“All web site providers will tell you that their sites are designed to place well in search engines. What do they mean?

What they are saying is that their web sites have Meta Tags (see #4 above).”

My, what a sweeping statement in which this self-anointed expert claims to know exactly what “all web site providers” mean by something. Granted, meta tags (particularly the keyword meta tag) are nearly worthless — and if a web site provider claims that his sites will do well in search engines because it has meta tags, you should run, not walk, away as fast as you can. But it is definitely not the case that “all web site providers” make that claim. (Straw-man arguments are so easy to knock down.)

Another major web site provider wrote:

“To place well for a long list of keywords requires the production of many doorway pages.”

Malarkey! (There seems to be an echo developing in here.) Placing well for a long list of keywords is the natural result of developing a web site with many pages of on-topic quality content. No doorway pages are needed (nor are they beneficial). Write lots of good content for your site, make sure the site is structured logically and is crawlable by the search engines, and get some inbound links to some of those pages of content, and your site is very likely to rank well for a long list of keywords.

There is lots more bad information out there. Let me know when you run across these kinds of things, and we’ll address them here.

Okay, enough of that. Today I’m going to talk about crawlability and web design. I got to browsing the Unique ID blog and read Zack’s post “Straight From Google, The Four Biggest Search Rank Factors,” in which “crawlability” is listed as the very top, highest priority, most important search engine ranking ractor for a web site. This week I’ve also been following a discussion at the High Rankings forums about whether web designers have any SEO responsibility when designing a web site.

The discussion at High Rankings opened with the story of a businessman who hired someone to design a web site for his business. The site was built in Flash, and, as might be expected, the businessman’s web site didn’t do so very well in the search engines. When he sought professional SEO help, he was flabbergasted to learn that an all-Flash site is likely to rank poorly, if at all, in the search engines.

He asked the SEO pro, “Why did the designer use Flash when he knew I wanted search engine visibility?”

A better question would be, why do designers design “search-engine hostile” web sites when they know clients want search engine visibility?

As things stand in the world of web design, anyone with some elementary graphic design skills can get themselves a copy of Dreamweaver or FrontPage and hang out their “Web Designer” shingle, offering their services for a fee to all comers.

Some of these designers do indeed have a lot of artistic talent with respect to creating pretty, aesthetically pleasing, visually attractive web sites.

What these designers lack is a fundamental understanding of the underlying code and structure of web pages, and a fundamental understanding of how search engines crawl and index web pages, and a fundamental understanding of how a web site needs to be structured in order to have a chance of getting search engine traffic.

So these web designers make a “pretty design” in Photoshop or Fireworks or Flash, and use the built-in export features from those programs to auto-generate the code or the Flash file. The client ends up with a very pretty site that hasn’t a snowball’s chance in hell of doing well in organic searches.

The web is still relatively new as a commercial medium, and there is still some level of technical knowledge required in order to build a crawlable web site. Daily we see self-labeled “professional” web designers creating all-Flash sites, or using fancy javascript-based rollover images for global navigation, or relying on other artsy-fartsy features that doom a site to search engine purgatory — a site that is uncrawlable by search engine spiders, and generally invisible in the search engines.

The client doesn’t understand why his beautiful site gets little or no search engine traffic. The client eventually discovers, if he’s lucky or persistent, that he now has to pay for his site all over again, this time to have someone else tear apart his beautiful artsy-fartsy site and re-build it using underlying code and techniques that the search engines can crawl.

Does it have to be this way?

Should it be this way?

I say no, it shouldn’t. Some people might argue that a web designer’s responsibility is to design pretty stuff, not to perform search engine optimization. That’s true up to a point — I wouldn’t posit that it’s the web designer’s responsibility to do link building or write linkbait articles or do keyword research, unless those activities are explicitly included in the agreement.

But I do argue that anyone who holds himself out as a “professional web designer” should have a broad and fundamental understanding of the technology of the medium and the factors that are required for success in that medium. I do argue that the “professional web designer” is holding himself out as an expert, and the client is relying on the expert’s knowledge and experience.

The client shouldn’t have to have a specialist’s knowledge of the medium — that’s why the client hires a professional. When I hire a contractor to build a house, I shouldn’t have to become an expert on building houses, and I shouldn’t have to give the contractor explicit detailed instructions about how to run the wiring so it doesn’t burn the house down. I expect the contractor — the professional — to have the knowledge and expertise to do that himself, even if the contract doesn’t explicitly state that the contractor will run the wiring so that it doesn’t burn the house down.

In an ideal world, building contractors would always run the wiring so that it doesn’t burn the house down, and in that same ideal world, web designers would always build crawlable web sites.

The only exception I would make to this general rule is when a client specifically requests features that will cause crawlability problems, and, after being educated by the web designer about the consequences of his request, the client insists that his aesthetic vision is more important than search engine visibility. The client is paying for the site, after all. But even then there are usually steps the web designer can take to mitigate and overcome the problems — including text links in the footer to complement the pretty Flash buttons at the top of the page, for example.

Meanwhile, we live in an imperfect “buyer beware” world where the web designers who understand the medium are competing against the web designers who don’t. Clients have to educate themselves sufficiently, and ask lots of questions of potential designers, in order to be sure they end up with a crawlable web site.

There is a host of issues surrounding this edict from Google’s pet spam fighter, all of them ugly.

Google’s pet nofollow tag is a non-standard bit of code, created by Google, supposedly to help combat link-dropping spam on blogs. At least, that’s what Google said about it when they first introduced this tag.

Then they expanded its “recommended” usage to telling us that we should put a nofollow link condom on all links that we can’t personally vouch for, or that we don’t want to pass link juice to.

During all of this, Google has been vague about exactly what nofollow does. Initially, Google told us that nofollow would simply cause a link to not pass link juice. The implication was that Googlebot would still follow the link and spider the target page:

Google’s Blog

“when Google sees the attribute (rel=”nofollow”) on hyperlinks, those links won’t get any credit when we rank websites in our search results. This isn’t a negative vote for the site where the comment was posted; it’s just a way to make sure that spammers get no benefit from abusing public areas like blog comments, trackbacks, and referrer lists.”

Then they told us that nofollow had the effect of the standard nofollow meta tag:

Google Webmaster Help Center

Meta tags can exclude all outgoing links on a page, but you can also instruct Googlebot not to crawl individual links by adding rel=”nofollow” to a hyperlink.

Then they told us again that such a link simply does not pass link juice:

Matt Cutts’ Blog

The rel=”nofollow” attribute is an easy way for a website to tell search engines that the website can’t or doesn’t want to vouch for a link…. In an ideal world, nofollow would only be for untrusted links.

Then they told us again that it acts like the nofollow meta tag:

Matt Cutts’ Blog

At a link level, you can add a nofollow tag on the granularity of individual links to prevent Googlebot from crawling individual links

Google can’t make up its mind what the nofollow tag even does — and now Google is instructing us that we are to use nofollow on all paid links — and asking us to snitch on our colleagues who might be in “violation” of Google’s “guidelines.” Mr. Cutts offered no explanation of what Google plans to do, or may do, with respect to sites selling such links, or buying such links. He offered no explanation with respect to what Google may do with the “paidlinks” reports he requested. Will Google ban the sites selling such links? Ban the sites buying such links? Devalue the link juice of the paid links? Devalue the link juice of all links on the “offending” site? How will Google itself verify whether a link is a paid link?

Further, Mr. Cutts made no distinction between links that are subject to editorial review versus free-for-all paid links that a site will sell to all comers. Google’s own webmaster guidelines recommend that we pony up the $300 yearly fee to Yahoo for a listing in Yahoo’s directory. Google says the Yahoo directory exercises editorial review and that you’re paying for the review, not the link. But what about Joe Blow’s directory, in which Joe Blow exercises editorial review before accepting a submission? What about trusted, authority sites that exercise extreme discretion in deciding what paid links to accept, only accepting those that the site deems to be worthy and of interest to the site’s readers?

Mr. Cutts offered no explanation of what Google might consider a “paid link.” A link can be “paid for” in any number of ways: An outright exchange of money for links. A trade of links for links. A trade of services for links. A link from a charitable organization in acknowledgment of a donation. A link in exchange for a discount on products or services. A link resulting from a paid review of a product. A link resulting from a free review of a product provided free to the reviewer. This list could go on endlessly.

Google is revealing an unsurpassed arrogance in expecting the world wide web to change its linking methods for all links on all sites that are paid for. To use non-standard code in order to satisfy Google’s failing method of ranking pages in search results. To use a tag that is shrouded in mystery, with no clear explanation of what it does, or what it is supposed to do, or what it might do in the future.

Google even goes so far as to “suggest” that if we don’t want to use nofollow, we should use javascript to generate these links — which would create a usability and accessibility issue that most of us would prefer to avoid.

Google’s recommendation, in fact, is completely contrary to the original intent of linkage. Here is what Tim Berners-Lee wrote about links:

The intention in the design of the web was that normal links should simply be references, with no implied meaning.

A normal hypertext link does NOT necessarily imply that

One document endorses the other;

Source: http://www.w3.org/DesignIssues/LinkLaw

I say, Google, fix your algorithm, but don’t expect — or instruct with heavy-handed arrogance — the webmasters of the world to change their standard linking practices to make it easier for you to clean up your act. For shame, that a company whose motto is “Do no evil” would commit such an evil.

• Join the YesFollow Project

First thing: You want to create multiple campaigns, with each campaign focused around a particular product, category or topic. For example, you could create Campaign A for Product A, Campaign B for Product B, and so forth. But you might also benefit by creating a campaign about Hot New Feature in Product A, in which you would target your keywords, ad text, and landing page to that particular feature, and another campaign about Great User Benefit in Product B, in which you would target that user benefit of product B.

Each campaign should be tightly focused on a single theme. Make up your keyword list for each campaign to reflect the theme of that particular campaign. Write the ad(s) for each campaign to reflect the theme of the campaign, using the keywords from the campaign. For example, if Product A’s hot new feature is something that will save users money, you might run ads with headlines like “Save Money With Hot New Feature,” “Hot New Feature Saves Money,” and so on, and the text of the ad should focus on that theme: “Product A Increases Your Bottom Line By Saving You Money”. (Please note: The sample ad text and headlines I’m using here are quick, off-the-top-of-my-head example, not well-thought-out ways to promote Hot New Feature of Product A.)

When you’re ready to focus on your Product B Great User Benefit campaign, you’d write your ads with ad headlines and text that use those keywords: “Great User Benefit in Product B” for the headline, and ad text that refers to or describes the user benefit.

What you want is for people who search for, say, “Hot New Feature,” to get your “Hot New Feature” campaign, with an ad headline that reads “Hot New Feature” and ad text that focuses on that feature. Someone else who searches for “Great User Benefit” would get your ad headlined “Great User Benefit” and ad text that promotes the user benefit of your product. Don’t make the mistake of lumping all your keywords and ads together under one campaign — think in terms of “laser targeting” your ads to the search words that people are using.

2. Opt Out of the Content Network (At Least Initially)

Second, after you create a campaign, be sure to go into “Edit Campaign Settings” and either opt OUT of the “content network” altogether, or else check the box “Content bids” to set separate prices for content clicks. This allows you to specify much-reduced bid amounts for your ads to appear on the content network. The “content network” is when ads appear not on Google’s search results page, but on regular websites that carry Google ads. A lot of these websites are junk, and they will NOT bring you good targeted leads. You’re opted in to the content network by default, which I think is pretty scummy of Google. I recommend that you opt out initially, and after you’ve spent some time learning your way around the system, only then experiment with opting in to the content network. There are a lot of good sites in the content network that you’ll probably want your ad to appear on, so you probably should opt in at some point, in a small way, with a much-reduced bid for content network ads — but not right away. First learn how to manage campaigns and write targeted ads and set up keywords lists, and only then should you venture into the content network.

3. Exclude myspace.com

If/when you do opt in to the content network, go into the “Tools” section, the Site Exclusion link, and opt OUT of myspace.com. When myspace.com started carrying Google ads, thousands of advertisers were suddenly hit with huge advertising bills because of all the kids on myspace who click-click-click willy-nilly on everything they see, with no intention of buying anything. And even after excluding myspace, keep an eye on the sources of clicks to see if your ads are getting clicks from any other sites sending you large amounts of worthless traffic. It’s one thing to pay for good traffic; it’s something else altogether to pay large amounts of money for huge amounts of worthless traffic.

4. No Search Network

While you’re in there editing the campaign settings, I also recommend unchecking the box to opt OUT of the “Search network.” There’s good stuff that you’ll be missing — AOL’s search, for example — but Google’s so-called “search network” is mostly garbage spam sites that will not bring you good traffic. I think until Google cleans up its “search network” everyone should opt out of the search network.

5. Keep Tabs On Your Campaigns

The first few days (even the first few weeks) that you’re running PPC ads, you should spend some time every day checking on the clicks you’re getting, and the ads and keywords you’re getting them from, and conversion rates. Don’t just set up some campaigns and then ignore them — go into your account EVERY DAY and see what’s going on. Dump the ads that aren’t performing, or edit the headlines and/or text. Very minor tweaks in the wording can make a huge difference. If you have ads that are performing well, create some additional ads very similar to those, but with minor changes, and see if they perform even better. If you have ads with good click-through rates, you ultimately end up paying less per click on those ads, even while the ads themselves get shown higher up in the block of paid ads. Ads with poor click-through rates will get shown lower down, and cost you more per click. So it’s well worth spending some time to experiment and find the ads that work best.

6. Learn How Keyword Matching Options Work

When setting up your keyword list, pay particular attention to the keyword matching options — broad matching, phrase matching, and exact matching. You should probably start with phrase matching. Broad matching can result in this scenario:

I have a client who runs humpback whale watching tours. There are sperm banks that (I’m guessing) bid on the phrase sperm bank, using broad matching. Google’s world-famous algorithm says to itself “sperm bank …. sperm whales …. humpback whales ….. Humpback whales must be related to sperm banks. I know! I’ll show sperm bank ads to people searching for humpback whales!” (I think the sperm banks may have wised up after a few clicks on those ads — I’m not seeing those so much anymore.) It’s a clever algorithm, but it’s not very smart. This is what happens when you let Google’s algorithm run wild with broad matching. Stick to phrase matching and/or exact matching. You might end up with your ad appearing for searches on bull sperm if you use broad matching. If/when you decide to experiment with broad matching, keep a *real* close eye on what’s happening with your ads. You might not like the results.

7. Setting Your Daily Budget

In your campaign settings, set your daily budget fairly high initially (although not so high that you’ll go broke from paying your AdWords bill, of course). If you set it too low, your ads hardly ever get shown. Be prepared to throw some money away those first few days with a high daily budget — higher than you really want to spend on an ongoing basis — so that you get enough data to get a feel for which ads are working and which aren’t. Then after a few days lower your daily budget to a more reasonable level. I recommend lowering it in fairly small increments, so that you can see what daily budget results in how many clicks results in how many sales. Tweak as needed until you reach a level you’re comfortable with.

You set the “cost per click” that you’re willing to pay for each click separately from the daily budget, and you can edit the cost-per-click separately not only for individual campaigns, but for individual keywords within each campaign. You might find, for example, that the keyword nutritious dog food converts better than the keyword healthy dog food, so you might be willing to pay more per click for that keyword.

8. Target Your Landing Pages

And last — at least, last for today’s entry: For the most part you don’t want to send people to the home page of your site. For your Hot New Feature in Product A ads, send them to a page about the hot new feature in product A. For the Great User Benefit in Product B ads, send them to a page about the great user benefit in product B. Create some additional pages on your site to serve as landing pages for your ads, if necessary.
Remember the “laser targeting” I mentioned above? Target with your landing page, too. Lead the user down the path that he’s already looking for: Targeted keywords that trigger targeted ads that lead to targeted landing pages. Every step of the way, the user is following a path that takes him to exactly what he’s looking for. And be sure to include a clear (and targeted) call-to-action on the landing page: “Buy Product A Now” or “Start Enjoying Great User Benefit in Product B Today.”

After you’ve explored the Google AdWords interface and set up a couple of campaigns, come back and read this again. It will all make much more sense then.

Read some good information on pay-per-click advertising for real estate agents.

Developing a new web site — or re-developing an old one, for that matter — typically involves consulting with the client to determine the site’s target audience and primary objective, creating an attractive and functional design, turning the design into properly coded, valid html, building out the pages of content, and writing the server-side programming to perform whatever dynamic features are needed. But there are several steps that are frequently overlooked before a site “goes live.”

A custom “404 not found” page should be created. At minimum, it should incorporate the site’s overall design and navigation links, and might also include a search form, links to the most popular sections of the site, and/or a way to contact the site owner or webmaster for assistance. And make sure that requests for non-existent pages actually get a “404 page not found” server response. If any other server response is returned — particularly a “200 OK” response — the site could easily become persona non grata in the search engines, among other problems.
A robots.txt file should be created to tell the search engine spiders what pages or parts of the site should not be spidered. Even if you want every page to be spidered, a robots.txt file should be placed in the document root of the site, so as to avoid filling up the site’s error logs with “not found” entries for a non-existent robots.txt. This makes it much easier to spot errors resulting from actual bad links when you examine the error logs.

A canonical URL redirect should be implemented to send all site traffic to your desired canonical URL — either www.example.com, or just example.com (without the www). Whichever you prefer, your should make sure that all traffic to the other form is redirected via a proper 301 redirect.

Test all forms and other interactive features. Submit every form. Attempt to submit forms without required information, or with invalid information. Try to break them. Try really really hard to break them. And make sure that whatever is supposed to be done after the form is submitted actually happens. If an e-mail is supposed to be sent to the site owner, test it, and make sure the owner gets that e-mail with all the appropriate information. If there’s a search engine, search for some things. If the site relies on cron jobs, set them up ahead of time, and make sure they’re running as scheduled and performing as expected.

Test all redirects and rewrites. If the site uses Apache’s mod_rewrite module to present search-engine-friendly URLs, test them all, and test the non-search-engine-friendly versions, to make sure that every bit of content can only be reached by one unique URL.

Check subdirectories for directory listings. Make sure that directory indexes are turned off, and/or for any subdirectory that doesn’t have an index page, plop one in there.

Test all internal links and all outgoing links. Make absolutely sure that every single link leads to the right place. We don’t need to stinkin’ dead links!

There’s simply no excuse for ever launching a site without having completed each of these steps.

I have only the most minor, off-hand reference in the site. There’s a quotation from Hank Hill in my quotation database, which is a simple example that I put on my database programming page to give a little demo of what can be done with databases, for clients who aren’t familiar with their use.

So I thought to myself, “Self, if you can rank at #66 and #32 respectively in Google and Yahoo for that search term without even trying, could you do better if you actually tried?” And so I logged into my GoDaddy account and registered hankhillquotes.com. Then I set up a simple site with a basic database, which I populated with a few quotes from each of the major characters.

I plan on spending a few minutes a day adding quotations to the database, and over time expanding the site somewhat too — perhaps adding character bios, episode descriptions, things like that.

I’ll post my progress here and let you know what shakes out. Hopefully, there won’t be some mean-spirited person who comes along and puts major effort into outranking me. This is just a small effort on my part to see what happens.

Web sites need to contain good content. It doesn’t matter how many people “ooh” and “aah” over how pretty your site is. If visitors don’t find what they’re looking for, they’ll go elsewhere.

Web sites need to be functional and usable. If your site contains exactly what your visitors are looking for, but they can’t find it because of a random, incomplete, or incoherent navigational structure, they’ll go elsewhere. If visitors find what they’re looking for, but when they fill out your contact form and hit “submit” they get an incomprehensible error message, they’ll go elsewhere.

Web sites need to be spiderable by search engine spiders, so that they can be found. If your site is extremely functional and usable, and contains great content, but your potential clients/customers can’t find you when they search for your product or service, they’ll go elsewhere.

Each of these factors can — and do — have entire books written about them. And there’s much more. In this brief post I’ve barely touched on the essential components of an effective web site. The web site owner shouldn’t need to become an expert in all of these things. But the web site developer should absolutely have more than a passing familiarity with all of the aspects of effective web site development, or should work with people who do.

Every day I see web sites developed by so-called “professionals,” for paying clients, that don’t even come close. I see sites in which every single page is a giant sliced-up graphic exported in toto out of Photoshop. I see sites that use Flash for all internal site navigation. I see sites that use frames and iframes for all or most of the site’s primary content. I see sites that have the same title and description for every single page. I see sites that return a “200 OK” header for non-existent pages. I see sites with plenty of pretty pages but virtually no content. Sometimes I see all of these things in a single site!

Attention, web site owners, and those in need of web sites: Tropical Web Works is not the only web site development firm that can develop an effective, functional site for your business. But please bear these points in mind and ensure that whoever you do hire understands these basic concepts.

Pam is not navigating directly to the web site she wants. She is searching for that web site, even though she knows the exact domain. She searches, and the site appears at the top of the search results, and she clicks the link to go to the site. This happens all the time. I see the results in the logfiles, too — the logfiles show me when someone has conducted a search for a domain and then clicked the domain in the search results to get there. Every time I see it, I shake my head in amazement.

This behavior is probably a symptom of “satisficing,” a term coined to refer to something that may not be ideal, but works sufficiently well for a person that they’re not willing to learn a better way. People are used to using that ubiquitous Google Toolbar for everything, or they perceive “the Internet” as being whatever they can get to through their Yahoo homepage. They are simply not aware of any other method of navigation.

Regardless of why this behavior occurs, it means that web site owners, who go to a lot of time and effort to promote their web sites, need to make darn sure that their site comes up at the #1 spot in a search for their domain.

I just signed on a new client this week, to do a complete redevelopment of an existing web site. The current site is so badly constructed that it does not come up #1 in Google in a search for the domain. It doesn’t even turn up in the first 5 pages of Google in a search for the domain. The site is #1 in MSN and Yahoo — but not in the first 5 pages of Google! This is a site that’s been around for several years — there’s no aging delay or sandbox effect at work here. There aren’t many guarantees in the world of search engines, but I will have this site turning up at #1 in Google in a search for the domain before long.

Another site I’m currently developing, a brand-new site, brand-new domain, registered for the first time 2 months ago and not yet fully indexed in any of the big 3 search engines, turns up #1 in all 3 search engines in a search for the domain. That is as it should be. There is no excuse for anything else.

What set me off this morning? A client sent me a Word file. A simple, one-page Word file. I double-clicked it.

When the file opened, it opened with an error message: The dimensions after resizing are too small or too large.

Huh? What does that mean? Does anyone have a clue? How could a programmer think that could possibly be a helpful or informative error message? Even if I had any idea what dimensions are being referred to, I don’t know if they’re too small, or if they’re too large.

Whoever wrote that error message needs to be smacked upside the head with a clue bat. And then promptly fired and banned from software development for the remainder of his natural life.

But sometimes that’s simply not possible. Particularly when the potential client is looking for improved search engine performance or better usability.

Sometimes a site can be improved dramatically by minor tweaks: Add unique, custom title tags to each page, add alt text to images where needed, add text-based site navigation, beef up the content, and a few other improvements. Bang, I’m done, and the client can look for improved performance whenever the search enginges see fit to recognize the changes to the site. (That might take days, it might be weeks, it might be months — I try to make sure the client is aware that I have no control over what the search engines do in that regard.)

But all too often the site is constructed so badly that nothing but a total redeveloment will do:

• Frames — frames-based sites are not only less usable for human visitors, but they still throw roadblocks in the path of the search engines trying to index the site.
• Frames Part Two — Even worse is sites that frame content from other sites. The framing site gets no credit in the SEs for the framed off-site content.
• All-Flash Sites — The search engines are working on their ability to index Flash sites, but your all-Flash site is unlikely to be the breakthrough. Flash elements should be used sparingly to add to the visitor’s experience, but Flash should not be the site.
• Search-engine-hostile Dynamic URLs — There are so many ways to go wrong here that it’s hard to list them all. Session IDs in URLs. Meaningless long numbers and section ids and category ids. URLs that display all the site’s content on the same page (using formats like “index.php?page=thispage”).It’s so easy to use the magic of server-side technology to write user-friendly and search-engine-friendly URLs, and to keep session IDs out of URLs. Which type of URL do you like better:
  • http://www.example.com/products/red-widgets.html, OR
  • http://www.example.com/index.php?cat=52 &section=355&prodid=125&sess_id =5348725987ofdj30487590fjglkae098t87q34085uofilajg

  Yeah, I thought so. Me too. The search engines like the first one better too.

• Lack of content — if there’s little or nothing for the search engines to spider, there’s little or no likelihood of any of the pages turning up in searches.
• Sites that requires the user to submit a form before seeing the content — Hey, search engine spiders don’t submit forms; they’ll never see all that great content on your site.
• Invalid tag soup — Badly coded sites with such badly formed html that it’s darn near impossible to work with the code. When I’m optimizing a site, I need to get down-and-dirty in the code, and pages with invalidly nested tables, invalidly nested divs, incomprehensible, invalid code everywhere — well, I just can’t work with it. It’s hard to even touch the code in a site like that, because you just don’t know what will happen.
• Rigid, inflexible design and code — This usually results from so-called “designers” who design a pretty image in Photoshop or Fireworks and then slice it up and export the entire thing from their image-editing program. The code for these sites is so rigid that you can’t make a single change without breaking the entire thing.
• Template sites based on some badly designed content management system — There are too many big companies out there who have built half-baked content management systems that allow anyone to “build their web site” with a few clicks of their mouse. Nice concept, but usually badly implemented. These sites are usually required to be hosted with the “big company” and runs off their database on their servers. These template sites often don’t allow custom title tags or custom descriptions for each page. They also don’t allow any access to the underlying code. This means I can’t make search-engine friendly URLs, and I can’t set up a 301 redirect, and I can’t eliminate the appearance of duplicate content, and I can’t do any of the easy tweaks that are needed.

All too often, I see sites that suffer from most or all of the above problems. I may be able to work around one or two of the above items, but when a site presents numerous serious technical problems, there’s just no point in attempting to patch the old wineskin.

It’s a shame when someone has paid good money for a site that is so badly constructed that it can’t be improved. But there are times that it makes more sense to throw out the old and build anew.