Sitemap    Home

Making Your Site Better

My problem is not unique in the computing world, in fact, people make a living off of what is now called "Search Engine Optimization", the field that deals solely with make a page rank higher in the SERPS.

Update 3/26/2005- If you are wondering why my PR is still 0 even though my site is most awesome. I think it is because I do not have a ROBOTS.TXT file.

Here's how to write a robots.txt file

Why isn't Google Crawling my sub-pages?

Because googlebot is being polite. Unless it finds a "robots.txt" file in the root directory, it won't crawl any other pages except default.htm(l) or index.htm(l). The easiest and best robots.txt file for this purpose is a text file with:
User-agent: *
as the contents is the best. Once this is in place all bots will crawl every one of our sub-pages. Files in directories outside of the htdocs, wwwroot, www etc. directory structure, should not be crawled if your server is set up properly. Forum that I need to set straight about Google not crawling sub-pages. They just don't get it.


what is a "site map" and how do you make one?

Another feature that helps the bots crawl your website is to make a file called "sitemap.html" with links to every one of your sub-pages, basically a directory listing. This prevents the bot from having to exhaustively search each page for links- one stop shopping. A good way to make one of these is to go to dos and do a dir>sitemap.html, then edit it and add all the anchor tags. How to add anchor tags. Once you have the sitemap drawn up, you need to link every page to it, preferrably at the top of the page.


Once I added a sitemap and a robots.txt file, inktomi immediately crawled all my pages. Then my site popped up to the top of many of the SERP's that I was on. Best example is "bader alaska" where I went from #47 to #1 overnight.

Anchor Text

Anchor Text is the part following the link betweent the > and the </a > All about anchors.

Domain Names

If you are serious about your site, you will get a decent domain name. They are cheap, and you would be surprised what is available with a little imagination. I snagged "www.thebestpageonearth" with no problem. Spend the $15/year, it's worth it.


Originally created so spammers couldn't spam forums to their site. The forum can place this with ever link, so they don't drain pagerank. This can also be put on any site. I'm doing this because google doesn't crawl my supages and my PR is 0. Help!

rel="nofollow" details

Has Google penalized your site?

Remote the REVISIT meta tag... there is much evidence that this causes penalties.


link popularity checker tool
On Altavista All The Web Yahoo Google and MSN had zero links to my site.

Google Datacenters

Q: What is a datacenter?

Q: How do I use one?

What does being "sandboxed" mean?

The sandbox is a virtual place that search engines place sites that are unknown. This way, the site can interact with the search engines without degrading other sites. Remember, the SERPS are like the WWW, everything is intertwined. If a site is new, or has done things that are "bad", then they will be placed in the sandbox. Some examples of bad behaviour are: link farms, listing a whole bunch of keywords that don't apply to a site or participating in spam. These actions tend to cause the SERPS to be inaccurate.

Other sources of info

Search Engine Optimization Forum W3 Consortium HTML Validator

Here's something to try in your spare time, run these searches into google:

I can't believe it! I let my main web site fallow while I was building my house. It no longer appears in Google's index. I've been submitting my URL to Google each time I update my site to no avail. I did some research on what page rank means, and I'm certain my page is a big fat ZERO. Maybe it has a negative page rank. Anyway, I want long-lost friends to be able to find me, so I need to be in the index. Here is my strategy to get back on: 1.Post everything I know in an organized manner. I'm going to crawl the vast depths of my brain, much like Google deep-crawls the web. 2.Scan almost all of my photos and place them in organized directories, indexed by location (world map), chronologically (timeline) and by subject (select thumbnails) 3. Continue to document my life with narration and digital photos (what's Charlie up to now?) I promise it will be interesting. Make it like a reality TV show. 4.Keep on typing! Good article on how Google's index is updated. It turns out that all I really need is patience. Since I haven't updated my website in about 2 years, Google doesn't fresh crawl my website. I need to wait for the next deep crawl which will happen in no more than 6 to 8 weeks. Geez, by that time my site should be really good!

Google Lost ME!

Update: 1/12/05 Google found me again after 12 days. Now I am the #1 googlehit on the search string "charlie bader". My strategy is working!

My main page is indexed, but the others are not. The only other indexed pages are music.html, moss_island_oyster_farm.html

Update: 2/1/05 Now I am #1 and #2 GOOGLEHIT on charlie bader (no quotes required)

Google Crawl Log

I'm keeping track of how often Google crawls my site (I'm SOOO bored!)
January 9th
January 25th
February 7th
February 16th 11 days
February 28th 12 days
March 8th, posted March 9th, evening total time since last crawl: 9 days
I missed a crawl, so now google won't crawl me as often. Silly ISP at the Ohio University Inn wouldn't link up to ACS Alaska.
Mar 18, 2005 18:45:10 GMT Posted 20th 10 days
Next crawl March 28th.

My current rating... I've slipped on a few.

charlie bader #1

bader alaska #42(climbing)now 106!

tracy anna bader #3(climbing)

"ron bader" alaska #1

"ed vey" #3(climbing)

yevda #9

phccna #33 (from #5 to #33, now that's a conspiracy! Someone doesn't seem to like my opinions about the PHCCNA!)

natca alaska #1 (my other site)

early February

charlie bader #1

bader alaska #49

tracy anna bader #5

"ron bader" alaska #1

"ed vey" #5

yevda #9

phccna #9

natca alaska #1 (my other site)

Here's how I rated Late January with Google on various searches

charlie bader #1, #2

bader alaska #47

tracy anna bader #4

"ron bader" alaska #3

"ed vey" #5

yevda #9

phccna #5

natca alaska #1 (my other site)

Google Index Criteria

Google says that they crawl/index/link to your sub-pages. I'm finding that this is not the case for low-ranking sites such as mine. This places increased importance on what is put on your main page.

Sitemap    Home