>Grant wrote: > >>Thanks a lot for the info Phillip. I'd like to clarify a couple things >>though... >> >> >> >>>Usually with a 301 it takes a couple of runs from most spiders to decide >>>to go anywhere else into the >>>site. >>> >>> >> >>What is the correct way to forward from your domain to your site's index >>page so the spiders don't get confused? >> >> >Icdevgroup uses 302 and does not get indexed but for the first page, as >you may have noticed, from google >most other search engines will tromp all over the site and produce gory >listings >My opinion and of many others I have spoke with at >http://www.webmastersworld.com (where the GoogleGuy hangs out) say 301 >is the only way if your going to redirect and expect rankings. You could >use a doorway page >that uses java script and just place links into the sight with some >keywords in it for the search engines, but feel that is >very unprofessional and spammy myself. > > So pretty much everyone uses a 301 or 302 to get to the index page of their site, and therefore has to deal with this issue? >> >> >>>Now depending on how long your system has been running with a 301 >>>if you move now it will cause >>>you more problems. Realize that 301 is just like you told the mailman >>>you have a new address and then >>>you send a new change of address to all of your magazine companies. >>>Now how long does it take for them to get around to sending them to your >>>new address? >>>Then sundenlly you decide to send them and your mailman a new change of >>>address again even before >>>they have actually acted on your old change of address. Well you will >>>have at least 2 monthns before you get >>>any magazines or a good part of your mail will end up in >different places. >>> >>>So usually using 301 in difference to 302 that says temp move don't keep >>>record of it. This is a very bad things >>>when it comes to spiders if you keep bouncing arround. >>> >>> >> >>Are you saying a 301 or a 302 is better for spiders? >> >> >Here is a link from Google that talks about what they feel you should do > >http://www.google.com/remove.html > >And the snippet that talks about 301 > >*Change the URL of your website* > >Since Google's crawler associates the content of a page with its URL, >there is no way to manually change the URL that is displayed for your >website. The URL will be updated the next time we crawl your site. The >crawler revisits each site according to an automatic schedule, and we >cannot manually accelerate the date on which your site will be recrawled. > >If the URL of your website has changed since we last crawled it, you may >use the URL submission form <http://www.google.com/addurl.html> and the >URL removal methods described below. However, the URL submission form >does not take effect immediately, so using the URL removal feature may >leave your website inaccessible from Google until we crawl your site again. > >Instead of requesting a change from Google, we recommend that you ask >the sites currently linked to your old site to update their links (to >point to your new site). Also, don't forget to change any entries you >may have in the Yahoo! directory and the Open Directory. Finally, if >your old URLs redirect to your new site using HTTP 301 (permanent) >redirects <http://www.ietf.org/rfc/rfc2616.txt>, our crawler will know >to use the new URL. Changes made in this way will take 6-8 weeks to be >reflected in Google. > >I feel 301 is better.and also pay close attention to the time google >says it will take for the crawler to understand the >new address (6-8 weeks) > >>>This is spoken completly from experience since I did this myself and >>>have seen its effects. >>> >>>Also all of your DMOZ entries also need to point to your redirected >>>location to get credit for it. >>> >>> >>> >> >>Where are these entries? >> >> >If your site has been submitted to DMOZ at http://www.dmoz.org and since >Google and other search engines use >these listing for supporting your rankings they should be set to always >match your expected site location > Ok, thank you. >> >> >>>Point is this if you have just started doing this move, then leave it >>>alone. It will take at least 2 months for >>>google and a few others to catch up. If you have done this for awhile >>>you could completley lose at least >>>a months worth of crawls until they get around to seeing the new move. >>> >>>This happend to me and I got impatient myself and moved around again. >>>Lost much traffic and after talking to some people at webmasterworld, >>>they just told me to not mess with it and be patient they will crawl >>>your site within one to two >>>months. If your sids are not showing they will jump on it soon. >>> >>>-- >>>Philip S. Hempel >>>debian/rules >>> >>> >> >>It seems like there must be a better way to go about all this that doesn't >>use 301s at all so the spiders will head straight inside. What would that >>be? >> >>- Grant >> >> >> >Most SE's do not like 302 (temp redirect) and almost all suggest the >usage of 301 (permanant redirect) >302 does not push ranking onto the main page since this is want you want. > >Since I quit using 302 and went to 301 and a few other things I >went from page 5 in >the rankings to number 1,2,3,4,5 for over 15 key word sets and >went from 100 users >to over 800 users (not search engines) in a day average. > >Goggle spiders over 200 pages on my site now and we have as of >today on the average >have over 10 sales a day (from 1 every 3 weeks). (this is good for >a supposed part time business) > > >Hope this helps if you need more ask. > >(and please excuse typos, wrote this in a rush) >-- >Philip S. Hempel >debian/rules Here's what Google's doing on my site: 64.68.82.70 - - [26/Nov/2002:08:30:13 -0800] "GET /robots.txt HTTP/1.0" 200 0 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)" 64.68.82.70 - - [26/Nov/2002:08:30:15 -0800] "GET / HTTP/1.0" 301 330 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)" 64.68.82.5 - - [26/Nov/2002:08:39:22 -0800] "GET /cgi-bin/shop/ HTTP/1.0" 200 38303 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)" 64.68.82.7 - - [26/Nov/2002:08:49:59 -0800] "GET /cgi-bin/shop/policies.html HTTP/1.0" 200 35830 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)" 64.68.82.28 - - [26/Nov/2002:08:52:20 -0800] "GET /cgi-bin/shop/moreinfo.html HTTP/1.0" 200 39917 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)" That's it. This shows that they are getting into the main site, past the 301. They're just looking at a couple of pages though. I've verified with the Sam Spade browser that IC is sanitizing the URLs when the Google User Agent is used. Also, it's GETing "/cgi-bin/shop/", but I have NO links to that particular path anywhere in the site. The redirect redirects to www.mystore.com/cgi-bin/shop/index.html. How could it be hitting "/cgi-bin/shop/"? Thanks a lot for all your help Phillip. Hopefully others will benefit from this discussion too. Any idea why the Googlebot wouldn't be hitting up more pages? There are a ton of links on that front index page to all of my product categories. - Grant _______________________________________________ interchange-users mailing list suppressed http://www.icdevgroup.org/mailman/listinfo/interchange-users
Mail converted by mhonarc 2.6.15
This archive provided courtesy of JSW4.NET, Internet Hosting Services for Small Business.