RE: [ic] HELP - Inktomisearch stuck on ord/basket.html page

["Andrew Rich" <suppressed>]

> Jamie Neil wrote:
> > Andrew Rich wrote:
> >
> >> I initially stopped it by adding an entry to robots.txt for
> >> /cgi-bin/cartname/ord/basket.html and this pushed it off the page.
> >>
> >> I then thought orders were reduced and removed the entry from
> >> robots.txt.  Inktomisearch came back and became stuck on the page
> again.
> >
> >
> > I don't see any reason why adding pages (or directories) to your
> > robots.txt file would affect orders - this is the preferred way of
> > limiting spider activity on your site.
> >
> > Use something like:
> >
> >   User-agent: *
> >   Disallow: /cgi-bin/cartname/ord/
> >   Disallow: /cgi-bin/cartname/scan/
> >   Disallow: /images/
> >   Disallow: /cgi-bin/cartname/login.html
> >
> > As well as "ord" we also ban robots from our images directory (to
stop
> > our graphics turning up in image searches), and "scan" to stop them
> > indexing "more" links.
> 
> I should add that banning robots from "scan" will prevent them
indexing
> _any_ category/search links on a stock foundation system, which is
> obviously undesirable. We use action maps to do most of our links,
e.g.:
> 
>    /cgi-bin/cartname/category/spanners.html
> 
> and so this is not a problem for us.
> 
> We also feed spiders slightly different search results to normal
users:
> normal users get 10 results per page while spiders get one page with
all
> the results on it. This allows the search engines to index all the
> products in a category without having to follow any "more" links.
> 

Thank you Jamie for your feedback.

As we receive about 75% of orders via search engines we do not want to
do anything to stop them indexing our site.

Thia would obviously be a good start.

   User-agent: *
   Disallow: /cgi-bin/cartname/ord/
   Disallow: /images/
   Disallow: /cgi-bin/cartname/login.html

I am not familiar with action maps so I will have to do some reading
there.

I have tried to move from the Foundation site to the new Standard site
structure (perhaps only new name) so not sure if this makes any
difference!

Regards,
Andrew Rich

_______________________________________________
interchange-users mailing list
suppressed
http://www.icdevgroup.org/mailman/listinfo/interchange-users