Re: [ic] Googlebot Getting 500 Errors ... but he's the only one

[Peter <suppressed>]

On 06/01/2005 01:06 PM, Bryan Gmyrek wrote:
> --- Bryan Gmyrek <suppressed> wrote:
>
>
> > Hi,
> >
> > By the way the site I'm having the trouble with is http://www.neartexpress.com/ 
> > I haven't been able to check email much lately so sorry for the late reply.
> > Thanks Jonathan for providing that script!  I ran it and got the same results as Mike
> > (everything
> > OK).
>
>
> Considering my access logs (i.e. googlebot gets some success and then failure) I altered
> Jonathan's code to loop until there was a 500 error and then die (see end of this email for the
> code).  Here are the results:
>
> $ perl test_google_bug.pl http://www.neartexpress.com/liROSSPL3153.html | tee out.log
> <many lines edited out ... all were ok status>
> ok 613 - check http status is 200 without if-modified-since
> #     header
> ok 614 - check http status is not 500 with if-modified-since
> #     header (status is 200)
> Tue May 31 13:31:10 MST 2005
> ok 615 - check http status is 200 without if-modified-since
> #     header
> ok 616 - check http status is not 500 with if-modified-since
> #     header (status is 200)
> Tue May 31 13:31:14 MST 2005
> ok 617 - check http status is 200 without if-modified-since
> #     header
> ok 618 - check http status is not 500 with if-modified-since
> #     header (status is 200)
> Tue May 31 13:31:18 MST 2005
> not ok 619 - check http status is 200 without if-modified-since
> #     header
> not ok 620 - check http status is not 500 with if-modified-since
> #     header (status is 500)
>
>
> So it eventually does get an error...  I'll have to look into it more though since Googlebot
> definitely gets the error more often than that.
>
> Best,
> Bryan
>
>
> Code::
> #!/usr/bin/perl -w
>
> use WWW::Mechanize;
> use Test::More tests => 2;
>
> my $url = $ARGV[0];
> my $ua = WWW::Mechanize->new;
> my $date = `date`;
>
> my $status = 200;
> while($status != 500){
> print `date`;
>
> $ua->get($url);
> ok($ua->status == 200, 'check http status is 200 without if-modified-since
>     header');
>
> #$ua->add_header('IF_MODIFIED_SINCE' => 'Wed, 08 Sep 2004 11:09:13 GMT');
> $ua->add_header('IF_MODIFIED_SINCE' => $date);
> $ua->get($url);
> ok($ua->status != 500, "check http status is not 500 with if-modified-since
>     header (status is ".$ua->status.")");
>
> $status = $ua->status;
> }

Just on a hunch, try setting the if-modified-since header to a date in 
the future for this test and see what happens...

Peter
_______________________________________________
interchange-users mailing list
suppressed
http://www.icdevgroup.org/mailman/listinfo/interchange-users