[mp1] Segfaults in ap_headers_out

[Christopher Shumway <suppressed>]

Hi folks,

About once every 15-45 minutes or so a httpd process will segfault when serving
a modperl page.  I've built apache httpd, modperl and perl with debuging
symbols, and have gotten a few core images.

Here are the relvent software versions:
FreeBSD 6.1-RELEASE-p14 running an SMP kernel
Perl 5.8.8 built for i386-freebsd-64int
apache+mod_ssl-1.3.37+2.8.28
mod_perl-1.29

Here's the gdb backtrace:

#0  0x08050c55 in ap_table_get (t=0x31412d38, key=0x9d67a88 "Expires") at alloc.c:1444
#1  0x2850a6df in XS_Apache_cgi_header_out (cv=0x8899c34) at Apache.c:2662
#2  0x285c5f88 in Perl_pp_entersub () at pp_hot.c:2913
#3  0x285a5ecb in Perl_runops_debug () at dump.c:1459
#4  0x2854bf19 in S_call_body (myop=0xbfbfead0, is_eval=0 '\0') at perl.c:2731
#5  0x2854bacb in Perl_call_sv (sv=0x889a258, flags=4) at perl.c:2646
#6  0x284f6497 in perl_call_handler (sv=0x889a258, r=0x88e0038, args=0x0) at mod_perl.c:1668
#7  0x284f5b24 in perl_run_stacked_handlers (hook=0x2851fd2f "PerlHandler", r=0x88e0038, handlers=0x889a090) at mod_perl.c:1381
#8  0x284f3bb1 in perl_handler (r=0x88e0038) at mod_perl.c:904
#9  0x08054d96 in ap_invoke_handler (r=0x88e0038) at http_config.c:476
#10 0x0806a3ed in process_request_internal (r=0x88e0038) at http_request.c:1299
#11 0x0806a44c in ap_process_request (r=0x88e0038) at http_request.c:1315
#12 0x08061177 in child_main (child_num_arg=8) at http_main.c:4898
#13 0x0806146c in make_child (s=0x8099038, slot=8, now=1172591147) at http_main.c:5077
#14 0x080617a8 in perform_idle_server_maintenance () at http_main.c:5262
#15 0x08061d6d in standalone_main (argc=2, argv=0xbfbfeeec) at http_main.c:5525
#16 0x08062391 in main (argc=2, argv=0xbfbfeeec) at http_main.c:5793

What looks like is happening is the table being passed to ap_table_get() is bogus.

(gdb) frame 0
#0  0x08050c55 in ap_table_get (t=0x31412d38, key=0x9d67a88 "Expires") at alloc.c:1444
1444        table_entry *elts = (table_entry *) t->a.elts;
(gdb) list
1439        t->a.nelts = 0;
1440    }
1441 
1442    API_EXPORT(const char *) ap_table_get(const table *t, const char *key)
1443    {
1444        table_entry *elts = (table_entry *) t->a.elts;
1445        int i;
1446 
1447        if (key == NULL)
1448            return NULL;
(gdb) print *t
Cannot access memory at address 0x31412d38

I'm not sure why, but the data structure Apache r doesn't appear to be getting
populated  correctly:

#1  0x2850a6df in XS_Apache_cgi_header_out (cv=0x8899c34) at Apache.c:2662
2662        if((val = (char *)table_get(r->headers_out, key))) 
(gdb) list
2657        char *val;
2658 
2659            SV *    RETVAL;
2660 
2661            r = sv2request_rec(ST(0), "Apache", cv);
2662        if((val = (char *)table_get(r->headers_out, key))) 
2663            RETVAL = newSVpv(val, 0);
2664        else
2665            RETVAL = newSV(0);
2666 
(gdb) print *r->headers_out
$5 = {a = {pool = 0x31412d38, elt_size = 0, nelts = 0, nalloc = 0, elts = 0x0}}
(gdb)

I'd love to get to the bottom of this one, but I could use some insight.