Instability at startup for Apache2/mod_perl2 using worker MPM

[David Scott <suppressed>]

OK this is my third message today, hopefully it will be the last.

I've built Apache 2.2.3 and mod_perl 2.0.3 with libapreq 2.08 and the 
worker MPM.  There appears to be some kind of race condition at startup 
that prevents the server from coming up.  This only happens once in a 
while, and can be fixed by some minor configuration or code change (ie, 
adding a 'print STDERR "foobar\n"' to one of my Perl modules can cause 
the problem to occur; moving the print statement by one line can fix it).

There are two manifestations of this problem: either

(1) The first httpd process goes to 99% CPU usage and stays there 
indefinitely, never doing a restart; or
(2) The restart occurs and the new parent process forks *one* child and 
then goes to 99% CPU usage indefinitely, never completing the startup 
sequence.

I've been porting a fairly complex pure-Perl legacy application from 
mod_perl 1 to mod_perl 2.  The reason we're trying this is because 
memory requirements in MP1 are horrendous and child process death is 
extremely expensive.  The application appears to work fine once the 
Apache2 server starts.

In addition, it's not unusual for the mod_perl 1 server to take a long 
time to start up or shut down.

I suspect that there is some nasty interaction between my Perl modules 
(of which there are many).  Does anyone have suggestions on how I can 
track down the problem?  Apache2::Status is useless until the server 
starts, obviously.

Here is my system information:

Apache/2.2.3 (Unix) mod_apreq2-20051231/2.6.0 mod_perl/2.0.3 Perl/v5.8.4
configured -- resuming normal operations
[Thu Nov 30 14:15:39 2006] [info] Server built: Nov 29 2006 17:32:31

Also, here is perl -V:

Summary of my perl5 (revision 5 version 8 subversion 4) configuration:
  Platform:
    osname=linux, osvers=2.6.15.4, archname=i386-linux-thread-multi
    uname='linux ninsei 2.6.15.4 #1 smp preempt mon feb 20 09:48:53 pst
2006 i686 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN
-Dcccdlflags=-fPIC -Darchname=i386-linux -Dprefix=/usr
-Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8
-Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5
-Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl/5.8.4
-Dsitearch=/usr/local/lib/perl/5.8.4 -Dman1dir=/usr/share/man/man1
-Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1
-Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl
-Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm
-Duseshrplib -Dlibperl=libperl.so.5.8.4 -Dd_dosuid -des'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define
usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS
-DDEBIAN -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN
-fno-strict-aliasing -I/usr/local/include'
    ccversion='', gccversion='3.3.5 (Debian 1:3.3.5-13)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=/lib/libc-2.3.2.so, so=so, useshrplib=true,
libperl=libperl.so.5.8.4
    gnulibc_version='2.3.2'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'


Characteristics of this binary (from libperl):
  Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES
PERL_IMPLICIT_CONTEXT
  Built under linux
  Compiled at Mar 23 2006 21:49:08
  @INC:
    /etc/perl
    /usr/local/lib/perl/5.8.4
    /usr/local/share/perl/5.8.4
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.8
    /usr/share/perl/5.8
    /usr/local/lib/site_perl
    .

Thanks again.

David