Re: 32 & 64 bit memory differences

["Mark Maunder" <suppressed>]

I can't overstate the efficiency gains from a reverse proxy config with mod_perl and a very light front-end reverse proxy. As I mentioned I'm using lighttpd as the front-end and apache 2.2 with mod_perl 2 as the back end. Here are two screenshots to illustrate what I'm talking about.

The first is the lighttpd status page from a very busy server of mine. The instant this page loaded, the server was serving 146 concurrent requests which is normal for this time of day:
http://markmaunder.com/light_status.gif

Most of those requests are routed via mod_proxy on lighttpd to the back-end mod_perl server. The mod_perl box is handling just under 30 requests per second with only 5 prefork processes. Usually only one of those is busy.
http://markmaunder.com/apache_status.gif

Without the front-end proxy, mod_perl might have to handle those 140+ concurrent connections with one process per connection. I have keepalive enabled on the front-end server, so this isn't a fair statement, but you see what I'm getting at. 

The memory used by each of those apache processes is around 30 Megabytes, although it's actually less because some of it is shared:
http://markmaunder.com/apache_top.gif

Both these servers are running on the same machine. I don't have any fancy load sharing gear and this works perfectly on a single box with very little hardware. I do have a small cluster, but each of my machines runs both the reverse proxy and mod_perl server. I'm handling 30 mod_perl requests per second with less than 150 Megs of memory and a not-insubstantial mod_perl app. I leave most of my 2 Gigabytes of RAM free on these boxes for filesytem caching by Linux. The app is very filesystem intensive. 

FYI, lighttpd is configured with mod_proxy and mod_rewrite enabled. Both are incredibly simple to configure under lighttpd (and incredibly inflexible), but combined with mod_rewrite on your back-end mod_perl server you can do just about any config. 

BTW, this config is common for other app servers like Ruby/Mongrel and PHP. Most app servers are memory hungry and don't like slow clients. 

Mark.

On Nov 6, 2007 10:41 AM, Gary Sewell < suppressed> wrote:

Hi again,

As I mentioned we run 4 mod perl servers, theya re all apache 1.3 , 2 are
64-bit 2 are 32-bit machines.

The discrepancies I mentioned in the size of each of our apache instances is
something I don't think I personally be able to get to the bottom of, I have
attempted Apache::Status, this gives out our loaded modules etc but not
specific sizes, so its hard to find what I taking up the 200mb. Lets put
that to the side.

I am interested in the reverse proxy idea. We currently also run a static
image/js/css server and a static php server that runs the static pages we
are able to crate which is very few, 99% of our pages are dynamic and change
every second. Due to the bulk of our code (100Mb @ 32-bit & 200Mb @ 64-bit)
we are only able to set Max Clients to 40. After arriving at a static php
homepage we refer subsequent pages randomly to one of the modperl servers to
share the stress.

Is our php server acting similar to a reverse-proxy or am I missing out on
something, would a reverse-proxy help us with our setup. We are unable to
cache content and hit live databases for every dynamic page we serve.
Images, js and css are all served from a slimmed down apache server so these
aren't a problem.

Thanks for the input.
GS

-----Original Message-----
From: Frank Wiles [mailto:suppressed ]
Sent: 06 November 2007 18:08
To: Mark Maunder
Cc: Perrin Harkins; Darryl Miles; Gary Sewell; suppressed
Subject: Re: 32 & 64 bit memory differences

On Sun, 4 Nov 2007 21:28:34 -0800
"Mark Maunder" <suppressed> wrote:

> I run lighttpd as reverse proxy in front of mod_perl configured with
> prefork. I average 100 to 200 concurrent connections on lighttpd and
> need 4 mod_perl processes with keepalive disabled to service all
> those requests. [thanks Perrin for suggesting this config!!]

 That's actually a pretty common ratio of front to backend servers.
 I *highly* recommend a reverse proxy for all but the smallest
 environments.

 -------------------------------------------------------
  Frank Wiles, Revolution Systems, LLC.
    Personal : suppressed  http://www.wiles.org
    Work     : suppressed http://www.revsys.com