Re: Memory Usage

[Jonathan Vanasco <suppressed>]

On Sep 1, 2006, at 11:15 AM, Daniel B. Hemmerich wrote:

> We are concerned about how much memory we are using now that we are  
> moving to modperl.
>
>  Are there any good tools/procedures that we could use to determine  
> how much memory our application is currently using – and then using  
> that as a benchmark, determine how much more or less memory we are  
> using after making changes to our application?
re: mp performance guide (in previous post) --

	its great for perfomance tuning.  GTOP works well for the current  
memory size.
	Iinstall Apache::Status, and all of the bells & whistles to get the  
aggregate memory info printed  ( i believe that means b::terse size  
and recdescent )
	None of the modules can accurately measure shared memory
		ignore any info re: shared memory.  just forget about it.  it never  
measures right.
		the only way to measure shared memory is to watch free/swap memory  
and start adding clients/requests, trying to find the point  where  
memory goes from free to swap ( the 'free' memory seems to be ok in  
top , but top can lie.  watching your disks  swap is a great  
indicator that you've maxxed out )

From my recent personal experience (assuming you are on *nix/*bsd,   
and using prefork)

	run the server with 1 child on start, 1 maxclients, 0 spareservers  
-- so you only have the parent and a single child
	before you request any pages, note the vitrtual and res size of each  
item using 'ps aux'

	run apache  status, and show the  main memory page
	i  copy-pasted that code into a text document then regexed and  
sorted it.
	profile your apps based on that, looking for the fattest modules.
		if a module is too big for the benefits it gives, axe it
			File::Find let me do a recursive search for templates to pre- 
compile in 8 lines of code.  but it had a 3mb memory overhead.  i  
replaced it with 15 lines of my own recursive search, that had a 4k  
overhead.
		on your own modules, replace  $debug varaiables with DEBUG  
constants so they're compiled away
			my session parsing class went from 867k to 240k of memory ( when  
DEBUG is off ),  and half as many opcodes
		
	compare the loaded modules pre- and post- request.  if you're seeing  
some modules get loaded after a request has been made, you should  
probably load them in a startup.pl

	shut down as many non-essential processes as you can on the machine  
in question.
	open a seperate terminal, run 'top -U www' or whomever your httpd  
runs as.  note the free system memory before you start apache  and after

	raise maxclients and startservers to 6, then 11 -- each time  
stopping and starting the server.  it'll give you an idea of the  
memory consumed by 5 and 10 additional clients
	make some requests with 1,6,11 clients , either using AB or some  
sort of programmatic test suite.

in my experience, using my own framework, i've got this:
	60mb  parent apache
	3mb  child on startup
	11mb child after 1 request
	+0-1.5mb per child per additional URI module (depending on page  
complexity)
	there's a slight growth per-child-per-request as well if you're  
using DBI -- DBI caches 120 SVs internally  for statement handles PER  
CHILD.  they don't get released until a counter hits 120,  then it  
goes up again.