Re: global vs multi-process

["David Willams" <suppressed>]

Perrin - Thank you for your excellent, detailed response and experience.  I did not consider your proxy scenario and I will probably revisit both it and your Berkley scenario a bit later.

Obviously, I'm not involved with apache internals, but are you saying it's architecturally impossible for apache to somehow manage a hash that can, at the same time, be available (read-only) to child processes?

- D

On 9/5/07, Perrin Harkins <suppressed> wrote:

On 9/6/07, David Willams <suppressed> wrote:
> When the translation happens on a random, ad hoc string of sometimes
> thousands of "words",  the process simply does something like this:
>
> Untranslated:
>   hfj kei hty ... jan oej wio
>
> Translated:
>   $CACHE{hfj} $CACHE{kei} $CACHE{hty} ... $CACHE{jan} $CACHE{oej}
> $CACHE{wio}

Well, there's nothing that can come even close to the speed of an
in-process perl hash, so if you really need it be that fast, you will
have to reload the hash when the data changes.  There are various
strategies for this: restart your server periodically (you can use
proxy tricks to avoid downtime), load a local hash of just the stuff
changed recently in each process and use it as an overlay (two lookups
will still be much faster than any out-of-process storage).

If you don't need quite that much speed, a simple read-only approach
with BerkeleyDB or SDBM file will still be very fast.  You can write
out the files from a cron job every few minutes with sequentially
numbered names, and the mod_perl processes can check periodically to
make sure they're reading the latest one.  Since the mod_perl
processes only read it, they can open it and just keep the handle open
in a global.

Don't use either of the Cache::* modules I mentioned unless you don't
mind doing a secondary lookup in your RDBMS for keys that aren't found
in the cache.  They both silently drop data when they run out of
space.

- Perrin