The Drupal Diaries, Day 10

If you need help with MySQL memory management, the MySQL Admin Guide (2nd edition) has nice recipes on p. 475, specifically for dealing with "little memory and lots of connections".

FYI - Make sure you use Apache's prefork mpm, and not worker mpm. This is from the php 5.2.x INSTALL file:

"We do not recommend using a threaded MPM in production with Apache2.
Use the prefork MPM instead, or use Apache1. For information on why,
read the related FAQ entry on using Apache2 with a threaded MPM."

You're looking at it naively -- the memory usage for each httpd process includes any shared libraries and other memory that shared (copy-on-write) with other httpd processes. Marginal overhead is much lower in most cases.

I agree, I am probably looking at it naively, as I've never been a Unix systems programmer, and I don't fully understand memory usage. The only thing I can go by is what I see with the top command, and my understanding of those results.

For instance, here's a subset of the current top command output related to httpd processes:

Mem:  20723928k total,   157496k used, 20566432k free,        0k buffers
Swap:        0k total,        0k used,        0k free,        0k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                             
16893 nobody    15   0 29636  17m 2684 S  0.0  0.9   0:00.52 httpd
16879 nobody    15   0 29060  17m 2700 S  0.0  0.9   0:00.42 httpd
16732 nobody    17   0 28876  17m 2700 S  0.0  0.8   0:01.22 httpd
16891 nobody    15   0 28788  16m 2700 S  0.0  0.8   0:00.51 httpd
16882 nobody    15   0 28596  16m 2712 S  0.0  0.8   0:00.51 httpd                                
16854 nobody    15   0 28372  16m 2716 S  0.0  0.8   0:00.56 httpd
16701 nobody    15   0 28292  16m 2704 S  0.0  0.8   0:00.92 httpd
16775 nobody    15   0 28108  16m 2700 S  0.0  0.8   0:00.46 httpd
16773 nobody    22   0 28020  16m 2716 S  0.0  0.8   0:00.56 httpd
16876 nobody    15   0 20160 8648 2628 S  0.0  0.4   0:00.04 httpd

The first two lines of this output are a little misleading, because this site was on a shared, virtual server, and there was really only 256 MB RAM available to me. I was supposed to be allowed "bursts" up to 1 GB, but I never saw RAM usage anywhere near that high; the system would always bog down before getting near that peak. I don't know if that was because of the throttling mechanism in place, or something else. As a practical matter, whenever RAM usage exceeded 300-350 MB, the system became non-responsive, and the best thing to do was restart Apache.

After the first two lines of output, all the remaining lines are related to httpd processes. The three columns I looked at were RES, SHR, and %MEM. I don't know how else to interpret these other than the output from the top man page, which looks like this:

RES   Resident size (kb)
      The non-swapped physical memory a task has used.
      RES = CODE + DATA.

SHR   Shared Mem size (kb)
      The amount of shared memory used by a task.
      It simply reflects memory that could be potentially 
      shared with other processes.

%MEM  Memory usage (RES)
      A task’s currently used share of available physical memory.

In summary, when I first started down this road, I would typically see many Apache httpd processes showing 16 or 17 MB in the RES column, with other smaller entries like the last one shown above, with 8648 in the RES column. And the "memory used" data above the columnar top output exceeded 256 MB RAM.

By tweaking Apache's settings as described in this series, I was able to get more of these smaller RAM processes, and fewer of the 17 MB entries. As far as I could tell, these tweaks kept memory use down, and kept the server alive longer than the initial Apache settings.

The part I don't fully understand is the 14-15 MB difference between RES and SHR for most of those httpd processes. For instance, I think the first row is saying that 17 MB RAM is used by this process, and 2,684 KB of this may be shared with other processes. If this output is saying that 2,684 KB is shared, that's nice to know, but as a sysadmin whose server appears to be running out of memory, my only real concern is whether this process -- and the processes below it -- are all taking 16-17 MB RAM.

Sorry for this lengthy reply, but thanks for your feedback.

With your memory so constrained, it makes sense to minimize the amount of memory apache and php are using for any request. But another, parallel approach is to minimize the actual number of hits per page request, by off-loading static image and/or file requests to a static file server.

Put another way - why waste all that memory on requests that don't need it - you may find that up to 75% of your traffic is this kind of static file request.

Start with http://drupal.org/project/cdn and then google for drupal and static file servers. Amazon S3 is ridiculously cheap, and even if you just move your static theme images onto it, you'll save yourself a bunch of hits on your drupal server.

Thank you, those are good suggestions. As you mention, they won't bring the memory use down, but they will help take the load off the server. In my case I have a couple of almost-free domains I can use to serve images from instead of something like S3, but we did use S3 at my former company. Thanks again.