Apache memory pools and C++ objects

This is the first stable release. It can be compiled on both Linux and Windows.

Here is an excerpt of the documentation available in release 0.4, it should be useful to understand the purpose of the project

What is it

A small library that contains a STL allocator and a sample Apache module. The allocator enables C++ programmers to instatiate objects in the Apache per request memory pools. Using this allocator it is easy to develop C++ Apache modules, most of the times you can allocate objects and forget the deletes. The package works on both Linux and Windows.

Preamble

Hi, my name is Giovanni Bricconi. First of all excuse me for my poor English, I will try to write a syntax correct english, but I know I will make a lot of mistakes; so if you find some errors or incomprensible phrares please let me know and I will try to fix the problems.

Where does it come from

During November 2004 I was trying to write my first Apache module.
A collegue was writing a pretty simple module to set up some environment wariable to be shown in plain HTML pages, using the "echo" SSI directive. He was writing his module in C.
Since I usually program in Java and C++, and since I was curios to undertand a little bit on how Apache works internally, I decided to write something similar in C++.
The example module provided in this package generates random values, depending on some per directory httpd.conf parameters.

Problems

It was pretty easy to write something that works, but the surprise arrived analyzing memory consumption at runtime: since the configuration is "per directory" it is necessary to hook the "request merge configuration" phase. In this phase the apache core invokes repeadetly the merge function passing a copule of configuration structures. The module must merge these values and return an enriched configuration.
Example: the random module allows to configure different variables for different locations. The inner-most directory inherits all the variables defined in the containing locations.
The Apache core invokes this merge function repeadetly but does not tell your module when it is safe to deallocate resources after the merge phase is completed (or better I and my collegue didn't know how to hook this event).
Even if Apache woult tell when the merged values are useles you should keep track of everything you allocated to delete it, usually unfeasible.
Therefore allocating C++ objects with the plain new operator is problematic: during the merge phase some object gets allocated and are never destroyed. Sooner or later the memory will get exhausted.

Allocators

My collegue was able to avoid this problem using apache's apr_pool_t. Per request apache makes available a memory pool that will be reused in the following request. My collegue allocated all the configuration structures created in the merge phase on the pool, therefore the consumed memory is reused in the next request.
I tried to solve the problem in various ways. One solution was to use a single configuration object and to deallocate it after it is used (the random variable was produced). This approach was error prone, and cluttered the code with various test, distracting me from the real problem to solve.
Fortunately the ANSI C++ library allows the use of allocators, and pthread library offers some interesting features.
The C++ library introduces so called "allocators": object used to allocate other object customizing how the memory is used. Using allocators it is possible to implement object pooling, arenas or other memory management policies. Allocators are used in STL classes such as basic_string, list, vector...
Thefefore I decided to try to write a STL compatible allocator to allocate objects on apache pool, instead of using the plain new operator.
The dark sides of this solution are:

You should continously pass apr_pool_t pointers between allocators to instantiate object of diffent types on the same pool.
Explicitly passing this pointer clutters the code: each method that allocates object must somehow receive a pointer to the apache pool.
You should explicitly instantiate you allocators, since no default constructor can be used (you must pass the apr_pool_t pointer)

Taking advantage of pthread lib

Fortunately pthread lib came in rescue!
Each pthread has a private memory area (Thread Local Storage, TLS): you can place in this area pointer to useful data structure.
Example: programming with the .NET framework you can set per thread user locale, therefore you should not continously pass the locale reference to the funtions that formats you messages. Each thread can have a different locale, once set the locale remains the same until explicitly changed.
Since pthread provide a similar solution I decided to write the allocator default constructor so that it retrieves the apr_pool_t directly from the pthread private memory area. In this way no pointer shoul be explicitly passed in your code, apr_pool_t * is always at hand.
You should simply remember to set, at the beginning of you hook code, the received apr_pool_t as the current per thread pool.

Pay attention

If your objects contain Pool_Allocator, these instances retains a pointer to the original apr_poll_t used. Therefore if an invoked method allocates more memory, this memory would be allocated in a pool that is not the request's apr_pool_t. In the code example I have followed this approach: object instantiated in the configuration phase are never changed, new object are allocated in the request phase to be sure that they will borrow memory from the correct pool.

Windows Compatibility

The second development step was to make the pool Window compatible. The code has changed to be Microsoft Visual Studio 7 compatible. Proper code to emulate missing pthread function has been written. Now you can use the Pool_Allocator also to develop Windows C++ modules.

Note:: The sources contained in this tar were tested with gnu C++ 3.2 and 3.3, and MSVC 7. I don't know if it could work with earlier versions, or even different compilers.

Last update: 3 Mar 2005.