Help with shared memory usage

[Maxim Dounin]

Hello!

On Fri, Jun 28, 2013 at 10:36:39PM -0300, Wandenberg Peixoto wrote:

> Hi,
> 
> I'm trying to understand how the shared memory pool works inside the Nginx.
> To do that, I made a very small module which create a shared memory zone
> with 2097152 bytes,
> and allocating and freeing blocks of memory, starting from 0 and increasing
> by 1kb until the allocation fails.
> 
> The strange parts to me were:
> - the maximum block I could allocate was 128000 bytes
> - each time the allocation fails, I started again from 0, but the maximum
> allocated block changed with the following profile
> 128000
> 87040
> 70656
> 62464
> 58368
> 54272
> 50176
> 46080
> 41984
> 37888
> 33792
> 29696
> 
> This is the expected behavior?
> Can anyone help me explaining how shared memory works?
> I have another module which do an intensive shared memory usage, and
> understanding this can help me improve it solving some "no memory" messages.
> 
> I put the code in attach.

I've looked into this, and the behaviour is expected as per 
nginx slab allocator code and the way you do allocations in your 
test.

Increasing allocations of large blocks immediately followed by 
freeing them result in free memory blocks split into smaller 
blocks, eventually resulting in at most page size allocations 
being possible.  Take a look at ngx_slab_alloc_pages() and 
ngx_slab_free_pages() for details.

Note that slab allocator nginx uses for allocations in shared 
memory is designed mostly for small allocations.  It works well 
for allocations less than page size, but large allocations support 
is very simple.  Probably it should be improved, but as of now 
nothing in nginx uses large allocations in shared memory.

-- 
Maxim Dounin
http://nginx.org/en/donation.html