Help with shared memory usage

[Wandenberg Peixoto]

Hello!

I see your point, and I will split the patch to do both actions,
on ngx_slab_free_pages() and on allocation when has a failure.

What would be an alternative to not loop on pool->pages?

Regards,
Wandenberg



On Mon, Jul 29, 2013 at 2:11 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Sat, Jul 27, 2013 at 04:10:51PM -0300, Wandenberg Peixoto wrote:
>
> > Hello Maxim.
> >
> > I've been looking into those functions and guided by your comments
> > made the following patch to merge continuous block of memory.
> > Can you check if it is ok?
> > Comments are welcome.
> >
> > --- src/core/ngx_slab.c    2013-05-06 07:27:10.000000000 -0300
> > +++ src/core/ngx_slab.c    2013-07-27 15:54:55.316995223 -0300
> > @@ -687,6 +687,25 @@ ngx_slab_free_pages(ngx_slab_pool_t *poo
> >      page->next->prev = (uintptr_t) page;
> >
> >      pool->free.next = page;
> > +
> > +    for (page = pool->pages; ((page->slab > 0) && (&page[page->slab] <
> > (ngx_slab_page_t *) (pool->start - sizeof(ngx_slab_page_t))));) {
> > +        ngx_slab_page_t *neighbour = &page[page->slab];
> > +        if (((ngx_slab_page_t *) page->prev != NULL) && (page->next !=
> > NULL) && ((page->prev & NGX_SLAB_PAGE_MASK) == NGX_SLAB_PAGE) &&
> > +            ((ngx_slab_page_t *) neighbour->prev != NULL) &&
> > (neighbour->next != NULL) && ((neighbour->prev & NGX_SLAB_PAGE_MASK) ==
> > NGX_SLAB_PAGE)) {
> > +
> > +            page->slab += neighbour->slab;
> > +
> > +            ((ngx_slab_page_t *) neighbour->prev)->next =
> neighbour->next;
> > +            neighbour->next->prev = neighbour->prev;
> > +
> > +            neighbour->slab = NGX_SLAB_PAGE_FREE;
> > +            neighbour->prev = (uintptr_t) &pool->free;
> > +            neighbour->next = &pool->free;
> > +            continue;
> > +        }
> > +
> > +        page += ((page->prev & NGX_SLAB_PAGE_MASK) == NGX_SLAB_PAGE) ?
> > page->slab : 1;
> > +    }
> >  }
>
> The patch doesn't look right (well, may be it works - but at least it's
> not something I would like to see committed).
>
> The pool->pages isn't something you should iterate though, it's
> just a preallocated storage space for ngx_slab_page_t structures.
>
> Additionally, doing a full merge of all free blocks on a free
> operation looks too much.  It might be something we want to do on
> allocation failure, but not on a normal path in
> ngx_slab_free_pages().  And/or something lightweight may be done
> in ngx_slab_free_pages(), e.g., checking if pages following pages
> we are freeing are free too, and merging them in this case.
>
> >
> >
> >
> > Regards,
> > Wandenberg
> >
> >
> > On Mon, Jul 1, 2013 at 8:36 AM, Maxim Dounin <mdounin at mdounin.ru> wrote:
> >
> > > Hello!
> > >
> > > On Fri, Jun 28, 2013 at 10:36:39PM -0300, Wandenberg Peixoto wrote:
> > >
> > > > Hi,
> > > >
> > > > I'm trying to understand how the shared memory pool works inside the
> > > Nginx.
> > > > To do that, I made a very small module which create a shared memory
> zone
> > > > with 2097152 bytes,
> > > > and allocating and freeing blocks of memory, starting from 0 and
> > > increasing
> > > > by 1kb until the allocation fails.
> > > >
> > > > The strange parts to me were:
> > > > - the maximum block I could allocate was 128000 bytes
> > > > - each time the allocation fails, I started again from 0, but the
> maximum
> > > > allocated block changed with the following profile
> > > > 128000
> > > > 87040
> > > > 70656
> > > > 62464
> > > > 58368
> > > > 54272
> > > > 50176
> > > > 46080
> > > > 41984
> > > > 37888
> > > > 33792
> > > > 29696
> > > >
> > > > This is the expected behavior?
> > > > Can anyone help me explaining how shared memory works?
> > > > I have another module which do an intensive shared memory usage, and
> > > > understanding this can help me improve it solving some "no memory"
> > > messages.
> > > >
> > > > I put the code in attach.
> > >
> > > I've looked into this, and the behaviour is expected as per
> > > nginx slab allocator code and the way you do allocations in your
> > > test.
> > >
> > > Increasing allocations of large blocks immediately followed by
> > > freeing them result in free memory blocks split into smaller
> > > blocks, eventually resulting in at most page size allocations
> > > being possible.  Take a look at ngx_slab_alloc_pages() and
> > > ngx_slab_free_pages() for details.
> > >
> > > Note that slab allocator nginx uses for allocations in shared
> > > memory is designed mostly for small allocations.  It works well
> > > for allocations less than page size, but large allocations support
> > > is very simple.  Probably it should be improved, but as of now
> > > nothing in nginx uses large allocations in shared memory.
> > >
> > > --
> > > Maxim Dounin
> > > http://nginx.org/en/donation.html
> > >
> > > _______________________________________________
> > > nginx-devel mailing list
> > > nginx-devel at nginx.org
> > > http://mailman.nginx.org/mailman/listinfo/nginx-devel
> > >
>
>
> > _______________________________________________
> > nginx-devel mailing list
> > nginx-devel at nginx.org
> > http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
>
> --
> Maxim Dounin
> http://nginx.org/en/donation.html
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20130729/8b95375d/attachment-0001.html>