memory leak in django app?

[Etienne Robillard]

Hi Antonis,

My development server appears unaffected by this problem. Plus I can get 
heap stats using guppy, which is pretty cool. :)

Both my production and development servers have __debug__ enabled in 
Python 2.7.13.

However when using ab -c 100 to benchmark my nginx server I get:

SSL handshake failed (1).

Do you have any ideas how to configure nginx to allow a minimum of 100 
concurrent connections when using SSL encryption?

I have worker_connections set to 512 in my nginx.conf

Regards,

Etienne


Le 2017-12-06 à 12:17, Antonis Christofides a écrit :
> Does this happen only in production? What about when you run a development
> server? What is the memory usage of your development server?
>
> Antonis Christofides
> http://djangodeployment.com
>
>
> On 2017-12-06 15:05, Etienne Robillard wrote:
>> Hi Antonis,
>>
>> Thank you for your reply. I installed the htop utility and found that 2 of my
>> 4 uWSGI processes are using 882M (42.7%) of resident memory each. Theses two
>> processes takes about  (85%) of the available RAM memory! That can explain why
>> I get "out of memory" errors when no more memory is available for sshd.
>>
>> I tried to debug memory allocation with guppy following instructions here:
>> https://www.toofishes.net/blog/using-guppy-debug-django-memory-leaks/
>>
>> But I get "hp.Nothing" when I attempt to get the heap stats from the master
>> uWSGI process.
>>
>> Example:
>>>>> hp.setref()
>>>>> hp.heap()
>> hp.Nothing
>>
>> Any help would be appreciated!
>>
>> Etienne
>>
>>
>> Le 2017-12-06 à 07:53, Antonis Christofides a écrit :
>>> Hello,
>>>
>>> the amount of memory you need depends on what Django does and how many workers
>>> (instances of Django) you run (which usually depends on how many requests you
>>> are getting and how I/O intensive your Django application is). For many
>>> applications, 512 MB is enough.
>>>
>>> Why are you worried? The only symptom you describe is that your free memory is
>>> decreasing. This is absolutely normal. The operating system doesn't like RAM
>>> that is sitting down doing nothing, so it will do its best to make free RAM
>>> nearly zero. Whenever there's much RAM available, it uses more for its caches.
>>>
>>> How much memory is your Django app consuming? You can find out by executing
>>> "top" and pressing "M" to sort by memory usage.
>>>
>>> Regards,
>>>
>>> Antonis
>>>
>>> Antonis Christofides
>>> http://djangodeployment.com
>>>
>>> On 2017-12-06 14:04, Etienne Robillard wrote:
>>>> Hi all,
>>>>
>>>> I'm struggling to understand how django/python may allocate and unallocate
>>>> memory when used with uWSGI.
>>>>
>>>> I have a Debian system running Python 2.7 and uwsgi with 2GB of RAM and 2 CPUs.
>>>>
>>>> Is that enough RAM memory for a uWSGI/Gevent based WSGI app running Django?
>>>>
>>>> I'm running uWSGI with the --gevent switch in order to allow cooperative
>>>> multithreading, but my free RAM memory is always decreasing when nginx is
>>>> running.
>>>>
>>>> How can I debug memory allocation in a Django/uWSGI app?
>>>>
>>>> I defined also in my sitecustomize.py "gc.enable()" to allow garbage
>>>> collection, but it does not appears to make any differences.
>>>>
>>>> Can you recommend any libraries to debug/profile memory allocation in Python
>>>> 2.7 ?
>>>>
>>>> Is Django more memory efficient with --pymalloc or by using the default linux
>>>> malloc() ?
>>>>
>>>> Thank you in advance,
>>>>
>>>> Etienne
>>>>

-- 
Etienne Robillard
tkadm30 at yandex.com
https://www.isotopesoftware.ca/