nginx performance on Amazon EC2

Thu Feb 21 07:29:43 UTC 2013

hello, 

is the setup of your vmware similar to you ec2 - instance? i talk esp. about
RAM/CPU-power
here. 

do you have a monitoring on your instances, checking for load/ram-zusage,
iowait etc? 

maybe you should start your ec-test with less than 500 concurrent
connections and work up 
to the point that instance starts to fail. 

> [error] 4167#0: *27229 connect() to unix:///tmp/pyapp.socket failed
> (11:
> Resource temporarily unavailable) while connecting to upstream,
> client:
> 127.0.0.1, server: mysite.com

looks like your django-app shuts down or isnt capable of handling that
ammount of connections.

oh, and you shouldnt expect a distant instance to have the same performance
as a 
machine in your local net. 

Rakan Alhneiti Wrote:
-------------------------------------------------------
> Hello,
> 
> Yes my vm machine is working on my local network. I am referring to it
> rather so show that it performs better & no issues appear there.
> I tried both Amazon EC2 small instance & a linode 2048 instance and
> both
> give the exact same result as well.
> 
> When doing apache benchmark, i can see the following in my nginx error
> log:
> 
> [error] 4167#0: *27229 connect() to unix:///tmp/pyapp.socket failed
> (11:
> Resource temporarily unavailable) while connecting to upstream,
> client:
> 127.0.0.1, server: mysite.com
> 
> and
> 
> upstream prematurely closed connection while reading response header
> from
> upstream, client: 127.0.0.1, server: mysite.com
> 
> Other than that, there's nothing in django error log but here's what i
> can
> see in uwsgi's daemon log:
> 
> Wed Feb 20 21:59:51 2013 - writev(): Broken pipe [proto/uwsgi.c line
> 124]
> during GET /api/nodes/mostviewed/9/?format=json (127.0.0.1)
> [pid: 4112|app: 0|req: 34/644] 127.0.0.1 () {30 vars in 415 bytes}
> [Wed Feb
> 20 21:59:42 2013] GET /api/nodes/mostviewed/9/?format=json =>
> generated 0
> bytes in 8904 msecs (HTTP/1.0 200) 3 headers in 0 bytes (0 switches on
> core
> 0)
> Wed Feb 20 21:59:51 2013 - writev(): Broken pipe [proto/uwsgi.c line
> 124]
> during GET /api/nodes/mostviewed/9/?format=json (127.0.0.1)
> [pid: 4117|app: 0|req: 1/645] 127.0.0.1 () {30 vars in 415 bytes} [Wed
> Feb
> 20 21:59:46 2013] GET /api/nodes/mostviewed/9/?format=json =>
> generated 0
> bytes in 5021 msecs (HTTP/1.0 200) 3 headers in 0 bytes (0 switches on
> core
> 0)
> 
> and stuff like:
> Wed Feb 20 20:01:01 2013 - uWSGI worker 1 screams: UAAAAAAH my master
> disconnected: i will kill myself !!!
> 
> What do you guys think?
> 
> Thanks alot
> 
> 
> 
> Best Regards,
> 
> *Rakan AlHneiti*
> Find me on the internet:
>  Rakan Alhneiti <http://www.facebook.com/rakan.alhneiti> |
> @rakanalh<https://twitter.com/rakanalh>
>  |  Rakan Alhneiti <http://www.linkedin.com/in/rakanalhneiti> |  
> alhneiti
> 
> ----- GTalk rakan.alhneiti at gmail.com
> ----- Mobile: +962-798-910 990
> 
> 
> 
> On Thu, Feb 21, 2013 at 7:33 AM, David | StyleFlare
> <david at styleflare.com>wrote:
> 
> >  Really you would have to do this test on 2 amazon servers and then
> see
> > if one was more performant.
> > Then you can assume something is wrong.
> >
> > Based on the configs everything looks right.
> >
> > The fact that your vmware server is better performing, is really not
> > saying much. Its really hard to directly compare.
> >
> > I would presume so many other factors.
> >
> > Is the vmware box on a local network?
> >
> >
> >
> >
> >
> > On 2/20/13 11:05 PM, Anoop Alias wrote:
> >
> >
> >
> > On Thu, Feb 21, 2013 at 1:43 AM, Rakan Alhneiti
> <rakan.alhneiti at gmail.com>wrote:
> >
> >> Hello,
> >>
> >>   I am running a django app with nginx & uwsgi on an amazon ec2
> instance
> >> and a vmware machine almost the same size as the ec2 one. Here's
> how i run
> >> uwsgi:
> >>
> >> sudo uwsgi -b 25000 --chdir=/www/python/apps/pyapp
> --module=wsgi:application --env DJANGO_SETTINGS_MODULE=settings
> --socket=/tmp/pyapp
> >>  .socket --cheaper=8 --processes=16  --harakiri=10 
> --max-requests=5000  --vacuum --master --pidfile=/tmp/pyapp-master.pid
> --uid=220 --gid=499
> >>
> >> & nginx configurations:
> >>
> >> server {
> >>     listen 80;
> >>     server_name test.com
> >>
> >>     root /www/python/apps/pyapp/;
> >>
> >>     access_log /var/log/nginx/test.com.access.log;
> >>     error_log /var/log/nginx/test.com.error.log;
> >>
> >>     #
> https://docs.djangoproject.com/en/dev/howto/static-files/#serving-stat
> ic-files-in-production
> >>     location /static/ {
> >>         alias /www/python/apps/pyapp/static/;
> >>         expires 30d;
> >>     }
> >>
> >>     location /media/ {
> >>         alias /www/python/apps/pyapp/media/;
> >>         expires 30d;
> >>     }
> >>
> >>     location / {
> >>         uwsgi_pass unix:///tmp/pyapp.socket;
> >>         include uwsgi_params;
> >>         proxy_read_timeout 120;
> >>     }
> >>
> >>     # what to serve if upstream is not available or crashes
> >>     #error_page 500 502 503 504 /media/50x.html;}
> >>
> >> Here comes the problem. When doing "ab" (ApacheBenchmark) on both
> >> machines i get the following results: (vmware machine being almost
> the same
> >> size as the ec2 small instance)
> >>
> >> *Amazon EC2:*
> >>
> >> nginx version: nginx version: nginx/1.2.6
> >>
> >> uwsgi version:1.4.5
> >>
> >> Concurrency Level:      500Time taken for tests:   21.954
> secondsComplete requests:      5000Failed requests:        126
> >>    (Connect: 0, Receive: 0, Length: 12
> >>  6, Exceptions: 0)Write errors:           0Non-2xx responses:     
> 4874Total transferred:      4142182 bytes
> >> HTML transferred:       3384914 bytesRequests per second:    227.75
> [#/sec] (mean)Time per request:       2195.384 [ms] (mean)Time per
> request:       4.391 [ms] (mean, across all concurrent
> requests)Transfer rate:          184.25 [Kbytes/sec] received
> >>
> >> *Vmware machine (CentOS 6):*
> >>
> >> nginx version: nnginx version: nginx/1.0.15
> >>
> >> uwsgi version: 1.4.5
> >>
> >> Concurrency Level:      1000Time taken for tests:   1.094
> secondsComplete requests:      5000Failed requests:        0Write
> errors:           0Total transferred:      30190000 bytes
> >> HTML transferred:       28930000 bytesRequests per second:   
> 4568.73 [#/sec] (mean)Time per request:       218.879 [ms] (mean)Time
> per request:       0.219 [ms] (mean, across all concurrent
> requests)Transfer rate:          26939.42 [Kbytes/sec] received
> >>
> >> As you can see... all requests on the ec2 instance fail with either
> >> timeout errors or "Client prematurely disconnected". However, on my
> vmware
> >> machine all requests go through with no problems. The other thing
> is the
> >> difference in reqs / second i am doing on both machines.
> >>
> >> What am i doing wrong on ec2?
> >>
> >>
> >> _______________________________________________
> >> nginx mailing list
> >> nginx at nginx.org
> >> http://mailman.nginx.org/mailman/listinfo/nginx
> >>
> >
> >
> > Is there any data in the nginX error logs?
> >
> >
> >  --
> > *Anoop P Alias*
> > GNUSYS <http://gnusys.net>
> >
> >
> > _______________________________________________
> > nginx mailing
> listnginx at nginx.orghttp://mailman.nginx.org/mailman/listinfo/nginx
> >
> >
> >
> > _______________________________________________
> > nginx mailing list
> > nginx at nginx.org
> > http://mailman.nginx.org/mailman/listinfo/nginx
> >
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

Posted at Nginx Forum: http://forum.nginx.org/read.php?2,236391,236408#msg-236408