Strange Unit failure

[Peter TKATCHENKO]

Thanks for your answer, Andrew.


>> External access to the application is proxied by HAProxy with sticky
>> sessions configured. Direct access to NGINX servers is possible too
>> (with another URLs).
> You mention proxies. Are you using the "proxy" setting in your config?

I have

proxy_pass http://unit_backend$request_uri;

in NGINX config, that's all. Nothing in Unit config.



>> Yesterday we began to (randomly) get 503 errors on different pages of
>> the application. The same page could be OK for one user, but failed
>> with 503 error for other user. The source of these errors was Unit. We
> Are the error cases using WebSockets?

Non, we don't use WebSockets.


>> could isolate the problems and solved them restarting Unit service on
>> 2 servers (of 3). We did not find any useful information in Unit logs.
>> Nothing interesting in NGINX logs neither.
>>
>> I suppose that some Unit workers were in strange semi-failured state,
>> but the master process did not kill them and continued to send them
>> the requests.
>>
>> Any ideas?
> Could you confirm if you are or aren't seeing actual crashes in the unit
> processes?
>
> Do you see process exited on signal 11 (or maybe 6) messages in the unit
> log for example or core dumps being generated?

Yes, I see many logs with signal 11 just after several writev failed:

2023/06/19 16:52:06 [info] 56492#102962 *39777 writev(21, 3) failed (32: 
Broken pipe)
2023/06/19 16:52:22 [info] 56492#102962 *39738 writev(22, 3) failed (32: 
Broken pipe)
2023/06/19 16:52:46 [info] 56492#102968 *39753 writev(30, 3) failed (32: 
Broken pipe)
2023/06/19 16:53:10 [alert] 56493#101705 app process 11701 exited on 
signal 11
2023/06/19 16:53:11 [alert] 56493#101705 app process 84631 exited on 
signal 11
2023/06/19 16:53:11 [alert] 56493#101705 app process 98305 exited on 
signal 11

I don't think Unitd worker can write his dumps somewhere as it is 
started as www user, so hi has no write permissions.


Peter