nginx upgrade fails due bind error on 127.0.0.1 in a FreeBSD jail

[Steven Hartland]

On 05/12/2016 17:12, Maxim Dounin wrote:
> Hello!
>
> On Mon, Dec 05, 2016 at 02:40:27PM +0000, Steven Hartland wrote:
>
>> On 05/12/2016 13:27, Maxim Dounin wrote:
>>> Hello!
>>>
>>> On Sun, Dec 04, 2016 at 09:39:59PM +0000, Steven Hartland wrote:
> [...]
>
>>>> I believe the change to add a localhost bind to the server in question
>>>> was relatively recent so I suspect it has something to do with that.
>>>>
>>>> The config for this is simply:
>>>> server {
>>>>        listen 127.0.0.1:81;
>>>>        server_name localhost;
>>>>
>>>>        location /status {
>>>>            stub_status;
>>>>        }
>>>> }
>>>>
>>>> The upgrade in this case was:
>>>> nginx: 1.10.1_1,2 -> 1.10.2_2,2
>>>>
>>>> Now this server is running under FreeBSD in a jail (10.2-RELEASE) and it
>>>> has 127.0.0.1 available yet it seems nginx has incorrectly bound the
>>>> address:
>>>> netstat -na | grep LIST | grep 81
>>>> tcp4       0      0 10.10.96.146.81        *.* LISTEN
>>> In a FreeBSD jail with a single IP address any listening address
>>> is implicitly converted to the jail address.  As a result, if you
>>> write in config "127.0.0.1" - upgrade won't work, as it will see
>>> inherited socket listening on the jail address (10.10.96.146 in
>>> your case) and will try to create a new listening socket with the
>>> address from the configuration and this will fail.
>> Thanks for the response Maxim.
>>
>> In our case we don't have a single IP in the jail we have 4 addresses:
>> 1 x localhost address (127.0.0.1)
>> 2 x external
>> 1 x private address (10.10.96.146)
>>
>> We have a number of binds the externals are just port binds the internal
>> a localhost e.g.
>> listen 443 default_server accept_filter=httpready ssl;
>> listen 80 default_server accept_filter=httpready;
>> ...
>> listen 80;
>> listen 443 ssl;
>> ...
>> listen 127.0.0.1:81;
>>
>> We're expecting the none IP specified listens to bind to * (this is what
>> happens)
>>
>> But the "listen 127.0.0.1:81" results in "10.10.96.146:81" instead.
>>
>> Given your description I would only expect this 127.0.0.1 wasn't present
>> in the jail and 10.10.96.146 was the only IP available.
>>
>> Did I miss-understand your description?
> Given that the real local address of the listening socket as shown
> by netstat is 10.10.96.146, it means that the socket was created
> when there were no explicit 127.0.0.1 in the jail.
>
> And, given that you are able to connect to it via "lwp-request
> http://127.0.0.1:81/status", it looks like that 127.0.0.1 is still
> not in the jail, but mapped to 10.10.96.146 instead.
>
> Note that the fact that you can use 127.0.0.1 in a jail doesn't
> mean that it is a real address available.  Normally, 127.0.0.1
> will be implicitly converted to the main IP of the jail, and most
> utilities won't notice.
>
> (Note well that since there is no real 127.0.0.1 in the jail, it
> doesn't provide any additional isolation compared to the jail IP
> address.  That is, a service which is listening on 127.0.0.1 is in
> fact listening on 10.10.96.146, and it is reachable from anywhere,
> not just the jail itself.)
>
>>> There are two possible solutions for this problem:
>>>
>>> - configure listening on the jail IP address to avoid this
>>>     implicit conversion;
>> As above I'm not sure I follow you correctly as 127.0.0.1 is one of the
>> IP's available in the jail.
> See above, looks like it's not, and it is implicitly converted to
> 10.10.96.146 instead.
>
Sorry its taken a while, but for those that are interested I committed 
the fixes for localhost binding in FreeBSD jails today:
https://svnweb.freebsd.org/changeset/base/316313 - IPv4 fix
https://svnweb.freebsd.org/changeset/base/316328 - IPv6 fix

With these fixes "nginx upgrade" will work for configurations which use 
explicit localhost bindings e.g. (127.0.0.1) if said IP is also added to 
the jail.

     Regards
     Steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170331/416eaf6d/attachment.html>