nginx serving corrupt images

Dan Swaney justdan23 at gmail.com
Fri Feb 24 02:32:54 UTC 2023


I tried to reproduce it with a simple web page and I'm seeing the issue
which I believe you are having.

By default, NGINX seems to default to some limit of 15k on responses of
images returned directly.

While the error.log in debug mode shows it constructs the response with the
right content length, it fails to return a response if the file is over 15k.

I'm using a build I did from NGINX 1.23.3.  While I know I have returned
files larger than this in proxy mode.  This seems to be happening in normal
web server page mode.

[image: image.png]


I even tried to increase it by adding:

        location / {
>             root   html;
>             index  index.html index.htm favicon.ico;
>
>             types {
>                text/html  html;
>                image/gif  gif;
>                application/octetstream jpg;
>                application/octetstream png;
>             }
>
>             set $max_image_size 50000;
>             client_max_body_size 50000;
>         }
>

On Thu, Feb 23, 2023 at 8:09 PM Dan Swaney <justdan23 at gmail.com> wrote:

> Try this:
>
>    1. At the very top of your nginx.conf file, add a definition:
>
>            error_log  logs/error.log debug;
>
>    2. Within your nginx install directory (where nginx.exe exists),
>    create a 'logs' folder if one does not exist.
>    3. Restart nginx.exe (if running as a Windows Service with NSSM, then
>    restart the service).
>    4. If NGINX does not start...
>       - Check the 'error.log' as it will tell you if your nginx.conf has
>       something weird in it that it doesn't like.
>    5. If NGINX started successfully...
>       - Run your test from your client browser.
>       - Go back to the Server and check your 'error.log' and look for the
>       URL request.
>       - It will show you everything it does with routing or looking for
>       files.
>
>
> On Thu, Feb 23, 2023 at 7:22 PM Saint Michael <venefax at gmail.com> wrote:
>
>> Now it remains broken and I have no idea how to fix it. I guess is
>> caching bad copies of the pictures.
>> I re-uploaded the two images.
>> Kindly let me know if this is "normal"
>> [image: image.png]
>>
>> On Thu, Feb 23, 2023 at 7:08 PM Dan Swaney <justdan23 at gmail.com> wrote:
>>
>>> Now your image001.jpg is returning your home page:
>>> [image: image.png]
>>>
>>> If I had to guess, your location routing is re-routing all sub-urls to
>>> your base URL at '/' which is defaulting to your index.html (or whatever
>>> you have set for your default).
>>> The tip was when it returns the content type as 'text/html' instead of
>>> 'image/jpg'.
>>>
>>> As Maxim cited, your 'location' directives are for routing URL paths and
>>> not files.  Think of the external viewed URL and the internal routed URL
>>> location.
>>>
>>> Looking at your previous cited partial nginx.conf:
>>>
>>>    - root /static/duc/;
>>>       - I usually define this in my 'location' base section only and
>>>       drop the initial '/' if it is relative to where NGINX is running.
>>>          - I then don't need it in any other 'location' sections for
>>>          sub-folders which have different security.
>>>       - You have it defined in your 'server' section.
>>>
>>>
>>>    - default_type  'text/html; charset=UTF-8';
>>>       - I usually define this at the top of my 'http' section.
>>>          - Normally I use 'default_type  application/octet-stream;'
>>>       - You have it defined in your 'server' section.
>>>       - I see it returning your images as 'text/html'.
>>>
>>>       - try_files $uri $uri/ /index.html;
>>>    - I usually define a default 'index' if at the root and nothing else
>>>       is added.
>>>          - For example: 'index index.html;'
>>>       - Remove the try_files like recommended earlier.
>>>       - If you need to restrict access to specific folder URL mappings,
>>>       then define a location of the URL mapping and add one line...
>>>          - 'deny all;'
>>>          - Otherwise leave it open to all sub URLs by doing nothing
>>>          more but use a single 'location /' rule.
>>>
>>>
>>> On Thu, Feb 23, 2023 at 6:15 PM Dan Swaney <justdan23 at gmail.com> wrote:
>>>
>>>> I just ran your test and it works fine from Chrome and in Visual Studio
>>>> Code:
>>>>
>>>> > wget https://x3x.us/index_files/image001.jpg
>>>>> StatusCode        : 200
>>>>> StatusCode        : 200
>>>>> StatusDescription : OK
>>>>> Content           : {255, 216, 255, 224...}
>>>>> RawContent        : HTTP/1.1 200 OK
>>>>>                     Connection: keep-alive
>>>>>                     Strict-Transport-Security: max-age=31536000;
>>>>> includeSubDomains
>>>>>                     Accept-Ranges: bytes
>>>>>                     Content-Length: 8834
>>>>>                     Content-Type: image/jpeg
>>>>>                     Date: Thu, 23 Feb 2023 23...
>>>>> Headers           : {[Connection, keep-alive],
>>>>> [Strict-Transport-Security, max-age=31536000; includeSubDomains],
>>>>> [Accept-Ranges, bytes],
>>>>>                     [Content-Length, 8834]...}
>>>>> RawContentLength  : 8834
>>>>
>>>>
>>>> [image: image.png]
>>>>
>>>>
>>>>
>>>> On Wed, Feb 22, 2023 at 7:36 PM Saint Michael <venefax at gmail.com>
>>>> wrote:
>>>>
>>>>> a) The error does not have a single line.
>>>>> b) restarting does not fix it
>>>>> c) my nginx is no acting as proxy
>>>>> d) it happened twice and both times I fixed it by turning gzip off,
>>>>> restarting, and back on.
>>>>> e) I also noticed that I requested the image file with wget, get a
>>>>> full HTML file for the whole document, but named as if it were the image
>>>>> file.
>>>>>
>>>>> wget https://x3x.us/index_files/image001.jpg
>>>>> but `stat image001.jpg' showed it was the entire text HTML file.
>>>>>
>>>>> http {
>>>>> client_max_body_size 1500M;
>>>>>     include       mime.types;
>>>>>    # default_type  application/octet-stream;
>>>>>
>>>>>     #log_format  main  '$remote_addr - $remote_user [$time_local]
>>>>> "$request" '
>>>>>     #                  '$status $body_bytes_sent "$http_referer" '
>>>>>     #                  '"$http_user_agent" "$http_x_forwarded_for"';
>>>>>
>>>>>     #access_log  logs/access.log  main;
>>>>>     sendfile        on;
>>>>>     tcp_nopush     on;
>>>>>     tcp_nodelay     on;
>>>>> gzip on;
>>>>> gzip_vary on;
>>>>> gzip_min_length 10240;
>>>>> gzip_proxied expired no-cache no-store private auth;
>>>>> gzip_types text/plain text/css text/xml text/javascript
>>>>> application/x-javascript application/xml;
>>>>> gzip_disable "MSIE [1-6]\.";
>>>>>     #keepalive_timeout  0;
>>>>>     keepalive_timeout  65;
>>>>>  types_hash_max_size 2048;
>>>>> proxy_headers_hash_max_size 1024;
>>>>> proxy_headers_hash_bucket_size 128;
>>>>>         #gzip  on;
>>>>>     #    error_log  /var/log/nginx/error.log debug;
>>>>>
>>>>> error_log /var/log/nginx/error.log error;
>>>>> error_log /var/log/nginx/error.log crit;
>>>>>
>>>>>         access_log  /var/log/nginx/access.log;
>>>>>
>>>>> server {
>>>>>         add_header Strict-Transport-Security "max-age=31536000;
>>>>> includeSubDomains" always;
>>>>>         default_type  'text/html; charset=UTF-8';
>>>>>     listen 208.78.161.6:80;
>>>>>     server_name x3x.us;
>>>>>
>>>>>     # Redirect all domains to https://x3x.us
>>>>>     if ($scheme != "https") {
>>>>>         return 301 https://x3x.us$request_uri;
>>>>>     }
>>>>> }
>>>>>
>>>>> server {
>>>>>        add_header Strict-Transport-Security "max-age=31536000;
>>>>> includeSubDomains" always;
>>>>>         default_type  'text/html; charset=UTF-8';
>>>>>     listen 208.78.161.6:443 ssl;
>>>>>     server_name x3x.us;
>>>>>
>>>>>     ssl_certificate /etc/letsencrypt/live/x3x.us/fullchain.pem;
>>>>>     ssl_certificate_key /etc/letsencrypt/live/x3x.us/privkey.pem;
>>>>>
>>>>> root /static/duc/;
>>>>>
>>>>>         location / {
>>>>>
>>>>> try_files $uri $uri/ /index.html;
>>>>>
>>>>>
>>>>> }
>>>>>
>>>>> } #server
>>>>>
>>>>> } #http
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Feb 22, 2023 at 7:17 PM Maxim Dounin <mdounin at mdounin.ru>
>>>>> wrote:
>>>>>
>>>>>> Hello!
>>>>>>
>>>>>> On Wed, Feb 22, 2023 at 02:46:29PM -0500, Saint Michael wrote:
>>>>>>
>>>>>> > It's not a misconfiguration, is a huge bug.
>>>>>> > A wasted two days of sleep for something that is 100% a bug.
>>>>>> > Please read here:
>>>>>> >
>>>>>> https://laracasts.com/discuss/channels/general-discussion/homestead-nginx-serving-wrong-images-and-only-cut-in-the-middle
>>>>>> > He mentions the same exact problem and also he points to
>>>>>> >
>>>>>> https://tech.blog.aknin.name/2011/11/04/nginxgzip-module-might-silently-corrupt-data-upon-backend-failure/
>>>>>> > where the author says that Niginx will not fix it.
>>>>>> > So he already tried he was rebuffed.
>>>>>>
>>>>>> The fun fact is that the referenced article doesn't state "will
>>>>>> not fix", but rather "not a top priority".  Further, proper error
>>>>>> propagation is available in nginx for about 10 years now, since
>>>>>> 2013 (http://hg.nginx.org/nginx/rev/d3eab5e2df5f, nginx 1.5.3).
>>>>>> Quoting CHANGES:
>>>>>>
>>>>>>     *) Change: now after receiving an incomplete response from a
>>>>>> backend
>>>>>>        server nginx tries to send an available part of the response
>>>>>> to a
>>>>>>        client, and then closes client connection.
>>>>>>
>>>>>> As long as nginx have an information about an error, it will
>>>>>> preserve this information and propagate it to the client.
>>>>>>
>>>>>> Also note that it is only expected to make a difference if you are
>>>>>> using nginx as a proxy, not to directly serve files.  And only in
>>>>>> case of errors.  That is, if you are seeing the behaviour
>>>>>> described, it might be a good idea to focus on the errors in the
>>>>>> first place.
>>>>>>
>>>>>> I don't think it's anyhow related though, as switching gzip off
>>>>>> and back on, as seems to be "the fix" described in the first link,
>>>>>> is not going to help with anything.  The important part is likely
>>>>>> "restarted the server", so I would rather assume that "the server"
>>>>>> (not sure if it refers to nginx or the whole server) was using an
>>>>>> incorrect configuration and/or was out of some resources, and
>>>>>> restart fixed it.
>>>>>>
>>>>>> Summing the above, if you want to find out what goes wrong in your
>>>>>> case - you may want to provide more details.  If you don't, nobody
>>>>>> will be able to do it, unfortunately.
>>>>>>
>>>>>> The most basic thing I would recommend in the first place is to
>>>>>> look into nginx error log, it is likely to contain important
>>>>>> information if something goes wrong.
>>>>>>
>>>>>> --
>>>>>> Maxim Dounin
>>>>>> http://mdounin.ru/
>>>>>> _______________________________________________
>>>>>> nginx mailing list
>>>>>> nginx at nginx.org
>>>>>> https://mailman.nginx.org/mailman/listinfo/nginx
>>>>>>
>>>>> _______________________________________________
>>>>> nginx mailing list
>>>>> nginx at nginx.org
>>>>> https://mailman.nginx.org/mailman/listinfo/nginx
>>>>>
>>>> _______________________________________________
>>> nginx mailing list
>>> nginx at nginx.org
>>> https://mailman.nginx.org/mailman/listinfo/nginx
>>>
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org
>> https://mailman.nginx.org/mailman/listinfo/nginx
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230223/059682db/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 47810 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230223/059682db/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 282588 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230223/059682db/attachment-0005.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 111869 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230223/059682db/attachment-0006.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 193204 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230223/059682db/attachment-0007.png>


More information about the nginx mailing list