nginx serving corrupt images

Saint Michael venefax at gmail.com
Fri Feb 24 02:47:01 UTC 2023


You have the root inside the location:
location / {
            root   html;

I had it outside

root static/duc/;

        location / {

What is the correct way to do this?



On Thu, Feb 23, 2023 at 9:42 PM Saint Michael <venefax at gmail.com> wrote:

> ok, so I am not insane. Is good to know.
> 23/02/24 02:38:44 [notice] 174292#0: using the "epoll" event method
> 2023/02/24 02:38:44 [notice] 174292#0: openresty/1.21.4.1
> 2023/02/24 02:38:44 [notice] 174292#0: built by gcc 11.3.0 (Ubuntu
> 11.3.0-1ubuntu1~22.04)
> 2023/02/24 02:38:44 [notice] 174292#0: OS: Linux 6.1.10-1-pve
> 2023/02/24 02:38:44 [notice] 174292#0: getrlimit(RLIMIT_NOFILE):
> 1000000:1000000
>
> can you please file a bug? They would not talk to me because  I am a
> newbie.
>
> Yours
> Federico
>
>
>
>
> On Thu, Feb 23, 2023 at 9:33 PM Dan Swaney <justdan23 at gmail.com> wrote:
>
>> I tried to reproduce it with a simple web page and I'm seeing the issue
>> which I believe you are having.
>>
>> By default, NGINX seems to default to some limit of 15k on responses of
>> images returned directly.
>>
>> While the error.log in debug mode shows it constructs the response with
>> the right content length, it fails to return a response if the file is over
>> 15k.
>>
>> I'm using a build I did from NGINX 1.23.3.  While I know I have returned
>> files larger than this in proxy mode.  This seems to be happening in normal
>> web server page mode.
>>
>> [image: image.png]
>>
>>
>> I even tried to increase it by adding:
>>
>>         location / {
>>>             root   html;
>>>             index  index.html index.htm favicon.ico;
>>>
>>>             types {
>>>                text/html  html;
>>>                image/gif  gif;
>>>                application/octetstream jpg;
>>>                application/octetstream png;
>>>             }
>>>
>>>             set $max_image_size 50000;
>>>             client_max_body_size 50000;
>>>         }
>>>
>>
>> On Thu, Feb 23, 2023 at 8:09 PM Dan Swaney <justdan23 at gmail.com> wrote:
>>
>>> Try this:
>>>
>>>    1. At the very top of your nginx.conf file, add a definition:
>>>
>>>            error_log  logs/error.log debug;
>>>
>>>    2. Within your nginx install directory (where nginx.exe exists),
>>>    create a 'logs' folder if one does not exist.
>>>    3. Restart nginx.exe (if running as a Windows Service with NSSM,
>>>    then restart the service).
>>>    4. If NGINX does not start...
>>>       - Check the 'error.log' as it will tell you if your nginx.conf
>>>       has something weird in it that it doesn't like.
>>>    5. If NGINX started successfully...
>>>       - Run your test from your client browser.
>>>       - Go back to the Server and check your 'error.log' and look for
>>>       the URL request.
>>>       - It will show you everything it does with routing or looking for
>>>       files.
>>>
>>>
>>> On Thu, Feb 23, 2023 at 7:22 PM Saint Michael <venefax at gmail.com> wrote:
>>>
>>>> Now it remains broken and I have no idea how to fix it. I guess is
>>>> caching bad copies of the pictures.
>>>> I re-uploaded the two images.
>>>> Kindly let me know if this is "normal"
>>>> [image: image.png]
>>>>
>>>> On Thu, Feb 23, 2023 at 7:08 PM Dan Swaney <justdan23 at gmail.com> wrote:
>>>>
>>>>> Now your image001.jpg is returning your home page:
>>>>> [image: image.png]
>>>>>
>>>>> If I had to guess, your location routing is re-routing all sub-urls to
>>>>> your base URL at '/' which is defaulting to your index.html (or whatever
>>>>> you have set for your default).
>>>>> The tip was when it returns the content type as 'text/html' instead of
>>>>> 'image/jpg'.
>>>>>
>>>>> As Maxim cited, your 'location' directives are for routing URL paths
>>>>> and not files.  Think of the external viewed URL and the internal routed
>>>>> URL location.
>>>>>
>>>>> Looking at your previous cited partial nginx.conf:
>>>>>
>>>>>    - root /static/duc/;
>>>>>       - I usually define this in my 'location' base section only and
>>>>>       drop the initial '/' if it is relative to where NGINX is running.
>>>>>          - I then don't need it in any other 'location' sections for
>>>>>          sub-folders which have different security.
>>>>>       - You have it defined in your 'server' section.
>>>>>
>>>>>
>>>>>    - default_type  'text/html; charset=UTF-8';
>>>>>       - I usually define this at the top of my 'http' section.
>>>>>          - Normally I use 'default_type  application/octet-stream;'
>>>>>       - You have it defined in your 'server' section.
>>>>>       - I see it returning your images as 'text/html'.
>>>>>
>>>>>       - try_files $uri $uri/ /index.html;
>>>>>    - I usually define a default 'index' if at the root and nothing
>>>>>       else is added.
>>>>>          - For example: 'index index.html;'
>>>>>       - Remove the try_files like recommended earlier.
>>>>>       - If you need to restrict access to specific folder URL
>>>>>       mappings, then define a location of the URL mapping and add one line...
>>>>>          - 'deny all;'
>>>>>          - Otherwise leave it open to all sub URLs by doing nothing
>>>>>          more but use a single 'location /' rule.
>>>>>
>>>>>
>>>>> On Thu, Feb 23, 2023 at 6:15 PM Dan Swaney <justdan23 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I just ran your test and it works fine from Chrome and in Visual
>>>>>> Studio Code:
>>>>>>
>>>>>> > wget https://x3x.us/index_files/image001.jpg
>>>>>>> StatusCode        : 200
>>>>>>> StatusCode        : 200
>>>>>>> StatusDescription : OK
>>>>>>> Content           : {255, 216, 255, 224...}
>>>>>>> RawContent        : HTTP/1.1 200 OK
>>>>>>>                     Connection: keep-alive
>>>>>>>                     Strict-Transport-Security: max-age=31536000;
>>>>>>> includeSubDomains
>>>>>>>                     Accept-Ranges: bytes
>>>>>>>                     Content-Length: 8834
>>>>>>>                     Content-Type: image/jpeg
>>>>>>>                     Date: Thu, 23 Feb 2023 23...
>>>>>>> Headers           : {[Connection, keep-alive],
>>>>>>> [Strict-Transport-Security, max-age=31536000; includeSubDomains],
>>>>>>> [Accept-Ranges, bytes],
>>>>>>>                     [Content-Length, 8834]...}
>>>>>>> RawContentLength  : 8834
>>>>>>
>>>>>>
>>>>>> [image: image.png]
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Feb 22, 2023 at 7:36 PM Saint Michael <venefax at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> a) The error does not have a single line.
>>>>>>> b) restarting does not fix it
>>>>>>> c) my nginx is no acting as proxy
>>>>>>> d) it happened twice and both times I fixed it by turning gzip off,
>>>>>>> restarting, and back on.
>>>>>>> e) I also noticed that I requested the image file with wget, get a
>>>>>>> full HTML file for the whole document, but named as if it were the image
>>>>>>> file.
>>>>>>>
>>>>>>> wget https://x3x.us/index_files/image001.jpg
>>>>>>> but `stat image001.jpg' showed it was the entire text HTML file.
>>>>>>>
>>>>>>> http {
>>>>>>> client_max_body_size 1500M;
>>>>>>>     include       mime.types;
>>>>>>>    # default_type  application/octet-stream;
>>>>>>>
>>>>>>>     #log_format  main  '$remote_addr - $remote_user [$time_local]
>>>>>>> "$request" '
>>>>>>>     #                  '$status $body_bytes_sent "$http_referer" '
>>>>>>>     #                  '"$http_user_agent" "$http_x_forwarded_for"';
>>>>>>>
>>>>>>>     #access_log  logs/access.log  main;
>>>>>>>     sendfile        on;
>>>>>>>     tcp_nopush     on;
>>>>>>>     tcp_nodelay     on;
>>>>>>> gzip on;
>>>>>>> gzip_vary on;
>>>>>>> gzip_min_length 10240;
>>>>>>> gzip_proxied expired no-cache no-store private auth;
>>>>>>> gzip_types text/plain text/css text/xml text/javascript
>>>>>>> application/x-javascript application/xml;
>>>>>>> gzip_disable "MSIE [1-6]\.";
>>>>>>>     #keepalive_timeout  0;
>>>>>>>     keepalive_timeout  65;
>>>>>>>  types_hash_max_size 2048;
>>>>>>> proxy_headers_hash_max_size 1024;
>>>>>>> proxy_headers_hash_bucket_size 128;
>>>>>>>         #gzip  on;
>>>>>>>     #    error_log  /var/log/nginx/error.log debug;
>>>>>>>
>>>>>>> error_log /var/log/nginx/error.log error;
>>>>>>> error_log /var/log/nginx/error.log crit;
>>>>>>>
>>>>>>>         access_log  /var/log/nginx/access.log;
>>>>>>>
>>>>>>> server {
>>>>>>>         add_header Strict-Transport-Security "max-age=31536000;
>>>>>>> includeSubDomains" always;
>>>>>>>         default_type  'text/html; charset=UTF-8';
>>>>>>>     listen 208.78.161.6:80;
>>>>>>>     server_name x3x.us;
>>>>>>>
>>>>>>>     # Redirect all domains to https://x3x.us
>>>>>>>     if ($scheme != "https") {
>>>>>>>         return 301 https://x3x.us$request_uri;
>>>>>>>     }
>>>>>>> }
>>>>>>>
>>>>>>> server {
>>>>>>>        add_header Strict-Transport-Security "max-age=31536000;
>>>>>>> includeSubDomains" always;
>>>>>>>         default_type  'text/html; charset=UTF-8';
>>>>>>>     listen 208.78.161.6:443 ssl;
>>>>>>>     server_name x3x.us;
>>>>>>>
>>>>>>>     ssl_certificate /etc/letsencrypt/live/x3x.us/fullchain.pem;
>>>>>>>     ssl_certificate_key /etc/letsencrypt/live/x3x.us/privkey.pem;
>>>>>>>
>>>>>>> root /static/duc/;
>>>>>>>
>>>>>>>         location / {
>>>>>>>
>>>>>>> try_files $uri $uri/ /index.html;
>>>>>>>
>>>>>>>
>>>>>>> }
>>>>>>>
>>>>>>> } #server
>>>>>>>
>>>>>>> } #http
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Feb 22, 2023 at 7:17 PM Maxim Dounin <mdounin at mdounin.ru>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hello!
>>>>>>>>
>>>>>>>> On Wed, Feb 22, 2023 at 02:46:29PM -0500, Saint Michael wrote:
>>>>>>>>
>>>>>>>> > It's not a misconfiguration, is a huge bug.
>>>>>>>> > A wasted two days of sleep for something that is 100% a bug.
>>>>>>>> > Please read here:
>>>>>>>> >
>>>>>>>> https://laracasts.com/discuss/channels/general-discussion/homestead-nginx-serving-wrong-images-and-only-cut-in-the-middle
>>>>>>>> > He mentions the same exact problem and also he points to
>>>>>>>> >
>>>>>>>> https://tech.blog.aknin.name/2011/11/04/nginxgzip-module-might-silently-corrupt-data-upon-backend-failure/
>>>>>>>> > where the author says that Niginx will not fix it.
>>>>>>>> > So he already tried he was rebuffed.
>>>>>>>>
>>>>>>>> The fun fact is that the referenced article doesn't state "will
>>>>>>>> not fix", but rather "not a top priority".  Further, proper error
>>>>>>>> propagation is available in nginx for about 10 years now, since
>>>>>>>> 2013 (http://hg.nginx.org/nginx/rev/d3eab5e2df5f, nginx 1.5.3).
>>>>>>>> Quoting CHANGES:
>>>>>>>>
>>>>>>>>     *) Change: now after receiving an incomplete response from a
>>>>>>>> backend
>>>>>>>>        server nginx tries to send an available part of the response
>>>>>>>> to a
>>>>>>>>        client, and then closes client connection.
>>>>>>>>
>>>>>>>> As long as nginx have an information about an error, it will
>>>>>>>> preserve this information and propagate it to the client.
>>>>>>>>
>>>>>>>> Also note that it is only expected to make a difference if you are
>>>>>>>> using nginx as a proxy, not to directly serve files.  And only in
>>>>>>>> case of errors.  That is, if you are seeing the behaviour
>>>>>>>> described, it might be a good idea to focus on the errors in the
>>>>>>>> first place.
>>>>>>>>
>>>>>>>> I don't think it's anyhow related though, as switching gzip off
>>>>>>>> and back on, as seems to be "the fix" described in the first link,
>>>>>>>> is not going to help with anything.  The important part is likely
>>>>>>>> "restarted the server", so I would rather assume that "the server"
>>>>>>>> (not sure if it refers to nginx or the whole server) was using an
>>>>>>>> incorrect configuration and/or was out of some resources, and
>>>>>>>> restart fixed it.
>>>>>>>>
>>>>>>>> Summing the above, if you want to find out what goes wrong in your
>>>>>>>> case - you may want to provide more details.  If you don't, nobody
>>>>>>>> will be able to do it, unfortunately.
>>>>>>>>
>>>>>>>> The most basic thing I would recommend in the first place is to
>>>>>>>> look into nginx error log, it is likely to contain important
>>>>>>>> information if something goes wrong.
>>>>>>>>
>>>>>>>> --
>>>>>>>> Maxim Dounin
>>>>>>>> http://mdounin.ru/
>>>>>>>> _______________________________________________
>>>>>>>> nginx mailing list
>>>>>>>> nginx at nginx.org
>>>>>>>> https://mailman.nginx.org/mailman/listinfo/nginx
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> nginx mailing list
>>>>>>> nginx at nginx.org
>>>>>>> https://mailman.nginx.org/mailman/listinfo/nginx
>>>>>>>
>>>>>> _______________________________________________
>>>>> nginx mailing list
>>>>> nginx at nginx.org
>>>>> https://mailman.nginx.org/mailman/listinfo/nginx
>>>>>
>>>> _______________________________________________
>>>> nginx mailing list
>>>> nginx at nginx.org
>>>> https://mailman.nginx.org/mailman/listinfo/nginx
>>>>
>>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org
>> https://mailman.nginx.org/mailman/listinfo/nginx
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230223/8f3c3cd0/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 47810 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230223/8f3c3cd0/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 282588 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230223/8f3c3cd0/attachment-0005.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 111869 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230223/8f3c3cd0/attachment-0006.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 193204 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230223/8f3c3cd0/attachment-0007.png>


More information about the nginx mailing list