nginx serving corrupt images
Dan Swaney
justdan23 at gmail.com
Fri Feb 24 03:17:03 UTC 2023
See my earlier response above for details. It works when you reduce the
ssl_buffer_size to 4k. You can try 8k for performance.
(Correction: use the 'server' section for this setting.)
server {
...
ssl_buffer_size 4k;
}
Reference to setting is on this page:
https://haydenjames.io/nginx-tuning-tips-tls-ssl-https-ttfb-latency/
The bug happens to be within the OpenSSL library I believe, but I found
when reducing the buffer size to 4k, it worked for my test.
There is a bug when it operates with the default of 16k and it fails to
write out the response in 16k chunks, but does work with 4k chunks.
On Thu, Feb 23, 2023 at 10:06 PM Saint Michael <venefax at gmail.com> wrote:
> Please read this, it's from 2011/11/04, and it hasn't been fixed :
>
> https://tech.blog.aknin.name/2011/11/04/nginxgzip-module-might-silently-corrupt-data-upon-backend-failure/
>
>
> On Thu, Feb 23, 2023 at 9:43 PM Dan Swaney <justdan23 at gmail.com> wrote:
>
>> Ah-ah...I caught the NGINX failure in the SSL response:
>>
>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http filename:
>>> "./html/images/image001.jpg"
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 add cleanup: 000002DC83A7CBE8
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http static fd: 732
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http set discard body
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http map started
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 parse header: "Cookie:
>>> SessionId SessionId="
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http map started
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http map: "" ""
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http script var: ""
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http map: "" ""
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http script copy: "SessionId
>>> SessionId="
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http script var: ""
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http script copy: ";AuthId="
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http script var: "SessionId
>>> SessionId="
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http script copy: " SessionId="
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http script var:
>>> "ab3e389a16b819b5477309665cfcc4672f47d5657e1c154381c6eb8336963721"
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http script copy: "; Domain=
>>> win10-web-svr.dreamstone.com; Path=/; HttpOnly; Secure"
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 HTTP/1.1 200 OK
>>> Server: nginx/1.23.3
>>> Date: Fri, 24 Feb 2023 02:24:49 GMT
>>> Content-Type: application/octetstream
>>> *Content-Length: 123197*
>>> Last-Modified: Wed, 30 Nov 2022 19:41:21 GMT
>>> Connection: keep-alive
>>> ETag: "6387b1e1-1e13d"
>>> Strict-Transport-Security: max-age=15768000; preload
>>> X-Frame-Options: SAMEORIGIN
>>> X-XSS-Protection: 1
>>> X-Content-Type-Options: nosniff
>>> Referred-Policy: strict-origin
>>> Set-Cookie: SessionId SessionId=;AuthId=SessionId SessionId=
>>> SessionId=ab3e389a16b819b5477309665cfcc4672f47d5657e1c154381c6eb8336963721;
>>> Domain=win10-web-svr.dreamstone.com; Path=/; HttpOnly; Secure
>>> Accept-Ranges: bytes
>>>
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 write new buf t:1 f:0
>>> 000002DC83A7E2F0, pos 000002DC83A7E2F0, size: 626 file: 0, size: 0
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http write filter: l:0 f:0
>>> s:626
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http output filter
>>> "/images/image001.jpg?"
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http copy filter:
>>> "/images/image001.jpg?"
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 malloc: 000002DC83A87340:32768
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http postpone filter
>>> "/images/image001.jpg?" 000002DC83A7E658
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 write old buf t:1 f:0
>>> 000002DC83A7E2F0, pos 000002DC83A7E2F0, size: 626 file: 0, size: 0
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 write new buf t:1 f:0
>>> 000002DC83A87340, pos 000002DC83A87340, size: 32768 file: 0, size: 0
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http write filter: l:0 f:1
>>> s:33394
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http write filter limit 2097152
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 malloc: 000002DC83A8F350:16384
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 SSL buf copy: 626
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 SSL buf copy: 15758
>>> *2023/02/23 21:24:49 [debug] 4768#4528: *1 SSL to write: 16384*
>>> 2023/02/23 21:24:49 [debug] 4768#4528: ssl remove session: B87DD7B9:32
>>> 2023/02/23 21:24:49 [debug] 4768#4528: shmtx lock
>>> 2023/02/23 21:24:49 [debug] 4768#4528: shmtx unlock
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 SSL_write: -1
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 SSL_get_error: 1
>>>
>>> *2023/02/23 21:24:49 [crit] 4768#4528: *1 SSL_write() failed (SSL:
>>> error:0A0C0103:SSL routines::internal error) while sending response to
>>> client, client: 192.168.80.130, server: win10-web-svr.dreamstone.com
>>> <http://win10-web-svr.dreamstone.com>, request: "GET /images/image001.jpg
>>> HTTP/1.1", host: "win10-web-svr.dreamstone.com
>>> <http://win10-web-svr.dreamstone.com>", referrer:
>>> "https://win10-web-svr.dreamstone.com/
>>> <https://win10-web-svr.dreamstone.com/>"*2023/02/23 21:24:49 [debug]
>>> 4768#4528: *1 http write filter FFFFFFFFFFFFFFFF
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http copy filter: -1
>>> "/images/image001.jpg?"
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http finalize request: -1,
>>> "/images/image001.jpg?" a:1, c:1
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http terminate request count:1
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http terminate cleanup count:1
>>> blk:0
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http posted request:
>>> "/images/image001.jpg?"
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http terminate handler count:1
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http request count:1 blk:0
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http close request
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 http log handler
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 run cleanup: 000002DC83A7CBE8
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 file cleanup: fd:732
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 free: 000002DC83A87340
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 free: 000002DC83A7BC00,
>>> unused: 0
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 free: 000002DC83A7DC20,
>>> unused: 1167
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 close http connection: 500
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 select del event fd:500 ev:768
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 reusable connection: 0
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 free: 000002DC83A647C0
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 free: 000002DC83A8F350
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 free: 000002DC8332EAB0,
>>> unused: 12
>>> 2023/02/23 21:24:49 [debug] 4768#4528: *1 free: 000002DC83A7DA10,
>>> unused: 384
>>>
>>
>> On Thu, Feb 23, 2023 at 9:32 PM Dan Swaney <justdan23 at gmail.com> wrote:
>>
>>> I tried to reproduce it with a simple web page and I'm seeing the issue
>>> which I believe you are having.
>>>
>>> By default, NGINX seems to default to some limit of 15k on responses of
>>> images returned directly.
>>>
>>> While the error.log in debug mode shows it constructs the response with
>>> the right content length, it fails to return a response if the file is over
>>> 15k.
>>>
>>> I'm using a build I did from NGINX 1.23.3. While I know I have returned
>>> files larger than this in proxy mode. This seems to be happening in normal
>>> web server page mode.
>>>
>>> [image: image.png]
>>>
>>>
>>> I even tried to increase it by adding:
>>>
>>> location / {
>>>> root html;
>>>> index index.html index.htm favicon.ico;
>>>>
>>>> types {
>>>> text/html html;
>>>> image/gif gif;
>>>> application/octetstream jpg;
>>>> application/octetstream png;
>>>> }
>>>>
>>>> set $max_image_size 50000;
>>>> client_max_body_size 50000;
>>>> }
>>>>
>>>
>>> On Thu, Feb 23, 2023 at 8:09 PM Dan Swaney <justdan23 at gmail.com> wrote:
>>>
>>>> Try this:
>>>>
>>>> 1. At the very top of your nginx.conf file, add a definition:
>>>>
>>>> error_log logs/error.log debug;
>>>>
>>>> 2. Within your nginx install directory (where nginx.exe exists),
>>>> create a 'logs' folder if one does not exist.
>>>> 3. Restart nginx.exe (if running as a Windows Service with NSSM,
>>>> then restart the service).
>>>> 4. If NGINX does not start...
>>>> - Check the 'error.log' as it will tell you if your nginx.conf
>>>> has something weird in it that it doesn't like.
>>>> 5. If NGINX started successfully...
>>>> - Run your test from your client browser.
>>>> - Go back to the Server and check your 'error.log' and look for
>>>> the URL request.
>>>> - It will show you everything it does with routing or looking
>>>> for files.
>>>>
>>>>
>>>> On Thu, Feb 23, 2023 at 7:22 PM Saint Michael <venefax at gmail.com>
>>>> wrote:
>>>>
>>>>> Now it remains broken and I have no idea how to fix it. I guess is
>>>>> caching bad copies of the pictures.
>>>>> I re-uploaded the two images.
>>>>> Kindly let me know if this is "normal"
>>>>> [image: image.png]
>>>>>
>>>>> On Thu, Feb 23, 2023 at 7:08 PM Dan Swaney <justdan23 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Now your image001.jpg is returning your home page:
>>>>>> [image: image.png]
>>>>>>
>>>>>> If I had to guess, your location routing is re-routing all sub-urls
>>>>>> to your base URL at '/' which is defaulting to your index.html (or whatever
>>>>>> you have set for your default).
>>>>>> The tip was when it returns the content type as 'text/html' instead
>>>>>> of 'image/jpg'.
>>>>>>
>>>>>> As Maxim cited, your 'location' directives are for routing URL paths
>>>>>> and not files. Think of the external viewed URL and the internal routed
>>>>>> URL location.
>>>>>>
>>>>>> Looking at your previous cited partial nginx.conf:
>>>>>>
>>>>>> - root /static/duc/;
>>>>>> - I usually define this in my 'location' base section only and
>>>>>> drop the initial '/' if it is relative to where NGINX is running.
>>>>>> - I then don't need it in any other 'location' sections for
>>>>>> sub-folders which have different security.
>>>>>> - You have it defined in your 'server' section.
>>>>>>
>>>>>>
>>>>>> - default_type 'text/html; charset=UTF-8';
>>>>>> - I usually define this at the top of my 'http' section.
>>>>>> - Normally I use 'default_type application/octet-stream;'
>>>>>> - You have it defined in your 'server' section.
>>>>>> - I see it returning your images as 'text/html'.
>>>>>>
>>>>>> - try_files $uri $uri/ /index.html;
>>>>>> - I usually define a default 'index' if at the root and nothing
>>>>>> else is added.
>>>>>> - For example: 'index index.html;'
>>>>>> - Remove the try_files like recommended earlier.
>>>>>> - If you need to restrict access to specific folder URL
>>>>>> mappings, then define a location of the URL mapping and add one line...
>>>>>> - 'deny all;'
>>>>>> - Otherwise leave it open to all sub URLs by doing nothing
>>>>>> more but use a single 'location /' rule.
>>>>>>
>>>>>>
>>>>>> On Thu, Feb 23, 2023 at 6:15 PM Dan Swaney <justdan23 at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I just ran your test and it works fine from Chrome and in Visual
>>>>>>> Studio Code:
>>>>>>>
>>>>>>> > wget https://x3x.us/index_files/image001.jpg
>>>>>>>> StatusCode : 200
>>>>>>>> StatusCode : 200
>>>>>>>> StatusDescription : OK
>>>>>>>> Content : {255, 216, 255, 224...}
>>>>>>>> RawContent : HTTP/1.1 200 OK
>>>>>>>> Connection: keep-alive
>>>>>>>> Strict-Transport-Security: max-age=31536000;
>>>>>>>> includeSubDomains
>>>>>>>> Accept-Ranges: bytes
>>>>>>>> Content-Length: 8834
>>>>>>>> Content-Type: image/jpeg
>>>>>>>> Date: Thu, 23 Feb 2023 23...
>>>>>>>> Headers : {[Connection, keep-alive],
>>>>>>>> [Strict-Transport-Security, max-age=31536000; includeSubDomains],
>>>>>>>> [Accept-Ranges, bytes],
>>>>>>>> [Content-Length, 8834]...}
>>>>>>>> RawContentLength : 8834
>>>>>>>
>>>>>>>
>>>>>>> [image: image.png]
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Feb 22, 2023 at 7:36 PM Saint Michael <venefax at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> a) The error does not have a single line.
>>>>>>>> b) restarting does not fix it
>>>>>>>> c) my nginx is no acting as proxy
>>>>>>>> d) it happened twice and both times I fixed it by turning gzip off,
>>>>>>>> restarting, and back on.
>>>>>>>> e) I also noticed that I requested the image file with wget, get a
>>>>>>>> full HTML file for the whole document, but named as if it were the image
>>>>>>>> file.
>>>>>>>>
>>>>>>>> wget https://x3x.us/index_files/image001.jpg
>>>>>>>> but `stat image001.jpg' showed it was the entire text HTML file.
>>>>>>>>
>>>>>>>> http {
>>>>>>>> client_max_body_size 1500M;
>>>>>>>> include mime.types;
>>>>>>>> # default_type application/octet-stream;
>>>>>>>>
>>>>>>>> #log_format main '$remote_addr - $remote_user [$time_local]
>>>>>>>> "$request" '
>>>>>>>> # '$status $body_bytes_sent "$http_referer" '
>>>>>>>> # '"$http_user_agent" "$http_x_forwarded_for"';
>>>>>>>>
>>>>>>>> #access_log logs/access.log main;
>>>>>>>> sendfile on;
>>>>>>>> tcp_nopush on;
>>>>>>>> tcp_nodelay on;
>>>>>>>> gzip on;
>>>>>>>> gzip_vary on;
>>>>>>>> gzip_min_length 10240;
>>>>>>>> gzip_proxied expired no-cache no-store private auth;
>>>>>>>> gzip_types text/plain text/css text/xml text/javascript
>>>>>>>> application/x-javascript application/xml;
>>>>>>>> gzip_disable "MSIE [1-6]\.";
>>>>>>>> #keepalive_timeout 0;
>>>>>>>> keepalive_timeout 65;
>>>>>>>> types_hash_max_size 2048;
>>>>>>>> proxy_headers_hash_max_size 1024;
>>>>>>>> proxy_headers_hash_bucket_size 128;
>>>>>>>> #gzip on;
>>>>>>>> # error_log /var/log/nginx/error.log debug;
>>>>>>>>
>>>>>>>> error_log /var/log/nginx/error.log error;
>>>>>>>> error_log /var/log/nginx/error.log crit;
>>>>>>>>
>>>>>>>> access_log /var/log/nginx/access.log;
>>>>>>>>
>>>>>>>> server {
>>>>>>>> add_header Strict-Transport-Security "max-age=31536000;
>>>>>>>> includeSubDomains" always;
>>>>>>>> default_type 'text/html; charset=UTF-8';
>>>>>>>> listen 208.78.161.6:80;
>>>>>>>> server_name x3x.us;
>>>>>>>>
>>>>>>>> # Redirect all domains to https://x3x.us
>>>>>>>> if ($scheme != "https") {
>>>>>>>> return 301 https://x3x.us$request_uri;
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> server {
>>>>>>>> add_header Strict-Transport-Security "max-age=31536000;
>>>>>>>> includeSubDomains" always;
>>>>>>>> default_type 'text/html; charset=UTF-8';
>>>>>>>> listen 208.78.161.6:443 ssl;
>>>>>>>> server_name x3x.us;
>>>>>>>>
>>>>>>>> ssl_certificate /etc/letsencrypt/live/x3x.us/fullchain.pem;
>>>>>>>> ssl_certificate_key /etc/letsencrypt/live/x3x.us/privkey.pem;
>>>>>>>>
>>>>>>>> root /static/duc/;
>>>>>>>>
>>>>>>>> location / {
>>>>>>>>
>>>>>>>> try_files $uri $uri/ /index.html;
>>>>>>>>
>>>>>>>>
>>>>>>>> }
>>>>>>>>
>>>>>>>> } #server
>>>>>>>>
>>>>>>>> } #http
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Feb 22, 2023 at 7:17 PM Maxim Dounin <mdounin at mdounin.ru>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hello!
>>>>>>>>>
>>>>>>>>> On Wed, Feb 22, 2023 at 02:46:29PM -0500, Saint Michael wrote:
>>>>>>>>>
>>>>>>>>> > It's not a misconfiguration, is a huge bug.
>>>>>>>>> > A wasted two days of sleep for something that is 100% a bug.
>>>>>>>>> > Please read here:
>>>>>>>>> >
>>>>>>>>> https://laracasts.com/discuss/channels/general-discussion/homestead-nginx-serving-wrong-images-and-only-cut-in-the-middle
>>>>>>>>> > He mentions the same exact problem and also he points to
>>>>>>>>> >
>>>>>>>>> https://tech.blog.aknin.name/2011/11/04/nginxgzip-module-might-silently-corrupt-data-upon-backend-failure/
>>>>>>>>> > where the author says that Niginx will not fix it.
>>>>>>>>> > So he already tried he was rebuffed.
>>>>>>>>>
>>>>>>>>> The fun fact is that the referenced article doesn't state "will
>>>>>>>>> not fix", but rather "not a top priority". Further, proper error
>>>>>>>>> propagation is available in nginx for about 10 years now, since
>>>>>>>>> 2013 (http://hg.nginx.org/nginx/rev/d3eab5e2df5f, nginx 1.5.3).
>>>>>>>>> Quoting CHANGES:
>>>>>>>>>
>>>>>>>>> *) Change: now after receiving an incomplete response from a
>>>>>>>>> backend
>>>>>>>>> server nginx tries to send an available part of the
>>>>>>>>> response to a
>>>>>>>>> client, and then closes client connection.
>>>>>>>>>
>>>>>>>>> As long as nginx have an information about an error, it will
>>>>>>>>> preserve this information and propagate it to the client.
>>>>>>>>>
>>>>>>>>> Also note that it is only expected to make a difference if you are
>>>>>>>>> using nginx as a proxy, not to directly serve files. And only in
>>>>>>>>> case of errors. That is, if you are seeing the behaviour
>>>>>>>>> described, it might be a good idea to focus on the errors in the
>>>>>>>>> first place.
>>>>>>>>>
>>>>>>>>> I don't think it's anyhow related though, as switching gzip off
>>>>>>>>> and back on, as seems to be "the fix" described in the first link,
>>>>>>>>> is not going to help with anything. The important part is likely
>>>>>>>>> "restarted the server", so I would rather assume that "the server"
>>>>>>>>> (not sure if it refers to nginx or the whole server) was using an
>>>>>>>>> incorrect configuration and/or was out of some resources, and
>>>>>>>>> restart fixed it.
>>>>>>>>>
>>>>>>>>> Summing the above, if you want to find out what goes wrong in your
>>>>>>>>> case - you may want to provide more details. If you don't, nobody
>>>>>>>>> will be able to do it, unfortunately.
>>>>>>>>>
>>>>>>>>> The most basic thing I would recommend in the first place is to
>>>>>>>>> look into nginx error log, it is likely to contain important
>>>>>>>>> information if something goes wrong.
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Maxim Dounin
>>>>>>>>> http://mdounin.ru/
>>>>>>>>> _______________________________________________
>>>>>>>>> nginx mailing list
>>>>>>>>> nginx at nginx.org
>>>>>>>>> https://mailman.nginx.org/mailman/listinfo/nginx
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> nginx mailing list
>>>>>>>> nginx at nginx.org
>>>>>>>> https://mailman.nginx.org/mailman/listinfo/nginx
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>> nginx mailing list
>>>>>> nginx at nginx.org
>>>>>> https://mailman.nginx.org/mailman/listinfo/nginx
>>>>>>
>>>>> _______________________________________________
>>>>> nginx mailing list
>>>>> nginx at nginx.org
>>>>> https://mailman.nginx.org/mailman/listinfo/nginx
>>>>>
>>>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org
>> https://mailman.nginx.org/mailman/listinfo/nginx
>>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> https://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230223/d4a1018c/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 47810 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230223/d4a1018c/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 282588 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230223/d4a1018c/attachment-0005.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 111869 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230223/d4a1018c/attachment-0006.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 193204 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20230223/d4a1018c/attachment-0007.png>
More information about the nginx
mailing list