proxy_pass and $content_type

[Cédric Jeanneret]

hmmm, yeah, we didn't think about that.
It can do the trick, we'll test it. It can lighten post-filtering,
even if S3 gives directory listing when there's no trailing /.

Thanks a lot!

Best regards,

C.

On Tue, Sep 15, 2009 at 6:14 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
> Hello!
>
> On Tue, Sep 15, 2009 at 05:53:41PM +0200, Cédric Jeanneret wrote:
>
>> Hmm, for file not found yes. Well, I spoke with my colleague, the big
>> dill is for directory listing: it returns 200 (that's correct) and an
>> XML (which we don't want to show up). Sooo... if S3 can be configured
>> so that it won't show some "default index", it would be great, but
>> that's not the case for now.
>
> So the solution is to just disallow directory listings via rewrite
> rules.  It should be simple enough at least if S3 do not try to
> return listings for URL's without trailing slash:
>
>    location / {
>        if ($uri ~ "/$") {
>            return 403;
>        }
>        proxy_pass ...
>    }
>
> or using regex locations:
>
>    location / {
>        proxy_pass ...
>    }
>
>    location ~ /$ {
>        return 403;
>    }
>
> Maxim Dounin
>
>>
>> I contacted nginx debian maintainer, but I don't know why, I'm pretty
>> sure he won't upgrade lenny's version. So we're stuck with some nasty
>> post-filtering.
>>
>> Thank you for your time anyway :)
>>
>> Best regards,
>>
>> C.
>>
>> On Tue, Sep 15, 2009 at 4:33 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
>> > Hello!
>> >
>> > On Tue, Sep 15, 2009 at 03:05:56PM +0200, Cedric Jeanneret wrote:
>> >
>> >> Hello,
>> >>
>> >> Well, if it was so simple... No, S3 returns 200 whatever happens. it's either an xml with a listing, or the file....
>> >> So proxy_intercept_errors won't work.
>> >
>> > According to their docs they return correct status code at least for
>> > REST interface.  Seen here:
>> >
>> > http://docs.amazonwebservices.com/AmazonS3/2006-03-01/index.html?UsingRESTError.html
>> >
>> > Maxim Dounin
>> >
>> >>
>> >> Ygor: well, lenny has legacy stable, and there's no backport.... and it seems that we can't (or won't) use other sources than the official debian ones. Any other way ?
>> >> For now, we're filtering upstream in varnish...
>> >>
>> >> Best regards,
>> >>
>> >> C.
>> >>
>> >> PS: sorry for my later mail (nginx-0.6.32-3: proxy_pass and $content_type), it was dupple because of my gmail account.... you can forget it -.-
>> >>
>> >> On Tue, 15 Sep 2009 16:33:19 +0400
>> >> Maxim Dounin <mdounin at mdounin.ru> wrote:
>> >>
>> >> > Hello!
>> >> >
>> >> > On Tue, Sep 15, 2009 at 01:40:06PM +0200, Cedric Jeanneret wrote:
>> >> >
>> >> > > Hello all,
>> >> > >
>> >> > > We're wanting to use nginx as a proxy between a varnish and a S3 storage. We're using debian lenny, nginx version is 0.6.32-3.
>> >> > >
>> >> > > What we're dowing:
>> >> > >
>> >> > > server {
>> >> > >   location / {
>> >> > >     proxy_pass ...
>> >> > >     if ($content_type !~* "image/") {
>> >> > >       return 403
>> >> > >     }
>> >> > >   }
>> >> > > }
>> >> > >
>> >> > > What happens ? well, 403 for all. We put in log "$content_type", and saw it's set to "-".
>> >> >
>> >> > You are testing *request* Content-Type, not response.  It's
>> >> > unlikely to be set at all for most requests.
>> >> >
>> >> > Additionally, this check happens at rewrite phase - i.e. before
>> >> > anything was got from upstream.
>> >> >
>> >> > >
>> >> > > Is it normal? Is there another way to filter by content type?
>> >> > > Our final goal is:
>> >> > > S3 sends either the file if it can find it, or an XML (so a content_type "text/xml" or smth like that). We don't want to give the xml, as it contains S3 bucket name...
>> >> >
>> >> > I believe S3 responds with appropriate http status code on errors,
>> >> > and you are able to intercept them via proxy_intercept_errors +
>> >> > error_page.
>> >> >
>> >> > See
>> >> >
>> >> > http://wiki.nginx.org/NginxHttpProxyModule#proxy_intercept_errors
>> >> >
>> >> > for details.
>> >> >
>> >> > Maxim Dounin
>> >> >
>> >> >
>> >> > >
>> >> > > If any of you has an idea/fixe/workaround...
>> >> > >
>> >> > > Thanks in advance
>> >> > >
>> >> > > Best regards,
>> >> > >
>> >> > > C.
>> >> > >
>> >> > >
>> >> > > --
>> >> > > Cédric Jeanneret                 |  System Administrator
>> >> > > 021 619 10 32                    |  Camptocamp SA
>> >> > > cedric.jeanneret at camptocamp.com  |  PSE-A / EPFL
>> >> >
>> >> >
>> >> >
>> >>
>> >>
>> >> --
>> >> Cédric Jeanneret                 |  System Administrator
>> >> 021 619 10 32                    |  Camptocamp SA
>> >> cedric.jeanneret at camptocamp.com  |  PSE-A / EPFL
>> >
>> >
>> >
>> >
>>
>
>