Amazon S3 authentication patch and X-Accell-Redirect

[Rafael Steil]

Hello,
I've been using the "Amazon S3 Auth Patch" (which was sent to the list some
days ago) for a while, and it works nicely, helping on a lot of issues. What
I would like to know is how to get it to work with X-Accel-Redirect, or
someway to accomplish such task, as I could not find a way to make it work
as it does without the header. I'll describe the scenario I have in order to
make it clear:

- Application (Rails, in the case) running with nginx + passenger.
- The rails app controller uses X-Accel-Redirect as described in many places
in the internet. The code is something like:
[....]
response.headers['X-Accel-Redirect'] = "/images/someImage.jpg"
response.headers['Content-Type'] = "image/jpeg"
[...]

- The nginx configuration is like this:
location /images {
internal;
proxy_pass http://myBucketName.s3.amazonaws.com;
 proxy_s3_auth on;
proxy_s3_bucket myBucketName;
proxy_s3_user accessKey;
 proxy_s3_pass secretKey;
}

If I do not use Rails + X-Accel-Redirect (and without the "internal" config
option, of course) this configuration does work pretty well. However, when I
try to use it with X-Accel-Redirect, the patch builds the "Authorization"
header for the path "/images/someImage.jpg" correctly, but what Amazon
receives is the original URL (e.g, the URL we see in the browser is "/
http://localhost:3000/myRailsController/actionName", and Amazon receives
from nginx the part "/myRailsController/actioName", instead of
"/images/someImage.jpg"), and thus the hashes doesn't match (I know that
because I added a lot of debug statements in nginx, and compared it with the
results Amazon responded and with the results I got without the
X-Accel-Redirect header).

Is there a way to get it work? Am I missing something?

In time: I had to make a small fix on the original code, which was crashing
when I started using the header. The original code is like

urlend=strchr(url, ' ');
*urlend='\0';
if (plcf->s3_secdown.len && ngx_strcmp(plcf->s3_secdown.data,"on")==0) {
url[strlen(url)-42]='\0';

However, "urlend" may be null, so I fixed it changing to

if (urlend) {
*urlend='\0';
}


That's it.

I appreciate any hint on this.

Cheers,
Rafael

--------------------------
Orignal Message
--------------------------
Hi

I've modified the proxy module to be able to authenticate with Amazon S3. It
also supports the secure download patch by removing the trailing ticket
(MD5/timestamp) from the URL.

The rationale behind this is: we're using S3 to offload all our app's static
files but need some of them to be public (images) and some private
(downloads for registered users). We're using nginx as a proxy to S3 to
cache requests and minimize the per-GET cost of S3, so our first approach
was a security through obscurity one: as the final user only sees our URL
and not the real bucket's URL, we could make the whole bucket public and use
the secure download patch just on some locations, so that the user couldn't
download files from there. But this protection was only on our server, if
some user guessed our S3 bucket he could download everything directly from
S3.

With this patch we can make parts of the bucket private, and authenticate
with S3 so that only our servers are able to download them. And it plays
nice with the secure download patch, so the files are secure both on S3 and
through our proxy.

Please find the patch attached. This is my first attempt at hacking more
than a couple lines into nginx, so I'm sure some code cleanup might apply. I
particularly don't like using static char arrays (should move to ngx_copy
and such) and couldn't get ngx_conf_set_flag_slot to work for config
options.

Config syntax is as follows:

        location /private/ {
                proxy_pass         http://BUCKET.s3.amazonaws.com/private/;
                proxy_s3_auth on;
                proxy_s3_secure_download on;   # optional if you're also
using secdownload
                proxy_s3_bucket BUCKET;
                proxy_s3_user S3_USER_ID;
                proxy_s3_pass S3_PASSWORD;
        }

Any comments, suggestions, and code corrections are welcome. :-)

Regards
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://nginx.org/pipermail/nginx/attachments/20100212/52b3970f/attachment.html>