Setting a custom header on a proxied request

[Maxime Henrion]

Hi all,


I'm currently writing an nginx module that sends a number of subrequests to upstream servers based on application logic, and then aggregates the results and send them back. That part currently works just fine, thanks to the help I was given here in the past.

However, I am now trying to add a custom header to the request that is being proxied and currently failing at it. For various technical reasons, I need to add a header that contains the IP address and port (from the point of view of nginx) of the upstream server currently being contacted. In order to get at that address, I have hooked my module into the peer selection mechanism, much like the keepalive, ip_hash modules and others do. So far so good, I get at that address and can log it just fine. However, I can't seem to be able to add it as a header to my request.

Here's my init_peer callback :

static ngx_int_t
ngx_http_dispatcher_upstream_init_peer(ngx_http_request_t *r,
    ngx_http_upstream_srv_conf_t *us)
{
        ngx_http_dispatcher_upstream_peer_data_t *dp;
        ngx_http_dispatcher_srv_conf_t *dscf;
        ngx_int_t rc;

        dscf = ngx_http_conf_upstream_srv_conf(us, ngx_http_dispatcher_module);

        dp = ngx_palloc(r->pool, sizeof(ngx_http_dispatcher_upstream_peer_data_t));
        if (dp == NULL) {
                return NGX_ERROR;
        }

        rc = dscf->original_init_peer(r, us);
        if (rc != NGX_OK) {
                return rc;
        }

        dp->request = r;
        dp->data = r->upstream->peer.data;
        dp->original_get_peer = r->upstream->peer.get;
        dp->original_free_peer = r->upstream->peer.free;

        r->upstream->peer.data = dp;
        r->upstream->peer.get = ngx_http_dispatcher_upstream_get_peer;
        r->upstream->peer.free = ngx_http_dispatcher_upstream_free_peer;

#if (NGX_HTTP_SSL)
        dp->original_set_session = r->upstream->peer.set_session;
        dp->original_save_session = r->upstream->peer.save_session;
        r->upstream->peer.set_session = ngx_http_dispatcher_upstream_set_session;
        r->upstream->peer.save_session = ngx_http_dispatcher_upstream_save_session;
#endif

        return NGX_OK;
}

Note that I save the pointer to the ngx_http_request_t object in my peer data structure to access it later. Now here's my peer selection hook :

static ngx_int_t
ngx_http_dispatcher_upstream_get_peer(ngx_peer_connection_t *pc, void *data)
{
        ngx_http_dispatcher_upstream_peer_data_t *dp = data;
        u_char addrbuf[NGX_SOCKADDR_STRLEN];
        ngx_http_request_t *r;
        ngx_table_elt_t *h;
        ngx_str_t peeraddr;
        ngx_int_t rc;

        rc = dp->original_get_peer(pc, dp->data);
        if (rc != NGX_OK) {
                return rc;
        }

        peeraddr.data = addrbuf;
        peeraddr.len = ngx_sock_ntop(pc->sockaddr, pc->socklen, addrbuf,
            sizeof(addrbuf), 1);

        ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0,
            "Selected peer address is: %V", &peeraddr);

        r = dp->request;

        h = ngx_list_push(&r->headers_in.headers);
        if (h == NULL) {
                return NGX_ERROR;
        }
        h->hash = 1;
        ngx_str_set(&h->key, "X-Shard-Addr");
        h->value.data = ngx_pstrdup(r->pool, &peeraddr);
        if (h->value.data == NULL) {
                return NGX_ERROR;
        }

        h->value.len = peeraddr.len;

        return NGX_OK;
}

So after getting at the actual address we're forwarding our request to, I try to add a custom header to r->headers_in, on the request object that I saved in my peer data.

I suspect I'm not actually dealing with the correct request here, because no matter what I try, I can't get this code to actually alter the headers of the forwarded request in any way.

Any help would be greatly appreciated!
Thanks,
Maxime