Keycloak login issue - incorrect /_token call to non-existent backend server

[Francis Daly]

On Sun, Oct 17, 2021 at 09:21:03AM +0200, Jernej Vodopivec wrote:

Hi there,

I don't know about Keycloak; and I don't have the answer to your question;
but I can see one thing that might not be doing what you expect:

> 1. Application: https://app.domain.com
> Application is running on backend IIS server https://appbackend.domain.com
> 
> NGINX Virtual host config:
> a) Headers set:
> proxy_set_header X-Forwarded-Proto $scheme;
> proxy_set_header Host $host;
> proxy_set_header Forwarded "$proxy_add_forwarded;proto=$scheme";
...

> c) location part
> auth_jwt "" token=$session_jwt;
> error_page 401 = @do_oidc_flow;
> auth_jwt_key_request /_jwks_uri;
> proxy_set_header username $jwt_claim_sub;
> proxy_pass      https://appbackend.domain.com;

If the "location" that handles this request includes "proxy_set_header
username $jwt_claim_sub;", then all of the "Virtual host config"
proxy_set_header lines will not apply to this request.

If the thing at https://appbackend.domain.com is expecting to use
anything like X-Real-IP or Host to decide how to handle the request,
it may not be getting the values that you want it to be getting.

Either remove the "proxy_set_header username", or add all of the other
"proxy_set_header" lines to the location{} block.

Other than that, your keycloak.domain.com-access.log does mention
"client_id=NGINX-Plus" -- if you have an active nginx-plus subscription,
you might find a more reliable and timely response if you contact the
nginx-plus support system directly.

(It's perfectly fine to ask here; asking there as well might get you a
better answer.)

> NGINX logs
>  /var/log/nginx/app.domain.com-access.log <==
> remote_addr=184.55.14.22 - remote_user=- time_local=[17/Oct/2021:09:06:17
> +0200] request="GET / HTTP/2.0" status=302 body_bytes_sent=145

That is: your client asked app.domain.com for /, and was told to talk to
keycloak first. (Here all we see is that the response was 302; I assume
that the redirection was to keycloak.)

> ==> /var/log/nginx/keycloak.domain.com-access.log <==
> remote_addr=184.55.14.22 - remote_user=- time_local=[17/Oct/2021:09:06:17
> +0200] request="GET
> /auth/realms/master/protocol/openid-connect/auth?response_type=code&scope=openid+profile+email+offline_access&client_id=NGINX-Plus&redirect_uri=https://app.domain.com:443/_codexch&nonce=5--Pw-iCkTs1hR-3V6wgLkd2vZNC0ys0NM9fRR4D1c8&state=0
> HTTP/2.0" status=302 body_bytes_sent=0 http_referer="-"

Here, the same client asks keycloak.domain.com for the /auth/... url,
and gets a 302 response, presumably back to the "redirect_uri" parameter
-- https://app.domain.com:443/_codexch.

> upstream_response_time="0.032" server_name=keycloak.domain.com
> uri="/auth/realms/master/protocol/openid-connect/auth"
> 1c8&state=0 HTTP/2.0", status=302, waf_policy=Complete_OWASP_Top_Ten,

(There is possibly a bit missing in between those two lines -- it may not matter here.)

> ==> /var/log/nginx/app.domain.com-error.log <==
> 2021/10/17 09:06:18 [error] 3352262#3352262: *406 connect() failed (111:
> Connection refused) while connecting to upstream, client: 184.55.14.22,
> server: app.domain.com, request: "GET
> /_codexch?state=0&session_state=0b783755-9b00-4b0f-9e63-1a047680272c&code=07ce9447-19a7-443f-abfb-54e92819a34a.0b783755-9b00-4b0f-9e63-1a047680272c.98d80b2d-9f0d-482a-bdfd-b680834bb9bc
> HTTP/2.0", subrequest: "/_token", upstream: "
> https://127.0.0.1:80/auth/realms/master/protocol/openid-connect/token",
> host: "app.domain.com"

And here, the same client asks app.domain.com for /_codexch with a
query_string, which seems to lead the nginx at app.domain.com to make a
subrequest for /_token, which it thinks it should handle by talking to
something below https://127.0.0.1:80/.

What is the proxy_pass line that app.domain.com has for /_token? It
looks like it possibly should be "proxy_pass $oidc_token_endpoint;",
and it looks like you have configured that variable to start with
"https://keycloak.domain.com/". Can you see any reason why that might
have become https://127.0.0.1:80/?

Good luck with it,

	f
-- 
Francis Daly        francis at daoine.org