Server (Proxy) Access by IP and Forward real IP to Proxy Logs

[basti]

Hello,
I have the following config

Frontend (with IP x.y.1.1) -> Proxy

In the Proxy settings I have "allow x.y.1.1" and this work very well.
Now I want to see the client IP how access to frontend in the proxy logs
and have add something like

# to get real IP who access
set_real_ip_from x.y.1.1/32;
real_ip_header X-Real-IP;

now I see the Real IP who access the frontend but I get no access to proxy.

I have also try to place the "allow x.y.1.1" before the set_real_ip_from
but this does not help.

Is there a way to get this working? In other words to allow connection
from Frontend and print client IP's to log (without iptables) ??

Best Regards,