Ingress get into CrashLoopBackOf state

[saggir]

Hi
We are trying to create ~4000 namespaces with ingress rules and we cannot
reach above ~2000 namespaces…
Our problematic use case is:
1.	We create k8s namespace with 1 pod (3 containers) and 1 ingress (3 rules
per ingress).
2.	As part of the creation of the namespace we run a few https queries and
sync some data to the containers  (GET & POST)
3.	After creating ~2050 namespaces the NGINX ingress controller get into
CrashLoopBackOf state and cannot restart anymore.

We use NGINX ingress controller with image 0.24.1 in AWS environment (mapped
to nlb), we run a replica of 4 pods with 4cpu and 12Gi memory
We tried a few configuration changes but it seems we cannot find the root
cause of the limitation.

In the logs of the nginx controller the problem start in the dynamic
configuration with reason "no memory":  
"[error] 50#50: *165 [lua] configuration.lua:182: call():
dynamic-configuration: error updating configuration: no memory
controller.go:220] Dynamic reconfiguration failed: unexpected error code:
400
controller.go:224] Unexpected failure reconfiguring NGINX"

It seems like we are missing some key configuration limit that causes k8s to
kill the pod…

Appreciate your assistance!

Saggi Raiter.
Saggi.raiter at sap.com

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,283864,283864#msg-283864