Nginx Ingress Controller with Prometheus Metrics Issue

[Owen Garrett]

Hi Bill,

The workqueue_ metrics relate to the processing of configuration updates.  Config updates can be triggered by an update to an Ingress Resource, Endpoints in a monitored Service, and other events.  The purpose of these metrics is to observe the backlog to determine how quickly IC can respond to changes.  They probably do not provide the data you want (“I basically am interested in analyzing the time spent for a request in the ingress controller”)?

When you use the Ingress Controller with NGINX open source or NGINX Plus, you can get this data from the log files: see https://github.com/nginxinc/kubernetes-ingress/tree/v1.10.0/examples/custom-log-format and the NGINX variable $request_time http://nginx.org/en/docs/http/ngx_http_log_module.html#var_request_time  When you use it with NGINX Plus, you additionally get aggregate request time data in the Prometheus data

Feel free to raise additional questions about Ingress Controller on the github page: https://github.com/nginxinc/kubernetes-ingress/issues

Owen

From: nginx <nginx-bounces at nginx.org> on behalf of William Shaya <wshaya at osisoft.com>
Reply to: "nginx at nginx.org" <nginx at nginx.org>
Date: Thursday, 28 January 2021 at 18:10
To: "nginx at nginx.org" <nginx at nginx.org>
Subject: Nginx Ingress Controller with Prometheus Metrics Issue


EXTERNAL MAIL: nginx-bounces at nginx.org
Hi,

I have a Kubernetes cluster and am using the nginx ingress controller.  I have followed the instructions at:
https://docs.nginx.com/nginx-ingress-controller/logging-and-monitoring/prometheus/

I can see these metrics in Prometheus, however, I DO NOT see:

Workqueue metrics. Note: the workqueue is a queue used by the Ingress Controller to process changes to the relevant resources in the cluster like Ingress resources. The Ingress Controller uses only one queue. The metrics for that queue will have the label name="taskQueue"
·         workqueue_depth. Current depth of the workqueue.
·         workqueue_queue_duration_second. How long in seconds an item stays in the workqueue before being requested.
·         workqueue_work_duration_seconds. How long in seconds processing an item from the workqueue takes.

Any help is determining what the issue is here would be greatly appreciated.  I basically am interested in analyzing the time spent for a request in the ingress controller

Thanks
Bill
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20210129/c750b9ad/attachment.htm>