<div dir="ltr"><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Hi All,</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">I have something like this. I tested the `</span>tail -f /var/log/nginx/error.log | awk -f  /var/log/nginx/test.awk` part and it just works fine. But when i try to run it through fluentd, it doesn't do anything. Any idea why ? </p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><source></span></p>
<p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">      @type exec</span></p>
<p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">      format json</span></p>
<p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">      tag sample</span></p>
<p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">      command tail -f /var/log/nginx/error.log | awk -f  /var/log/nginx/test.awk</span></p>
<p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">  </source></span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);min-height:13px"><span style="font-variant-ligatures:no-common-ligatures"></span></p>
<p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);min-height:13px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></p>
<p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><match sample ></span></p>
<p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">  @type stdout</span></p>
<p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"></match></span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Also </span>/var/log/nginx/test.awk, is as follow : </p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><br></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(82,48,225)"><span style="font-variant-ligatures:no-common-ligatures">################# tst.awk #################</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(213,59,211)"><span style="font-variant-ligatures:no-common-ligatures">BEGIN</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> {</span><span style="font-variant-ligatures:no-common-ligatures">FS</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures">=</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">","</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> }</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">{</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">    </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(51,187,200)">split</span><span style="font-variant-ligatures:no-common-ligatures">(</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">$1,</span><span style="font-variant-ligatures:no-common-ligatures"> m</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">,</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">\ </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">"</span><span style="font-variant-ligatures:no-common-ligatures">)</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">    </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(51,187,200)">gsub</span><span style="font-variant-ligatures:no-common-ligatures">(</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">/</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">/,</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">""</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">,</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">$2</span><span style="font-variant-ligatures:no-common-ligatures">)</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">    </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(51,187,200)">split</span><span style="font-variant-ligatures:no-common-ligatures">(</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">$2,</span><span style="font-variant-ligatures:no-common-ligatures"> a</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">,</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">":"</span><span style="font-variant-ligatures:no-common-ligatures">)</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">    </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(51,187,200)">gsub</span><span style="font-variant-ligatures:no-common-ligatures">(</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">/</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">/,</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">""</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">,</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">$3</span><span style="font-variant-ligatures:no-common-ligatures">)</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">    </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(51,187,200)">split</span><span style="font-variant-ligatures:no-common-ligatures">(</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">$3,</span><span style="font-variant-ligatures:no-common-ligatures"> b</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">,</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">":"</span><span style="font-variant-ligatures:no-common-ligatures">)</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">    </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(51,187,200)">gsub</span><span style="font-variant-ligatures:no-common-ligatures">(</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">/</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">/,</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">""</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">,</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">$4</span><span style="font-variant-ligatures:no-common-ligatures">)</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">    </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(51,187,200)">split</span><span style="font-variant-ligatures:no-common-ligatures">(</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">$4,</span><span style="font-variant-ligatures:no-common-ligatures"> c</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">,</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">":"</span><span style="font-variant-ligatures:no-common-ligatures">)</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">    </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(51,187,200)">gsub</span><span style="font-variant-ligatures:no-common-ligatures">(</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">/</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">/,</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">""</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">,</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">$5</span><span style="font-variant-ligatures:no-common-ligatures">)</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">    </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(51,187,200)">split</span><span style="font-variant-ligatures:no-common-ligatures">(</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">$5,</span><span style="font-variant-ligatures:no-common-ligatures"> d</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">,</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">":"</span><span style="font-variant-ligatures:no-common-ligatures">)</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">    </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(205,121,35)">printf</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">%s</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">,</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">"{"</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(213,59,211)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">    </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(205,121,35)">printf</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">"</span><span style="font-variant-ligatures:no-common-ligatures">\"%s\"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)"> : </span><span style="font-variant-ligatures:no-common-ligatures">\"%s\"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">,"</span><span style="font-variant-ligatures:no-common-ligatures">,</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">a[</span><span style="font-variant-ligatures:no-common-ligatures">1</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">]</span><span style="font-variant-ligatures:no-common-ligatures">,</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> a[</span><span style="font-variant-ligatures:no-common-ligatures">2</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">]</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(213,59,211)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">    </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(205,121,35)">printf</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">"</span><span style="font-variant-ligatures:no-common-ligatures">\"%s\"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)"> : </span><span style="font-variant-ligatures:no-common-ligatures">\"%s\"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">,"</span><span style="font-variant-ligatures:no-common-ligatures">,</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">b[</span><span style="font-variant-ligatures:no-common-ligatures">1</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">]</span><span style="font-variant-ligatures:no-common-ligatures">,</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> b[</span><span style="font-variant-ligatures:no-common-ligatures">2</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">]</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(82,48,225)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">    </span><span style="font-variant-ligatures:no-common-ligatures">#printf "%s",$3 ","</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(82,48,225)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">    </span><span style="font-variant-ligatures:no-common-ligatures">#printf "%s",$5 ","</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(82,48,225)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">    </span><span style="font-variant-ligatures:no-common-ligatures">#printf "%s",$4 ","</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">    </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(205,121,35)">printf</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">\"%s\"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)"> : </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">%s</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">,"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">,</span><span style="font-variant-ligatures:no-common-ligatures">c[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">1</span><span style="font-variant-ligatures:no-common-ligatures">]</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">,</span><span style="font-variant-ligatures:no-common-ligatures"> c[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">2</span><span style="font-variant-ligatures:no-common-ligatures">]</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">    </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(205,121,35)">printf</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">\"%s\"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)"> : </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">%s</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">,"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">,</span><span style="font-variant-ligatures:no-common-ligatures">d[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">1</span><span style="font-variant-ligatures:no-common-ligatures">]</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">,</span><span style="font-variant-ligatures:no-common-ligatures"> d[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">2</span><span style="font-variant-ligatures:no-common-ligatures">]</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">    </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(51,187,200)">split</span><span style="font-variant-ligatures:no-common-ligatures">(m[</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">10</span><span style="font-variant-ligatures:no-common-ligatures">]</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">,</span><span style="font-variant-ligatures:no-common-ligatures"> e</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">,</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(213,59,211)">\"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">"</span><span style="font-variant-ligatures:no-common-ligatures">)</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(213,59,211)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">    </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(205,121,35)">printf</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">" </span><span style="font-variant-ligatures:no-common-ligatures">\"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">reason</span><span style="font-variant-ligatures:no-common-ligatures">\"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">: </span><span style="font-variant-ligatures:no-common-ligatures">\"%s</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)"> </span><span style="font-variant-ligatures:no-common-ligatures">%s</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)"> </span><span style="font-variant-ligatures:no-common-ligatures">%s</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)"> </span><span style="font-variant-ligatures:no-common-ligatures">%s</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)"> </span><span style="font-variant-ligatures:no-common-ligatures">%s\"</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">}</span><span style="font-variant-ligatures:no-common-ligatures">\n</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(195,55,32)">"</span><span style="font-variant-ligatures:no-common-ligatures">,</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> m[</span><span style="font-variant-ligatures:no-common-ligatures">6</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">]</span><span style="font-variant-ligatures:no-common-ligatures">,</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> m[</span><span style="font-variant-ligatures:no-common-ligatures">7</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">]</span><span style="font-variant-ligatures:no-common-ligatures">,</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> m[</span><span style="font-variant-ligatures:no-common-ligatures">8</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">]</span><span style="font-variant-ligatures:no-common-ligatures">,</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> m[</span><span style="font-variant-ligatures:no-common-ligatures">9</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">]</span><span style="font-variant-ligatures:no-common-ligatures">,</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)"> e[</span><span style="font-variant-ligatures:no-common-ligatures">2</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">]</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);min-height:13px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);min-height:13px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);min-height:13px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">}</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)">

























</p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(82,48,225)"><span style="font-variant-ligatures:no-common-ligatures">#############################################</span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p></div><div class="gmail_extra"><br><div class="gmail_quote">On 10 January 2018 at 17:53, mohit Agrawal <span dir="ltr"><<a href="mailto:mohit3081989@gmail.com" target="_blank">mohit3081989@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Thanks Aziz for this, I get your point, but can we do awking in fluentd cons file ? Basically we are looking for realtime awking a nginx error log file, how heavy this would be according to you.</div><div class="gmail_extra"><div><div class="h5"><br><div class="gmail_quote">On 10 January 2018 at 17:44, Aziz Rozyev <span dir="ltr"><<a href="mailto:arozyev@nginx.com" target="_blank">arozyev@nginx.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">If you need parse exactly the same format, as you’ve shown in you question, it’s fairly easy to create something e.g. perl/awk/sed script.<br>
<br>
for instance:<br>
<br>
################# tst.awk #################<br>
BEGIN {FS = "," }<br>
{<br>
    split($1, m, "\ ")<br>
    printf "%s", "{ "<br>
    printf "%s",$2<br>
    printf "%s",$3<br>
    printf "%s",$5<br>
    printf "%s",$4<br>
    printf "reason: %s %s %s %s \"%s\"\n", m[6], m[7], m[8], m[9], m[10]<br>
    print " }”<br>
<br>
}<br>
##############################<wbr>###############<br>
<br>
<br>
result:<br>
<br>
echo 2018/01/10 06:26:31 [error] 13485#13485: *64285471 limiting connections by zone "rl_conn", client: xx.xx.xx.xx, server: <a href="http://www.xyz.com" rel="noreferrer" target="_blank">www.xyz.com</a>, request: "GET /api/xyz HTTP/1.1", host: "<a href="http://www.xyz.com" rel="noreferrer" target="_blank">www.xyz.com</a>" | awk -f /tmp/test.awk<br>
{  client: xx.xx.xx.xx server: <a href="http://www.xyz.com" rel="noreferrer" target="_blank">www.xyz.com</a> host: <a href="http://www.xyz.com" rel="noreferrer" target="_blank">www.xyz.com</a> request: GET /api/xyz HTTP/1.1reason: limiting connections by zone "rl_conn"<br>
 }<br>
<br>
<br>
br,<br>
Aziz.<br>
<div class="m_3250296651361855296HOEnZb"><div class="m_3250296651361855296h5"><br>
<br>
<br>
<br>
<br>
> On 10 Jan 2018, at 14:45, mohit Agrawal <<a href="mailto:mohit3081989@gmail.com" target="_blank">mohit3081989@gmail.com</a>> wrote:<br>
><br>
> Yeah I have tried grok / regex pattern as well. But not extensive success that I achieved. grok didn't work for me, I tried regex then it was able to segregate time , pid, tid, log_level and message. I also need message break up for above pattern<br>
><br>
> On 10 January 2018 at 17:12, Aziz Rozyev <<a href="mailto:arozyev@nginx.com" target="_blank">arozyev@nginx.com</a>> wrote:<br>
> Hi Mohit,<br>
><br>
> check the second reply. I’m not sure that there is a conventional pretty printing<br>
> tools for nginx error log.<br>
><br>
><br>
> br,<br>
> Aziz.<br>
><br>
><br>
><br>
><br>
><br>
> > On 10 Jan 2018, at 14:37, mohit Agrawal <<a href="mailto:mohit3081989@gmail.com" target="_blank">mohit3081989@gmail.com</a>> wrote:<br>
> ><br>
> > Hi Aziz,<br>
> ><br>
> > log_format directive only provides formatting for access log, I am looking to format error.log which doesn't take log_format directive.<br>
> > Above example that I gave is just for nginx error logs.<br>
> ><br>
> > Thanks<br>
> ><br>
> > On 10 January 2018 at 15:26, Aziz Rozyev <<a href="mailto:arozyev@nginx.com" target="_blank">arozyev@nginx.com</a>> wrote:<br>
> > btw, after re-reading the your questing, it looks like you need something like logstash grok filter.<br>
> ><br>
> > br,<br>
> > Aziz.<br>
> ><br>
> ><br>
> ><br>
> ><br>
> ><br>
> > > On 10 Jan 2018, at 11:45, mohit Agrawal <<a href="mailto:mohit3081989@gmail.com" target="_blank">mohit3081989@gmail.com</a>> wrote:<br>
> > ><br>
> > > Hi ,<br>
> > ><br>
> > > I am looking to parse nginx error log so as to find out which particular IP is throttled during specific amount of time on connection throttling  / request throttling. The format looks like :<br>
> > ><br>
> > > 2018/01/10 06:26:31 [error] 13485#13485: *64285471 limiting connections by zone "rl_conn", client: xx.xx.xx.xx, server: <a href="http://www.xyz.com" rel="noreferrer" target="_blank">www.xyz.com</a>, request: "GET /api/xyz HTTP/1.1", host: "<a href="http://www.xyz.com" rel="noreferrer" target="_blank">www.xyz.com</a>"<br>
> > > And the sample that I am looking for is :<br>
> > ><br>
> > > {client: "xx.xx.xx.xx", server: "<a href="http://www.xyz.com" rel="noreferrer" target="_blank">www.xyz.com</a>", host: "<a href="http://www.xyz.com" rel="noreferrer" target="_blank">www.xyz.com</a>", "request": "GET /api/xyz HTTP/1.1", reason: "limiting connections by zone "rl_conn""}<br>
> > > so that I can pass it through ELK stack and find out the root ip which is causing issue.<br>
> > ><br>
> > ><br>
> > > --<br>
> > > Mohit Agrawal<br>
> > > ______________________________<wbr>_________________<br>
> > > nginx mailing list<br>
> > > <a href="mailto:nginx@nginx.org" target="_blank">nginx@nginx.org</a><br>
> > > <a href="http://mailman.nginx.org/mailman/listinfo/nginx" rel="noreferrer" target="_blank">http://mailman.nginx.org/mailm<wbr>an/listinfo/nginx</a><br>
> ><br>
> > ______________________________<wbr>_________________<br>
> > nginx mailing list<br>
> > <a href="mailto:nginx@nginx.org" target="_blank">nginx@nginx.org</a><br>
> > <a href="http://mailman.nginx.org/mailman/listinfo/nginx" rel="noreferrer" target="_blank">http://mailman.nginx.org/mailm<wbr>an/listinfo/nginx</a><br>
> ><br>
> ><br>
> ><br>
> > --<br>
> > Mohit Agrawal<br>
><br>
><br>
><br>
><br>
> --<br>
> Mohit Agrawal<br>
<br>
</div></div></blockquote></div><br><br clear="all"><div><br></div></div></div><span class="HOEnZb"><font color="#888888">-- <br><div class="m_3250296651361855296gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr">Mohit Agrawal<br></div></div></div></div>
</font></span></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr">Mohit Agrawal<br></div></div></div></div>
</div>