sflow module and increasing useragent

[Mark Moseley]

On Wed, Aug 10, 2011 at 12:42 PM, Neil Mckee <neil.mckee.ca at gmail.com> wrote:
>
> On Aug 10, 2011, at 9:55 AM, Mark Moseley wrote:
>
>> On Tue, Aug 9, 2011 at 10:37 PM, Neil Mckee <neil.mckee.ca at gmail.com> wrote:
>>> It looks like it may be (erroneously!) ignoring that limit and sending the whole user-agent string.  Thanks for pointing it out.
>>>
>>> If you are using sflowtool to print the output at the collector then it's probably chopping the field there,  so you would just need to tweak sflowtool/src/sflow.h file and recompile sflowtool.
>>>
>>> The user-agent can be kilobytes long in some cases.   How many bytes is enough?   Please submit comments to the HTTP thread on http://groups.google.com/group/sflow.
>>>
>>> Neil
>>>
>>>
>>> On Aug 9, 2011, at 6:21 PM, Mark Moseley wrote:
>>>
>>>> If I wanted to increase the amount of bytes logged by the sflow module
>>>> for the user agent to, say, 128 bytes, what would I need to change?
>>>>
>>>> I tried changing this:
>>>>
>>>> (in ngx_http_sflow.h)
>>>> #define SFLHTTP_MAX_USERAGENT_LEN 128
>>>>
>>>> but I still get the user agent truncated at 64 bytes. Nor does it seem
>>>> like SFLHTTP_MAX_USERAGENT_LEN is used anywhere in the code (or any
>>>> SFLHTTP_MAX_* define).
>>>>
>>>> I'm sure I could whack on it long enough to get it to work, but I ask
>>>> mainly because I don't want to cause some buffer/format/etc overflow
>>>> somewhere else down the line. Thanks!
>>
>> Thanks for the info. That did the trick. I wrongly assumed the limit
>> was being applied in the module itself.
>>
>> BTW, is there anywhere that has info on APIs for host sflow? I've been
>> googling to no avail. The perl Net::sFlow module chokes badly
>> (presumably just expecting standard sflow), and I've not been able to
>> track down a python module (which is what I'm really interested in).
>> Just asking since we have this thread going, but I can repost to the
>> google group if you think it'd be better asked there. Thanks!
>
> Do you mean you are looking for a Python equivalent of what sflowtool.c does?
>
> The data is all XDR-encoded,  so you might start with Python's  "xdrlib".   However it may be easier and more compact to just unpack the data manually.  The C implementation for Ganglia-gmond is an example of that.  It is much more compact than sflowtool.c,  and might be a better place to start if you just want the sFlow-HOST structures:
> http://ganglia.svn.sourceforge.net/viewvc/ganglia/trunk/monitor-core/gmond/sflow.c?revision=2638&view=markup
> (see the "process_sflow_datagram()" function near the bottom.)
>
> sFlow XDR specs that have been finalized are published here:
> http://www.sflow.org/developers/specifications.php
>
> Neil
>
> P.S. The Perl library should be able to skip over structures it doesn't recognize,  so you might want to point that out to it's author(s).

Cool, thanks, I'll take a look at that -- though working with XDR
looks like it could be somewhat painful :)

On a related note, I get a trickle of errors like this in the nginx error log:

sFlow agent error: sfl_agent_error: receiver: flow sample too big for datagram

which must be hitting the in-code limit of 1400 bytes. Would it be
quite horrible if I were to bump up SFL_DEFAULT_DATAGRAM_SIZE over
1460? I'm assuming under normal conditions, it'll just fragment, which
I'm ok with at the rate they're occurring now. But again, I'm worried
about some data structure in the code (that my casual reading of the
code isn't turning up) will overflow.

Cool module, btw!

Thanks!