Syslog output removes SIDs greater than 32 characters

Telegraf
1

While I was testing the syslog output, I noticed that when SIDs are greater than 32 characters, the output doesnot contain SIDs at all.

The telegraf output/input configuration is given below.

[[inputs.tail]]
        from_beginning = true
        files = ['/home/ec2-user/syslog.line']
        data_format = "influx"

[[outputs.syslog]]
        framing = "non-transparent"
        trailer = "LF"
        sdids = ["fce4b930-e7a4-4228-a05f-a33@41058"]
        sdparam_separator = "_"
        default_severity_code = 5
        default_facility_code = 1
        address = "tcp://logs-01.loggly.com:514"
        #tls_cert = "/etc/rsyslog.d/keys/ca.d/logs-01.loggly.com_sha12.crt"
        namepass = ['loggly_logs']
        tagexclude = ['dc', 'env', 'host', 'path', 'qcinstance']

When my input file have following entry

loggly_logs,appname=Woohoo-Container fce4b930-e7a4-4228-a05f-a33@41058_tag=“Container”,msg=“Testing containers”

I can see in the tcpdump there are no Structured Datas in the packet

192.168.38.222.47956 > 52.24.177.168.514: Flags [P.], cksum 0xcdce (incorrect → 0x18db), seq 452503436:452503533, ack 2968423998, win 211, options [nop,nop,TS val 3089823145 ecr 607085399], length 97
0x0000: 4500 0095 50e0 4000 ff06 5d3b c0a8 26de E…P.@…];…&.
0x0010: 3418 b1a8 bb54 0202 1af8 a78c b0ee 8e3e 4…T…>
0x0020: 8018 00d3 cdce 0000 0101 080a b82a f5a9 …*…
0x0030: 242f 6357 3c31 333e 3120 3230 3230 2d30 $/cW<13>1.2020-0
0x0040: 342d 3137 5430 363a 3338 3a33 375a 2069 4-17T06:38:37Z.i
0x0050: 702d 3139 322d 3136 382d 3338 2d32 3232 p-192-168-38-222
0x0060: 2057 6f6f 686f 6f2d 436f 6e74 6169 6e65 .Woohoo-Containe
0x0070: 7220 2d20 6c6f 6767 6c79 5f6c 6f67 7320 r.-.loggly_logs.
0x0080: 2d20 5465 7374 696e 6720 636f 6e74 6169 -.Testing.contai
0x0090: 6e65 7273 00 ners.

When I remove one character from the SDID like below

loggly_logs,appname=Woohoo-Container fce4b930-e7a4-4228-a05f-a33@1058_tag=“Container”,msg=“Testing containers”

I can see the structured data in the packet.

06:38:00.000912 IP (tos 0x0, ttl 255, id 33480, offset 0, flags [DF], proto TCP (6), length 198)
192.168.38.222.60514 > 75.101.233.97.514: Flags [P.], cksum 0x1d06 (incorrect → 0xf11a), seq 83451929:83452075, ack 4149530591, win 211, options [nop,nop,TS val 2413407202 ecr 801391507], length 146
0x0000: 4500 00c6 82c8 4000 ff06 dc1b c0a8 26de E…@…&.
0x0010: 4b65 e961 ec62 0202 04f9 6019 f754 cfdf Ke.a.b…`…T…
0x0020: 8018 00d3 1d06 0000 0101 080a 8fd9 abe2 …
0x0030: 2fc4 4393 3c31 333e 3120 3230 3230 2d30 /.C.<13>1.2020-0
0x0040: 342d 3137 5430 363a 3337 3a34 385a 2069 4-17T06:37:48Z.i
0x0050: 702d 3139 322d 3136 382d 3338 2d32 3232 p-192-168-38-222
0x0060: 2057 6f6f 686f 6f2d 436f 6e74 6169 6e65 .Woohoo-Containe
0x0070: 7220 2d20 6c6f 6767 6c79 5f6c 6f67 7320 r.-.loggly_logs.
0x0080: 5b66 6365 3462 3933 302d 6537 6134 2d34 [fce4b930-e7a4-4
0x0090: 3232 382d 6130 3566 2d61 3333 4031 3035 228-a05f-a33@105
0x00a0: 3820 7461 673d 2243 6f6e 7461 696e 6572 8.tag="Container
0x00b0: 225d 2054 6573 7469 6e67 2063 6f6e 7461 "].Testing.conta
0x00c0: 696e 6572 7300 iners.

What I observed is when the length of SDID is less than 32 characters the structured data is present in the packet. When it exceeds 32 characters it is absent.

I have a 42 character long SDID. Is this a bug. Is there any workaround for this.

Thanks,
Robert

2

I haven’t looked too closely into the cause of this within the code, but the ABNF in RFC 5424 does say the max length of the SD-NAME is 32 characters.

As a workaround, you might be able to use the “left” function of the string processor, but you would need to adjust your input around a bit since it only operates on the field values.

3

Thanks for the reply.

I’m not sure if left function will help me here. As per documentation left function helps to Trims strings based on width.

It may help me to truncate the SDID and add it into my structured data. However it will not be the desired value.

Or am I missing something here.

Please clarify.

4

Sorry for the late response, it seems you will need to shrink down the SDID somehow to fit in 32 chars. Using left is one way you could do that but there are other ways as well. However, it doesn’t seem that it is allowed to have the full unmodified value and still be a valid syslog message.