Appendable JSON input format?

Telegraf
1

I have an event log like so:

{"type": "page", "action": "create", "user": "fred", "time": 0.23}
{"type": "page", "action": "update", "user": "fred", "time": 0.31}

Notably, each event adds a new line / a new JSON object; each line is valid JSON, but the file as a whole isn’t. Can telegraf parse that?

  • Parsing as-is gets “invalid character ‘{’ after top-level value”
  • sticking a “,” between them gets “invalid character ‘,’ after top-level value”
  • sticking a “[” at the start gets “unexpected end of JSON input” (since I’m constantly appending new entries, I can’t add a matching “]” at the end)

I can’t see a way to make the file as a whole be valid JSON, but also for it to be appendable :frowning:

2

Is this a file?

What input plugin are you using?
You might be able to accomplish by using the file input plugin and changing the data format to JSON. You will have to fiddle around with it to test

1 Like
3

How did you manage to solve your issue @shish ? I am in a similar situation reading from an Amazon Kiensis Data Stream where each record contains hundreds of JSONs so I’m getting the invalid character ‘{’ after top-level value

4

I never solved this issue :frowning: I sidestepped it by having my tools write to influxdb directly (ie over TCP) using the native influx format