Some events get lost between telegraf and influx

Telegraf
,
1

Hello everybody,
I am developing a Go application that sends events (logs) to a Telegraf instance, configured with the network_socket plugin and listening to TCP port 8094.

The application sends logs using the influxdb line protocol (the client was developed by me)

Unfortunately not all my events show up in the influx instance (let’s say 1/4 of total events).
Following the advice of a developer in the influxdb Slack channel I have grabbed some TCP dumps of the entire data flow (app->telegraf and telegraf->influx)

I have checked telegtraf and influx logs and I didn’t find any error.

tcpdump_influx: tcpdump_influx - Pastebin.com
tcpodump_telegraf: tcpdump_telegraf - Pastebin.com
select * from log:

> select * from log
name: log
time                Command        command          componentName    host       level message                         parserName     path                 source            version
----                -------        -------          -------------    ----       ----- -------                         ----------     ----                 ------            -------
1498997687733343800                SetCodeStatus    SetCodeStatus    influxline 4i    Registering component                                               ComponentResolver SingleCode commands - 1.0.0-alfa - SingleCode commands component
1498997687733842900 GetBookedItems GetConfiguration GetConfiguration influxline 4i    Registering component           GetBookedItems                      ComponentResolver Node commands - 1.0.0-alfa - Node commands management components
1498997687734350700 AppendItems    SetLocalTime     SetLocalTime     influxline 4i    Registering component           SetItems                            ComponentResolver Node commands - 1.0.0-alfa - Node commands management components
1498997687735846100 AGEXIO         SetNtin          SetNtin          influxline 4i    Startup                         AGEXIO                              ETS               Node commands - 1.0.0-alfa - Node commands management components
1498997688739575400                                                  influxline 4i    Registered block dispatch paths                /SetItems/*/SetItems engine.Line
1498997792084983300                SetCodeStatus    SetCodeStatus    influxline 4i    Registering component                                               ComponentResolver SingleCode commands - 1.0.0-alfa - SingleCode commands component
1498997792085978300 AppendItems    SetLocalTime     SetLocalTime     influxline 4i    Registering component           SetItems                            ComponentResolver Node commands - 1.0.0-alfa - Node commands management components
1498997792088480100 AGEXIO         SetNtin          SetNtin          influxline 4i    Startup                         AGEXIO                              ETS               Node commands - 1.0.0-alfa - Node commands management components
1498997793090443800                                                  influxline 4i    Registered block dispatch paths                /Legacy/*/Legacy     engine.Line

What could I be missing?

Thank you

2

Picking the first couple points in the log:

log,level=4i message="Registering component",version="Legacy commands - 1.0.0-alfa - Legacy commands management components",source="ComponentResolver",componentName="Legacy",command="Legacy" 1498997792085978300
log,level=4i Command="Legacy",source="CommandResolver",message="Registering parser",parserName="Legacy" 1498997792085978300
log,level=4i message="Registering component",command="Echo",version="Node commands - 1.0.0-alfa - Node commands management components",source="ComponentResolver",componentName="Echo" 1498997792085978300

You’re writing the same measurement and tags, with the same timestamp.

This is a duplicate point. With duplicate points, the last set of fields written wins. Frequently Asked Questions | InfluxData Documentation Archive

1 Like
3

you are right, I was not expecting Go’s time.Now() to yield the same timestamp for subsequent calls.
And, I was not aware of influx duplicate point policy (silent overwrite).

I am now manually handling duplicate timestamp as described in the article you linked.

Thank you for the help

1 Like