Invalid number on Exec input

,
1

Hi everyone.
Im using exec to read the following metrics:

dt,entity=SERVICE-F08D806E7EF685CF timeseriesId=“com.dynatrace.builtin:service.responsetime”,metric=56167.75

I had been using exec the entire month to read not numeric values and send them to influxdb with no problem at all. Im almost 100% certain the metric above is OK for the Influx Line Protocol. Im getting the following error when i run telegraf.

2017-12-28T15:52:31Z E! Error in plugin [inputs.exec]: invalid number invalid number invalid number invalid number invalid number invalid number invalid number

I’m stuck here, tried almost everything to solve it with no success.
Thanks in advance.

PD: I do have another Exec running reading string metrics and working like a charm with the same protocol. Thats why i really dont know whats going on. @daniel i tag you here since you have helped me before with this development. Thank you!

2

I can successfully parse the example line. Flipping through the parser code this ugly error can occur in a number of cases, and unfortunately it doesn’t show what the input buffer contains so it is difficult to be sure exactly what is the cause.

Could you try editing your config like this so that the output is also written to a file, run until the error occurs and then upload the file here?

[[inputs.exec]]
   # before 
   #commands = ["sh exec.sh]

   # after 
   commands = ["sh exec.sh | tee /tmp/telegraf.out"]
3

I did what you mentioned but there is no log file anywhere to be found.
Any ideas? Debug is on TRUE.

4

Sorry my example above had a few issues, try this instead:

commands = ['sh -c "sh exec.sh | tee -a /tmp/telegraf.out"']
5

Hi Daniel!
Happy New Year and thanks for your answers!

With the new line you provided, i get the telegraf.out at /tmp/

telegraf.txt (49.2 KB)

I do switched the file type to .txt since i could not upload it as .out
The protocol seems to be alright, maybe there is a problem with the length of some metrics?

EDIT: Just to remember, the error is the following:

E! Error in plugin [inputs.exec]: invalid number invalid number invalid number invalid number invalid number invalid number invalid number

6

It looks like the output file dos line endings, can you try changing your script to output with unix line endings?

I opened an issue on the github issue tracker: Invalid number error when parsing line protocol with dos line endings · Issue #3631 · influxdata/telegraf · GitHub

7

I could solve the problem by quoting the numeric value.
Metrics now have this look:

m,entity=SERVICE-F08D806E7EF685CF timeseriesId=“com.dynatrace.builtin:service.responsetime”,metric=“62523.25”
m,entity=SERVICE-F08D806E7EF685CF timeseriesId=“com.dynatrace.builtin:service.responsetime”,metric=“67909.0”
m,entity=SERVICE-F08D806E7EF685CF timeseriesId=“com.dynatrace.builtin:service.responsetime”,metric=“59529.5”

Although im having a new problem right now.
Current .sh file running by exec looks like this:

#!/bin/bash

python getmetrics.py
cat metrics.csv

When running Telegraf, i get the following:

2018-01-03T14:03:19Z D! Output [influxdb] buffer fullness: 110 / 10000 metrics.
2018-01-03T14:03:19Z D! Output [influxdb] buffer fullness: 110 / 10000 metrics.
2018-01-03T14:03:19Z D! Output [influxdb] wrote batch of 110 metrics in 63.891535ms
2018-01-03T14:03:19Z D! Output [influxdb] wrote batch of 110 metrics in 118.005204ms

But when i do a select * from m_dt on the influx, there is only one line.
It is writing only 1 metric per time, although saying otherwise.

8

You should be able to do a quick fix with the dos2unix util:

#!/bin/bash

python getmetrics.py
dos2unix metrics.csv
cat metrics.csv

I wouldn’t quote the value because it will then be stored as a string.

9

The dos2unix did the work for the invalid number.
Although the problem causing influxdb to write only 1 metric per time is still there.

2018-01-03T14:03:19Z D! Output [influxdb] buffer fullness: 110 / 10000 metrics.
2018-01-03T14:03:19Z D! Output [influxdb] buffer fullness: 110 / 10000 metrics.
2018-01-03T14:03:19Z D! Output [influxdb] wrote batch of 110 metrics in 63.891535ms
2018-01-03T14:03:19Z D! Output [influxdb] wrote batch of 110 metrics in 118.005204ms

It does say its writing more, but only 1 line appears per writing in the influxdb.

10

At any timestamp each measurement+tagset can only have one value for each field in InfluxDB. This is a feature though, as it allows you to change values and makes writing idempotent so you don’t need to worry about duplicate insertions. There are a couple changes commonly made to store all the data in cases like these, depending on what is being stored and the storage needs:

  • Add a new tag to each line that differentiates it from other points occurring at the same time
  • Aggregate the data for over each timestamp
  • Change the timestamp of each line