Input Plugin Exec giving Invalid Field Format

,
1

Hi everyone.
I’m using Exec with a python script to download some metrics from Dynatrace and put them into Influxdb.
My script gives this output:

metrics,timeseriesId=ruxit.jmx.Hadoop.HDFS:BlocksRead displayName=“Blocks”,detailedSource=“Apache Hadoop HDFS”

I have read the Influx Line Protocol plus some examples i found on the internet and this output SHOULD be fine. But when i try to make it work:

E! Error in plugin [inputs.exec]: metric parsing error, reason: [invalid field format], buffer: [metrics,timeseriesId=rux], index: [56] metric parsing error, reason: [invalid field format], buffer: [metrics,timeseriesId=ruxit.perfmon.apppoolV1:Requests Failed disp], index: [60]d",detailedSource=“ruxit.perfmon.apppoolV1”

  • Plugin: inputs.exec, Collection 1

I have been a long time trying to solve this but can’t find the answer.

2

I tried to reproduce your issue but it is parsing okay for me, I did fix the quotes to be ascii but I think that is just an artifact of this webpage:

[[inputs.exec]]
   commands = ["sh ./test.sh"]
   timeout = "25s"
   data_format = "influx"

$ cat test.sh
echo 'metrics,timeseriesId=ruxit.jmx.Hadoop.HDFS:BlocksRead displayName="Blocks",detailedSource="Apache Hadoop HDFS"'
$ sh test.sh
metrics,timeseriesId=ruxit.jmx.Hadoop.HDFS:BlocksRead displayName="Blocks",detailedSource="Apache Hadoop HDFS"
$ tg --test
2017/12/11 12:03:12 I! Using config file: /home/dbn/.telegraf/telegraf.conf
* Plugin: inputs.exec, Collection 1
> metrics,timeseriesId=ruxit.jmx.Hadoop.HDFS:BlocksRead,host=loaner displayName="Blocks",detailedSource="Apache Hadoop HDFS" 1513022593000000000
3

Hi Daniel, thanks for you answer.
I have tried simulating exactly what you did and it worked, but it is still not working when I dont simulate it and use the real script/metrics.

This is the script listed on the exec configuration:

sed -i ‘s/*/"/g’ metrics.csv
cat metrics.csv

The sed command is to replace * with " to wrap the strings since the python script was giving me problems to print " into the file.

This is what the metrics.csv look like before being read by exec:

metrics,ruxit.jmx.Hadoop.HDFS:BlocksRead displayName=“Blocks”,detailedSource=“Apache Hadoop HDFS”
metrics,ruxit.jmx.Hadoop.HDFS:BlocksRemoved displayName=“Removed”,detailedSource=“Apache Hadoop HDFS”
metrics,ruxit.jmx.Hadoop.HDFS:BlocksReplicated displayName=“Replicated”,detailedSource=“Apache Hadoop HDFS”

and so on…

Im getting this error when trying:

2017-12-12T13:11:54Z E! Error in plugin [inputs.exec]: metric parsing error, reason: [missing tag value], buffer: [metrics,ruxit.jmx.Hadoop.HD], index: [40]displayName=“Blocks”,detailedSource=“Apache Hadoop HDFS”

and so with every line.

Im stuck here, i dont understand why if i echo a line with the same protocol it would work, but doing it with the entire program and python scripts wont. Thanks for your help on advance.

4

Hi Alexis,

I think you are having an incorrect line protocol output.

metrics,ruxit.jmx.Hadoop.HDFS:BlocksReplicated displayName=“Replicated”,detailedSource=“Apache Hadoop HDFS”

You are not having a tag name provided in the output. You should try to ouput it with a tag name to avoid the [missing tag value] error.

metrics,timeseriesId=ruxit.jmx.Hadoop.HDFS:BlocksReplicated displayName=“Replicated”,detailedSource=“Apache Hadoop HDFS”

So that it is in proper line protocol format. Hope it helps.

5

Also remember not quote the tag value:

metrics,timeseriesId=ruxit.jmx.Hadoop.HDFS:BlocksReplicated displayName=“Replicated”,detailedSource=“Apache Hadoop HDFS”

If you have spaces in a tag value use backslash to escape them.

1 Like
6

@daniel and @4fz
Thanks both of you for your help. Now its working like a charm.
Although i have one problem i dont know if you will be able to help me, but cant lose anything by asking :slight_smile:

I did implement this on my host (notebook running ubuntu) and worked with no problem at all.
But the idea is to run this implementation on a EC2 on AWS.
After installing everything exactly the same, including configuration files and etc, its starts working as it should, but after a minute or so, i get this problem.

E! Error in plugin [inputs.exec]: metric parsing error, reason: [missing fields], buffer: [dt,timeseriesId=ruxit.python.elasticsearch:node.indice], index: [54]

Im pretty sure there arent any missing fields, since im running same metrics on my host and having no problem at all. Any idea?

Thanks again for all the help.

7

Looking over the code, this means that the end of the line was reached while parsing the measurement or tags. So it looks to me that there were no fields. Maybe you can check for this in your python code and write debugging info to a log file?