Json into Telegraf not working

[poeterdebier]

Hi Guys,

I try to get some data into my InfluxDB through Telegraf. The data is from a p1 monitor application (which is called ‘p1 monitor’. P1 monitor has a couple of API calls which I cannot get to work into Telegraf. Obviously I am doing something wrong. Can somebody help me. Let me know if there is need for some extra information.

This is what I have in the telegraf.conf (indents are a little weird due to copying and pasting)

[[inputs.http]]
urls = [
 "http://192.168.2.5/api/v1/powergas/day", 
]
method = "GET"
data_format = "json"

Also:

[[inputs.httpjson]]
servers = [
 "http://192.168.2.5/api/v1/smartmeter",
 "http://192.168.2.5/api/v1/status",
 "http://192.168.2.5/api/v1/powergas/day",
 "http://192.168.2.5/api/v1/powergas/hour?limit=10&sort=asc&json=object&round=on&starttime=2018-01-03 12:03:55",
 "192.168.2.5/api/v1/powergas/minute?limit=1",
 "192.168.2.5/api/v1/powergas/minute?limit=1&json=object",
 "192.168.2.5/api/v1"
]
method = "GET"

As you can see I tried some options already but everything is giving me errrors:

2019-12-13T18:14:55Z E! [inputs.httpjson] Error in plugin: must be an object or an array of objects
2019-12-13T18:14:55Z E! [inputs.httpjson] Error in plugin: must be an object or an array of objects
2019-12-13T18:14:55Z E! [inputs.http] Error in plugin: [url=http://192.168.2.5/api/v1/powergas/day]:         must be an object or an array of objects
2019-12-13T18:14:55Z E! [inputs.httpjson] Error in plugin: must be an object or an array of objects
2019-12-13T18:14:55Z E! [inputs.http] Error in plugin: [url=http://192.168.2.5/api/v1/smartmeter]: must be an object or an array of objects

The json information I am trying to get looks like the following:

[rawkode]

Sorry @poeterdebier, but this will be difficult to help with unless you can provide the full JSON payload.

Can you curl the URL from your terminal and paste the JSON here please?

My initial thought is you need to specify json_query to the part of the JSON response that has your values to store

More here: telegraf/README.md at master · influxdata/telegraf · GitHub

[poeterdebier]

He Rawkode,

when I curl the JSON I get the following:

[["2019-12-14 15:50:25", 1576335025, 0, 1117.87, 1304.513, 0.0, 0.0, "D", 256, 0, 1041.323]["2019-12-14 15:50:16", 1576335016, 0, 1117.869, 1304.513, 0.0, 0.0, "D", 266, 0, 1041.323], ["2019-12-14 15:50:07", 1576335007, 0, 1117.869, 1304.513, 0.0, 0.0, "D", 282, 0, 1041.308], ["2019-12-14 15:49:58", 1576334998, 1, 1117.868, 1304.513, 0.0, 0.0, "D", 273, 0, 1041.308], ["2019-12-14 15:49:48", 1576334988, 1, 1117.867, 1304.513, 0.0, 0.0, "D", 265, 0, 1041.308], ["2019-12-14 15:49:39", 1576334979, 1, 1117.867, 1304.513, 0.0, 0.0, "D", 255, 0, 1041.308], ["2019-12-14 15:49:30", 1576334970, 1, 1117.866, 1304.513, 0.0, 0.0, "D", 273, 0, 1041.308], ["2019-12-14 15:49:21", 1576334961, 1, 1117.865, 1304.513, 0.0, 0.0, "D", 248, 0, 1041.308], ["2019-12-14 15:49:11", 1576334951, 1, 1117.865, 1304.513, 0.0, 0.0, "D", 257, 0, 1041.308], ["2019-12-14 15:49:02", 1576334942, 1, 1117.864, 1304.513, 0.0, 0.0, "D", 251, 0, 1041.308], ["2019-12-14 15:48:53", 1576334933, 1, 1117.863, 1304.513, 0.0, 0.0, "D", 259, 0, 1041.308], ["2019-12-14 15:48:43", 1576334923, 1, 1117.863, 1304.513, 0.0, 0.0, "D", 202, 0, 1041.308], ["2019-12-14 15:48:34", 1576334914, 1, 1117.862, 1304.513, 0.0, 0.0, "D", 211, 0, 1041.308], ["2019-12-14 15:48:25", 1576334905, 1, 1117.862, 1304.513, 0.0, 0.0, "D", 243, 0, 1041.308], ["2019-12-14 15:48:16", 1576334896, 1, 1117.861, 1304.513, 0.0, 0.0, "D", 218, 0, 1041.308], ["2019-12-14 15:48:07", 1576334887, 1, 1117.861, 1304.513, 0.0, 0.0, "D", 209, 0, 1041.308], ["2019-12-14 15:47

When converted to pretty print see attached picture. I cannot get the formatting right in this post formatting (tips for that?).

[rawkode]

I’m not sure if that can be processed by Telegraf, the JSON data isn’t keyed / structured.

Do you have control over it?

[poeterdebier]

Nope, no control. Only the API / link. So it is JSON but you cannot actually do something with this? Some users in the forum from p1 monitor mentioned that they were able to get the data to Influx through Telegraf, but they did not provide detailed enough information/configuration. That is why a thought to try here. Provided an additional screenshot just in case.

Thanks in any case,

Regards, Piet

[rawkode]

Sorry, but I’m not aware of how to parse this. Maybe @daniel will, but you’ll need to wait until next week for him to take a look.

[poeterdebier]

Thanks anyway for the effort.

Happy Holidays

[poeterdebier]

He Guys I managed this a while ago and it was actually quite simple. My biggest mistake was that I was continuously focused on “write” message when running:

telegraf --config /users/somethingsomething/telegraf.conf

But this is actually not showed here as an entry. Only errors and the stuff during startup. So the plugin was actually working for a while but I did just not see it. I would have noticed it sooner if I had looked at the Influx/Chronograf log but I didn’t.

The JSON help page (http://192.168.2.5/api/v1/status/help) gave me the following link:

“/api/v1/status?json=object”

which eventually became:

All data is flowing into Influx now.

greetz Piet

[Jeffry]

Hi Piet,

Trying to get my P1 Monitor data into Influx / Grafana but having almost same issue it seams. Getting the following errors in Telegraf log:

2020-01-01T14:10:40Z E! [inputs.httpjson] Error in plugin: must be an object or an array of objects
2020-01-01T14:10:45Z E! [inputs.httpjson] Error in plugin: Get http://192.168.178.50/api/v1/smartmeter: net/http: request canceled (Client.Timeout exceeded while awaiting headers)

When looking at the telegraf database entries in InfluxDB explorer I do not see any P1Monitor data so any idea where I need to start. Using same config settings as you described in your last screenshot, only added http:// in front of location else I got at protocol error

Regards,
Jeffry

[poeterdebier]

Hi Jeffry,

Are you able to read data from Influx using Grafana? Just asking to make sure that Influx is running properly and that Grafana is also setup properly.

As a new user I cannot upload files so ultimately I can give you a link to my current telegraf.conf file which is working for me at the moment. Compare files to see if you have some different settings or some settings are still commented. Let me know if necessary.

Did you setup the output (line 97)
Did you setup the data setup to “json” (line 2613)? Sorry this you cannot see in the screenshot.

data_format = "json"

Regards, Piet

[Jeffry]

Hi Piet,

Thanks for the reply. Yes I am able to read data from Influx using Grafana. Importing from both Home Assistant and Glance without issues.

In regards to data_format I have the below listed. If you could share a link to compare that would be appreciated if possible. Running all in a Hassio docker setup by the way.

# Read formatted metrics from one or more HTTP endpoints

[[inputs.http]]

## One or more URLs from which to read formatted metrics

urls = [
“http://192.168.178.50/api/v1/smartmeter”,
“http://192.168.178.50/api/v1/status”,
]

## HTTP method

method = "GET"
data_format = "json"

[Jeffry]

Not sure what changed but error message has changed

2020-01-02T22:15:45Z E! [inputs.http] Error in plugin: Get URL://192.168.178.50/api/v1/smartmeter: net/http: request canceled (Client.Timeout exceeded while awaiting headers)

[poeterdebier]

Here it is:

https://www.dropbox.com/s/5nbv5nuzv23t5iu/telegraf.conf.txt?dl=0

I noticed that I made some mistake posting the screenshot. This must be still a screenshot for when I was testing/failing. You will see this when looking at the telegraf.conf file.

I also put http:// in front of the url but the biggest mistake was that I use “inputs.http” instead of “inputs.httpjson”. Sorry for that I will change the screenshot to the correct one so that other people will not copy my mistakes.

Gr Piet

[poeterdebier]

Cannot edit the post anymore. So here is a new (and this time correct) screenshot.

So just to be clear: the screenshot in the post from December 27th is wrong!

Gr Piet