How to collect stock market data from a datafeed (EODHistorical) through Telegraf?

Hi Guys, I am new here and in InfluxDB/Telegraf. Currently I use a python script to daily retrieve stock market data from EODHistorical (a supplier of market data) through their APIs. I collect the data and save it in CSV format in a Vultr server.

I would like to change this process of collection using Telegraf and instead CSVs, save the data in InfluxDB. I already installed InfluxDB and Telegraf, but I am stuck here in how to create a Telegraf plug in to do this (or use my python script in Telegraf so it can collect the data - i know it is in Go, but maybe there is an easier way to do it ?).

Does anybody has a similar template for this, maybe the same thing but collecting the data from another provider and then I can adapt the plugin to connect to my provider ?

Thanks for the help !

Best Regards

Flavio

This should theoretically be possible. The API returns JSON? You need the inputs.http plugin:

https://docs.influxdata.com/telegraf/v1.20/plugins/#http

How easy it is to process the data in Telegraf depends on how the data is structured that the API request returns. Expect to invest some time to configure the json parser properly.


If you prefer to use your Python code as a starting point, that would theoretically work too. For this you would have to adjust the output of your Python code ideally so that influx line protocol is output there and then call the script with the inputs.exec or inputs.execd plugin:

https://docs.influxdata.com/telegraf/v1.20/plugins/#execd


Which approach of both is easier, I can not say at a distance, both should work.

1 Like

Hey Frank thank you very much for the answer ! I will explore your suggestions. Regards