I am trying to read CSV files on a FTP server (protected with username and password) into Influxdb. These files are produced constantly. I have 1 file for each day plus a the csv file for today, which is filled with data up to now.
Please see below the structure of the csv files:
The URL of the files looks like this:
Is there a way to capture this data in indluxdb?
Many thanks in advance!
Very easy to do with Node-RED. Have you explored that product at all?
Thanks for the tip! I am only using the node-red application in my home assistant setup to automate my zigbee lights. Is there a tutorial out there on how to feed that data into influxdb?
Many thanks and best regards
Not that I am aware of, but I will send you via private message a few links and the Node-RED forum folks can probably help you along if you get stuck.
You can also use Telegraf for this if you want a more performant solution. I agree though Node-Red is the easiest! I am a massive fan!
This file has been truncated.
# Directory Monitor Input Plugin
This plugin monitors a single directory (traversing sub-directories),
and takes in each file placed in the directory. The plugin will gather all
files in the directory at the configured interval, and parse the ones that
haven't been picked up yet.
This plugin is intended to read files that are moved or copied to the monitored
directory, and thus files should also not be used by another process or else
they may fail to be gathered. Please be advised that this plugin pulls files
directly after they've been in the directory for the length of the configurable
`directory_duration_threshold`, and thus files should not be written 'live' to
the monitored directory. If you absolutely must write files directly, they must
be guaranteed to finish writing before the `directory_duration_threshold`.
## Global configuration options <!-- @/docs/includes/plugin_config.md -->
In addition to the plugin-specific configuration settings, plugins support
additional global and plugin configuration settings. These settings are used to
modify metrics, tags, and field or create aliases and configure ordering, etc.
Directory Monitor allows you to monitor a specific directory and reads any csv’s you wish to parse there before placing them in a separate directory
Thank you very much for your help! I have checked out the documentation but I am still unsure how to enter the URL/IP and the username/password of the FTP server next to the path.
Can you help me out here?
@Globgogabgalab, sorry that was my misunderstanding.
So there is currently no telegraf plugin which will allow you to read files directly from an FTP server. So i agree with
@grant1 I woudl use node-red for the easiest approach and then use the InfluxDB plugin for node red to write that data to influxDB. Otherwise you would need to use the execd plugin of Telegraf and write a custom script to copy over the file and parse in telegraf. Note if telegraf can be run directly on your FTP server then you can use the directory plugin.