Is possible to read only new CSV files via Telegraf?

JMU · February 14, 2022, 3:34pm

If there is a directory where every X time a new .csv file is generated, can Telegraf read only the new files and write on InfluxDB.

I appreciate the answers and links of examples if it is necessary.

Thanks in advance

Anaisdg · February 14, 2022, 11:24pm

Hello @JMU,
You can use the tail plugin to tail an existing CSV.

github.com

influxdata/telegraf/blob/master/plugins/inputs/tail/README.md

# Tail Input Plugin

The tail plugin "tails" a logfile and parses each log message.

By default, the tail plugin acts like the following unix tail command:

```shell
tail -F --lines=0 myfile.log
```

- `-F` means that it will follow the _name_ of the given file, so
that it will be compatible with log-rotated files, and that it will retry on
inaccessible files.
- `--lines=0` means that it will start at the end of the file (unless
the `from_beginning` option is set).

see <http://man7.org/linux/man-pages/man1/tail.1.html> for more details.

The plugin expects messages in one of the [Telegraf Input Data
Formats](../../../docs/DATA_FORMATS_INPUT.md).

This file has been truncated. show original

But for new files I think you’d have to use the execd input plugin and create your own solution

github.com

influxdata/telegraf/blob/master/plugins/inputs/execd/README.md

# Execd Input Plugin

The `execd` plugin runs an external program as a long-running daemon.  The
programs must output metrics in any one of the accepted [Input Data Formats][]
on the process's STDOUT, and is expected to stay running. If you'd instead like
the process to collect metrics and then exit, check out the [inputs.exec][]
plugin.

The `signal` can be configured to send a signal the running daemon on each
collection interval. This is used for when you want to have Telegraf notify the
plugin when it's time to run collection. STDIN is recommended, which writes a
new line to the process's STDIN.

STDERR from the process will be relayed to Telegraf as errors in the logs.

## Global configuration options <!-- @/docs/includes/plugin_config.md -->

In addition to the plugin-specific configuration settings, plugins support
additional global and plugin configuration settings. These settings are used to
modify metrics, tags, and field or create aliases and configure ordering, etc.

This file has been truncated. show original

Plugin Spotlight: Exec & Execd | InfluxData.

Topic		Replies	Views
Telegraf read, handle, write big log files Telegraf telegraf	8	4079	June 6, 2019
Plugin file for telegraf Telegraf telegraf	2	743	March 28, 2019
Reading csv using telegraf	3	1651	January 20, 2020
Will tail plugin work for this scenario Telegraf influxdb , telegraf , csv , tail	4	796	March 4, 2022
Data ingestion continuously Telegraf telegraf	2	896	November 25, 2019

Is possible to read only new CSV files via Telegraf?

Related topics