tomd
December 22, 2022, 9:17am
1
Hello, I have about 100 RSS feeds that I need to monitor, auto remediate and to store in a searchable repository.
Would 100 URLs with the HTTP input plug-in bog down telegraf?
Can I send the full RSS items to BigQuery and parsed data to Loki or would it be better to have one output and do the processing/routing outside of telegraf?
Any other insights in this use case?
Anaisdg
December 27, 2022, 6:22pm
2
Hello @tomd ,
You can do processing with telegraf as well.
# Execd Processor Plugin
The `execd` processor plugin runs an external program as a separate process and
pipes metrics in to the process's STDIN and reads processed metrics from its
STDOUT. The programs must accept influx line protocol on standard in (STDIN)
and output metrics in influx line protocol to standard output (STDOUT).
Program output on standard error is mirrored to the telegraf log.
Telegraf minimum version: Telegraf 1.15.0
## Caveats
- Metrics with tracking will be considered "delivered" as soon as they are passed
to the external process. There is currently no way to match up which metric
coming out of the execd process relates to which metric going in (keep in mind
that processors can add and drop metrics, and that this is all done
asynchronously).
- it's not currently possible to use a data_format other than "influx", due to
the requirement that it is serialize-parse symmetrical and does not lose any
show original
Heres an example of using the execd plugins:
However I don’t know how to answer this question really because it depends on what type of processing you’re looking to do.
tomd
January 5, 2023, 6:32am
3
Thank you. This is a perfect answer and just the help I needed.
