Correct way to config telegraf execd to read stdout of python script generating line protocol

[Bill_L]

I am attempting to use telegraf execd with a python script running inside a virtual environment in windows that outputs line protocol once per second (reading from a wss websocket). If I use the influx api, the lines will write to database.

I created a batch script, to activate and run, but when I point to it from the config file, this seems to cause the agent to immediately stop the inputs.

Can anyone point me in the right direction? I tried to write a command instead of using a batch file, but I don’t have enough experience with this, and am not sure if it’s even possible to have more than one command in the config file?

The batch looks like this:
@ECHO OFF
call C:\Users\wll\src\venv\alas\Scripts\activate.bat
C:\Users\wll\src\wss.py
pause >nul

Config file points to it:
[[inputs.execd]]
command = [‘C:\Program Files\Telegraf\wss.bat’]

Cheers and thanks!

PS C:\Windows\System32> telegraf --debug --config C:"Program Files"\Telegraf\telegraf.conf --test
2023-07-05T01:24:11Z I! Loading config: C:\Program Files\Telegraf\telegraf.conf
2023-07-05T01:24:11Z I! Starting Telegraf 1.27.1
2023-07-05T01:24:11Z I! Available plugins: 237 inputs, 9 aggregators, 28 processors, 23 parsers, 59 outputs, 4 secret-stores
2023-07-05T01:24:11Z I! Loaded inputs: execd
2023-07-05T01:24:11Z I! Loaded aggregators:
2023-07-05T01:24:11Z I! Loaded processors:
2023-07-05T01:24:11Z I! Loaded secretstores:
2023-07-05T01:24:11Z W! Outputs are not used in testing mode!
2023-07-05T01:24:11Z I! Tags enabled: host=wll-instance-1
2023-07-05T01:24:11Z D! [agent] Initializing plugins
2023-07-05T01:24:11Z D! [agent] Starting service inputs
2023-07-05T01:24:11Z I! [inputs.execd] Starting process: C:\Program Files\Telegraf\wss.bat
2023-07-05T01:24:11Z D! [agent] Stopping service inputs
2023-07-05T01:24:16Z E! [inputs.execd] Error in plugin: read |0: file already closed
2023-07-05T01:24:16Z E! [inputs.execd] Error in plugin: error reading stderr: read |0: file already closed
2023-07-05T01:24:16Z I! [inputs.execd] Process C:\Program Files\Telegraf\wss.bat shut down
2023-07-05T01:24:16Z D! [agent] Input channel closed
2023-07-05T01:24:16Z D! [agent] Stopped Successfully
2023-07-05T01:24:16Z E! [telegraf] Error running agent: input plugins recorded 2 errors

[Bill_L]

In case this might help anyone else (without much experience like myself), my issue was more a lack of understanding of configuration file setting generally than telegraf itself. steps for a solution that worked on my end:

activate python virtual environment in powershell

exced plugin: command =[“path_to_python_within_venv”, “path_to_script”]
need to use double backslashes \ in the path.

an online toml linter was useful as is the execd readme

Only issue now is that batches are not being written until:

2023-07-05T08:49:30Z E! [inputs.execd] stderr: “sent 1011 (unexpected error) keepalive ping timeout; no close frame received”

Then a whole bunch of batches write.

After ctrl-c:
2023-07-05T08:49:49Z I! [agent] Hang on, flushing any cached metrics before shutdown

And a bunch more writes from the cache.

The rest of the time nothing is getting through:
2023-07-05T08:48:58Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100 metrics

Adding wss functionality to socket_listener is a little out of my league (unless I somehow missed this or another pre-built solution). Getting closer with this workaround

[jpowers]

@Bill_L

2023-07-05T08:49:30Z E! [inputs.execd] stderr: “sent 1011 (unexpected error) keepalive ping timeout; no close frame received”

If I am not mistaken as this error is on stderr and should be coming from your application itself.

The rest of the time nothing is getting through:

What signal are you using to call your external script? Are you expecting it to report metrics. It sounded like that was the case, when you said every second, in which case I assume you are using signla of none? Or do you want telegraf to call your script at every collection interval via a newline (e.g. signal = stdin).

[Bill_L]

yes, thanks! I’ve got it up and running now.

signal = none. haven’t confirmed it yet, but the error must be from the websocket client.

for telegraf to see what I was seeing in stdout, the script needed to flush each write with: sys.stdout.flush()

the writes to stderr might have also caused a flush of stdout