Telegraf Postgres SQL output plugin misbehaviour

Telegraf
, ,
1

I’m trying postgres output plugin on telegraf 1.31.0.
Scenario: drop a PSQL table while telegraf is running
Expected behavior: telegraf should be able to recreate the table if it does not exists
Actual behavior: Telegraf throws error saying table does not exists hence dropping the metrics batch! (doesnt even buffer them)

This scenario was tried with table template as ‘create table if not exists’ and ‘create table’ both.
How do I make telegraf create the tables while it is already running and tables are dropped because of some unexpected cleanup.

2

Hi,

You have not shared a config or full logs so I am not sure if you are concurrently or sequentially writing or the full scenario of what is going on.

However, if you are sequentially writing, the plugin looks through the metrics as they come in to ensure the necessary columns exist and if they do not exist then they are created. As a part of this, if the table does not exist then it is created.

3

Hi I’m not sure if telegraf is writing sequentially or concurrently but the error log says:

E! [outputs.postgresql] PG Prepare failed - map[err:ERROR: relation "abc" does not exist (SQLSTATE 42P01)
E! [outputs.postgresql] write error (permanent, dropping sub-batch): ERROR: relation "abc" does not exist (SQLSTATE 42P01)
4

Hi @jpowers , would really appreciate your help :smiley:

5

@Nistha please open a feature request for recreating the table if it vanishes after Telegraf started up…

6

@srebhan sure I’ll create a feature request for that. I would also like to ask what should one use if we want to execute a shell command on a specific condition. For example:

echo "Hello"

when metrics.fields[‘used’] > 100MBs.

I explored starlark functions like ctx.actions.run_shell but it didnt work for me. Could you please point me to some resource for this use case?
Thank you so much

7

Just use the exec output plugin with the metricpass filtering I would say…

8

@srebhan thank you, that looks like a very nice option. I tried it but I’m unable to pass the right command in the attribute. Suppose I simply want to pass the metrics into a shell script say:
/usr/local/bin/hello.sh <metrics>

How should the config look like?

[[outputs.exec]]
  namepass = ['cpu']
  command = ["/usr/local/bin/hello.sh"]
9

If you read the documentation for the exec plugin (link in my previous post) you will see

This plugin sends telegraf metrics to an external application over stdin.

The metric(s) is/are sent via stdin!

10

Thank you for your reply…I was able to make it work :smiley:

1 Like