“Error in plugin: metric parse error: expected tag at”

hi after i manage clear my problem in this documentation

it’s worked but not what i’m expected

  1. my powershell freeze after i run the config
  2. there and error in telegraf log after i send a payload in mqtt mainly this type of error:
[inputs.mqtt_consumer] Error in plugin: metric parse error: expected tag at 1:4: "356"

in error above i type this value in payload

356

this is my config went this error log happen

[agent]
  interval = "20s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "5s"
  flush_interval = "10s"
  flush_jitter = "5s"
  precision = "1ms"
  debug = true
  quiet = true
  logtarget = "file"
  logfile = "telegraflog"
  logfile_rotation_interval = "1d"
  logfile_rotation_max_size = "0"
  logfile_rotation_max_archives = -1
  log_with_timezone = "Asia/Jakarta"
  hostname = ""
  omit_hostname = false

[[outputs.influxdb_v2]]  
  urls = ["<deleted>"]
  token = "$env:INFLUX_TOKEN"
  organization = "<rejected>"
  bucket = "<deleted>t"
  exclude_bucket_tag = false
  timeout = "600s"
  user_agent = "telegraf"

[[inputs.mqtt_consumer]]
  servers = ["<deleted>"]
  topics = ["sensor/lokasi/kedalaman/#"]
  qos = 1
  connection_timeout = "600s"
  max_undelivered_messages = 1000
  persistent_session = true
  ## If unset, a random client ID will be generated.
  client_id = "telegraf_mqtt_client_id"
  username = "<deleted>"
  password = "<DELETED>"  # assuming typo here
  data_format = "influx"
  
  influx_parser_type = "internal"
  influx_timestamp_precision = "1ns"

[[inputs.mqtt_consumer.topic_parsing]]
    topic = "sensor/lokasi/kedalaman/#"
    measurement = "Hasil_Data"
    tags = "_/lokasi/_/_"
    fields = "sensor/lokasi/kedalaman/#"

    [inputs.mqtt_consumer.topic_parsing.types]
     data_type = "float"   

so i’m using mqtt ex and mqttx to send my payload in bot RAW (plain text) or (JSON) the error in log still exist

How do you know it froze? You are logging to a file, so your shell probably isn’t frozen. All output is sent to that telegraflog file.

in error above i type this value in payload
data_format = “influx”

The example of 356 is not influx line protocol. You could use the value data format to parse the single value.

so after like running and suppossingly logging to a file, my shells “_” keep blinking non stop. at that process i cant do anything to my shells, typing or even the normal dir and type doesnt even appeared on my screen, it does that for an hour or 2 before i decided to just ctrl + C. went i knowed is running went i tried messing aroung with my mqtt and sent message to the other broker, i checked my telegraf log to see if something happened and there a bunch of error message coming.

Now into the next problem.
After i followed your suggestion into changing my data_format into “value”

I had this bunch of errors went I try to sent a RAW (plaintext) value as a payload:

E! [outputs.influxdb_v2] When writing to <redacted>: failed to write metric to MQTT input (401 Unauthorized): unauthorized: unauthorized access
E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
E! [outputs.influxdb_v2] When writing to <Redacted>: failed to write metric to MQTT input (401 Unauthorized): unauthorized: unauthorized access
E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
E! [outputs.influxdb_v2] Failed to write metric to MQTT input (will be dropped: 405 Method Not Allowed): 405 Method Not Allowed

I know the problem of this is incorrect method to put it in influxDB database, is there away to convert plaintext into influxdb line protocol (or like json, and xml) so that i dont have to see the above errors or future errors?

If you are launching Telegraf manually, then yes it will take over the shell. What I think you might want is to run Telegraf as a service so it runs in the background. This is something you can set up after you get a working config.

If you want to see the progress while you start to get this set up I would, remove the logtarget = "file" and logfile = "telegraflog" config options for now.

I know the problem of this is incorrect method to put it in influxDB database

That’s not the problem. What Telegraf does is reads the value from MQTT and converts it into something that InfluxDB expects when it writes to it. Telegraf does the work of converting the value to line protocol for you! The issue here is these error messages:

(401 Unauthorized): unauthorized: unauthorized access

Whatever credentials you provided seem to be incorrect.

i already fix that problem by changing the value on array in the topic, it somehow fixed it. but the problem that still stand is

Then

i thought it need another lines of code into the configuring file to make sure that works. for now the problem above still stand

E! [outputs.influxdb_v2] Failed to write metric to MQTT input (will be dropped: 405 Method Not Allowed): 405 Method Not Allowed

This error message does not look right. Why is the output talking about the MQTT input?

i thought it need another lines of code into the configuring file to make sure that works. for now the problem above still stand

What InfluxDB are you writing to? Open source? Cloud? v2? v3?

user_agent = “telegraf”

Why are you setting this?

You haven’t shared your URL, are you certain you are using the right URL endpoint? That error can mean that the endpoint you are trying to write to isn’t actually right.

idk about that one for now i disables it

i’m using influxdb cloud server us-east

thats the one i don’t know why the errors said that. i tried running it again just a second ago the log still said that
gonna send the new conf, the test, the log is that error up there
new conf

[agent]
  interval = "20s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "5s"
  flush_interval = "10s"
  flush_jitter = "5s"
  precision = "1ms"
  debug = true
  quiet = true
  logtarget = "file"
  logfile = "telegraflog"
  logfile_rotation_interval = "1d"
  logfile_rotation_max_size = "0"
  logfile_rotation_max_archives = -1
  log_with_timezone = "Asia/Jakarta"
  hostname = ""
  omit_hostname = false

[[outputs.influxdb_v2]]  
  urls = ["https://us-east-1-1.aws.cloud2.influxdata.com/orgs/<org ids>"]
  token = "$env:INFLUX_TOKEN"
  organization = "<__>"
  bucket = "MQTT input"
  exclude_bucket_tag = false
  timeout = "600s"
  #user_agent = "telegraf"

[[inputs.mqtt_consumer]]
  servers = ["mqtt://192.168.1.4:1883"]
  topics = ["sensor/lokasi/kedalaman/#"]
  qos = 1
  connection_timeout = "600s"
  max_undelivered_messages = 1000
  persistent_session = true
  ## If unset, a random client ID will be generated.
  client_id = "telegraf_mqtt_client_id"
  username = "<__>"
  password = "<__>"  # assuming typo here
  data_format = "value"
  influx_parser_type = "internal"
  #influx_timestamp_precision = "1 millisecond"

[[inputs.mqtt_consumer.topic_parsing]]
    topic = "sensor/lokasi/kedalaman/#"
    measurement = "Hasil_Data"
    tags = "_/lokasi/_/_"
    fields = "sensor/lokasi/kedalaman/#"

    [inputs.mqtt_consumer.topic_parsing.types]
     data_type = "float"   

below is me testing the mqtt by sent a value to the certain topic


below is the fresh error that i just got

E! [outputs.influxdb_v2] Failed to write metric to MQTT input (will be dropped: 405 Method Not Allowed): 405 Method Not Allowed

1 Like

This is the wrong URL as I suspected. It should be:

urls = ["https://us-east-1-1.aws.cloud2.influxdata.com/"]

with this quote i have another error stating that 401 unauthorized

E! [outputs.influxdb_v2] When writing to [https://us-east-1-1.aws.cloud2.influxdata.com/]: failed to write metric to MQTT input (401 Unauthorized): unauthorized: unauthorized access
E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
E! [outputs.influxdb_v2] When writing to [https://us-east-1-1.aws.cloud2.influxdata.com/]: failed to write metric to MQTT input (401 Unauthorized): unauthorized: unauthorized access
E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)

what type of credentials that i failed to put here, can you give an examples or somekind?

This is also not right, if you are trying to reference the enviornment variable INFLUX_TOKEN then use "$INFLUX_TOKEN".

I would expect the token to look sort of like a random string of around 88 or so charachters:

_0nEjmLxKfcAHHV5-wMS0TpfpbmYpJLJPcp3...

I would try the following:

  1. Put the token directly into your config and confirm it works first.
  2. Once it does, then set the env variable and use it as I have above.

so i deleted my api token to get a new one to test this one out by generating random token sugessted by setup instruction on influxdb server
now the whole process of 401 unnauthorized is there in power shell as if my token wasnt read by the power shell.

do i need to type it myself or should i copy pasted it and then tried again?

Not sure I follow, but when you generate a token, you must copy it down again as it is not retrievable afterwards and then update the token in your config.

I do that i copy them and put it in the save place, afterward i copy the same token and put into my telegraf config and got that 401 error.

Just to make sure, i need to copy this example that i will be seeing in influxdb setup instruction in telegraf section of token into my conf file and into different file (.txt) correct?

Or should i copy the whole thing that include the

Export INFLUX_TOKEN ="<The Token>"

Your token in telegraf config should only be the actual token.

token = "_0nEjmLxKfcAHHV5-wMS0TpfpbmYpJLJPcp3..."

so iwanna to see the full of what happenned in the log what’s happening by turning off my quiet into false
this is my config before i try the log:

[agent]
  interval = "20s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "5s"
  flush_interval = "10s"
  flush_jitter = "5s"
  precision = "1ms"
  debug = true
  quiet = false
  logtarget = "file"
  logfile = "telegraflog"
  logfile_rotation_interval = "1d"
  logfile_rotation_max_size = "0"
  logfile_rotation_max_archives = -1
  log_with_timezone = "Asia/Jakarta"
  hostname = ""
  omit_hostname = false

[[outputs.influxdb_v2]]  
  urls = ["https://us-east-1-1.aws.cloud2.influxdata.com"]
  token = "$env:INFLUX_TOKEN" #i need to set it that way to gain access to my telegraf log went the thing is running
  organization = "63aee332ce56a036"
  bucket = "MQTT input"
  exclude_bucket_tag = true
  timeout = "600s"
  #user_agent = "telegraf"

[[inputs.mqtt_consumer]]
  servers = ["mqtt://192.168.1.4:1883"]
  topics = ["sensor/lokasi/kedalaman/#"]
  qos = 1
  connection_timeout = "600s"
  max_undelivered_messages = 1000
  persistent_session = true
  ## If unset, a random client ID will be generated.
  client_id = "telegraf_cf820687"
  username = "Rahman"
  password = "sin356"  # assuming typo here
  data_format = "value"
  influx_parser_type = "internal"
  #influx_timestamp_precision = "1 millisecond"

[[inputs.mqtt_consumer.topic_parsing]]
    topic = "sensor/lokasi/kedalaman/#"
    measurement = "Hasil_Data"
    tags = "_/lokasi/_/_"
    fields = "sensor/lokasi/kedalaman/#"

    [inputs.mqtt_consumer.topic_parsing.types]
     data_type = "float"   

the error is actually still the same the 401 unauthorized but maybe if you see the log it can help in where i was wrong on that config

I! Starting Telegraf 1.30.2 brought to you by InfluxData the makers of InfluxDB
I! Available plugins: 233 inputs, 9 aggregators, 31 processors, 24 parsers, 60 outputs, 5 secret-stores
I! Loaded inputs: mqtt_consumer
I! Loaded aggregators: 
I! Loaded processors: 
I! Loaded secretstores: 
I! Loaded outputs: influxdb_v2
I! Tags enabled: host=<Host_Name>
I! [agent] Config: Interval:20s, Quiet:false, Hostname:<Host_Name>, Flush Interval:10s
D! [agent] Initializing plugins
D! [agent] Connecting outputs
D! [agent] Attempting connection to [outputs.influxdb_v2]
D! [agent] Successfully connected to outputs.influxdb_v2
D! [agent] Starting service inputs
I! [inputs.mqtt_consumer] Connected [mqtt://192.168.1.4:1883]
D! [inputs.mqtt_consumer] Session found [mqtt://192.168.1.4:1883]
D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
E! [outputs.influxdb_v2] When writing to [https://us-east-1-1.aws.cloud2.influxdata.com]: failed to write metric to MQTT input (401 Unauthorized): unauthorized: unauthorized access
[outputs.influxdb_v2] Buffer fullness: 1 / 10000 metrics
E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)

[outputs.influxdb_v2] Buffer fullness: 1 / 10000 metrics
this is went im sent my payload as a test

D! [agent] Successfully connected to outputs.influxdb_v2

it said here in my log before i sent anything that my telegraf successfully connected to my influxdb but after that why error 401 occurs said its an unauthorized access. that is the part that confuse me

[1] token = “$env:INFLUX_TOKEN” is the same as
[2] token = “Ib2r5KcrNSG2El_XSPGqhQr6niKp_aSMn0wBT3h8PuWyCIBpDyOMhhSB8KB1NWILgHgBbDpF9HjCM-27O8EEVg==”
if i use the [1] the error will be transmitted into my telegraf log
if i use the [2] the error will be transmitted into my shells wihout further information

an also i want to ask this, is this server appropriate for influxDB cloud:

because i went and check the documentation for mqtt_consumer Input plug in and see the example for the server which is like this

  ## Broker URLs for the MQTT server or cluster.  To connect to multiple
  ## clusters or standalone servers, use a separate plugin instance.
  ##   example: servers = ["tcp://localhost:1883"]
  ##            servers = ["ssl://localhost:1883"]
  ##            servers = ["ws://localhost:1883"]
  servers = ["tcp://127.0.0.1:1883"]

is this the reason?

You will want to delete that token as it appear to be a working token :wink:

I took your config for influxdb v2 and used it to send a few CPU metrics successfully:

2024-05-10T13:08:55Z D! [agent] Attempting connection to [outputs.influxdb_v2]
2024-05-10T13:08:55Z D! [agent] Successfully connected to outputs.influxdb_v2
2024-05-10T13:08:55Z D! [agent] Starting service inputs
2024-05-10T13:08:56Z D! [agent] Stopping service inputs
2024-05-10T13:08:56Z D! [agent] Input channel closed
2024-05-10T13:08:56Z I! [agent] Hang on, flushing any cached metrics before shutdown
2024-05-10T13:08:56Z D! [outputs.influxdb_v2] Wrote batch of 33 metrics in 555.28239ms
2024-05-10T13:08:56Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2024-05-10T13:08:56Z I! [agent] Stopping running outputs
2024-05-10T13:08:56Z D! [agent] Stopped Successfully

So you will probably now see a CPU table.

it said here in my log before i sent anything that my telegraf successfully connected to my influxdb but after that why error 401 occurs said its an unauthorized access. that is the part that confuse me

When the output reports it successfully set up its client, it does not mean it can authenticate correctly. The InfluxDB output does not actually “connect” and establish a connection. It is a basic HTTP API client that sends data when it needs to, like on sending metrics. So we do not know if we are able to successfully connect until we send data.

This is the exact config I used, with the token shortened:

[[outputs.influxdb_v2]]
  urls = ["https://us-east-1-1.aws.cloud2.influxdata.com"]
  token = "Ib2r5Kcr...." #i need to set it that way to gain access to my telegraf log went the thing is running
  organization = "63aee332ce56a036"
  bucket = "MQTT input"
  exclude_bucket_tag = true
  timeout = "600s"

So the output fine and worked well

For now i deleted it and gonna change it tomorrow. but, i still don’t get it if this worked in your device why can’t it worked on mine right now my suspect is in input server of mqtt_consumer or idk.