Telegraf mqtt msg to influxdb filter

Telegraf
42

Hi， @Jay_Clifford
I want to try to store multiple configurations with telegraf. D. each configuration monitors different mqtt topics into different ifluxdb buckets :pray: :pray:

43

can you upload your config files as .txt files here and i will check them

44

Upload your current one and I shall split it for you

45

Wait a minute. Maybe I need to change the suffix of the file

46

[Uploading: image.png…]
It is found that the file is grayed out :bowing_woman:

47

image
image1175×359 30.1 KB

48 
#################################################################################

################  telegraf.conf ####################

#################################################################################

# Configuration for telegraf agent
[agent]
  ## Default data collection interval for all inputs
  interval = "10s"
  ## Rounds collection interval to 'interval'
  ## ie, if interval="10s" then always collect on :00, :10, :20, etc.
  round_interval = true

  ## Telegraf will send metrics to outputs in batches of at most
  ## metric_batch_size metrics.
  ## This controls the size of writes that Telegraf sends to output plugins.
  metric_batch_size = 1000

  ## Maximum number of unwritten metrics per output.  Increasing this value
  ## allows for longer periods of output downtime without dropping metrics at the
  ## cost of higher maximum memory usage.
  metric_buffer_limit = 10000

  ## Collection jitter is used to jitter the collection by a random amount.
  ## Each plugin will sleep for a random time within jitter before collecting.
  ## This can be used to avoid many plugins querying things like sysfs at the
  ## same time, which can have a measurable effect on the system.
  collection_jitter = "0s"

  ## Default flushing interval for all outputs. Maximum flush_interval will be
  ## flush_interval + flush_jitter
  flush_interval = "10s"
  ## Jitter the flush interval by a random amount. This is primarily to avoid
  ## large write spikes for users running a large number of telegraf instances.
  ## ie, a jitter of 5s and interval 10s means flushes will happen every 10-15s
  flush_jitter = "0s"

  ## By default or when set to "0s", precision will be set to the same
  ## timestamp order as the collection interval, with the maximum being 1s.
  ##   ie, when interval = "10s", precision will be "1s"
  ##       when interval = "250ms", precision will be "1ms"
  ## Precision will NOT be used for service inputs. It is up to each individual
  ## service input to set the timestamp at the appropriate precision.
  ## Valid time units are "ns", "us" (or "µs"), "ms", "s".
  precision = ""

  ## Log at debug level.
  # debug = false
  ## Log only error level messages.
  # quiet = false

  ## Log target controls the destination for logs and can be one of "file",
  ## "stderr" or, on Windows, "eventlog".  When set to "file", the output file
  ## is determined by the "logfile" setting.
  # logtarget = "file"

  ## Name of the file to be logged to when using the "file" logtarget.  If set to
  ## the empty string then logs are written to stderr.
  # logfile = ""

  ## The logfile will be rotated after the time interval specified.  When set
  ## to 0 no time based rotation is performed.  Logs are rotated only when
  ## written to, if there is no log activity rotation may be delayed.
  # logfile_rotation_interval = "0d"

  ## The logfile will be rotated when it becomes larger than the specified
  ## size.  When set to 0 no size based rotation is performed.
  # logfile_rotation_max_size = "0MB"

  ## Maximum number of rotated archives to keep, any older logs are deleted.
  ## If set to -1, no archives are removed.
  # logfile_rotation_max_archives = 5

  ## Pick a timezone to use when logging or type 'local' for local time.
  ## Example: America/Chicago
  # log_with_timezone = ""

  ## Override default hostname, if empty use os.Hostname()
  hostname = ""
  ## If set to true, do no set the "host" tag in the telegraf agent.
  omit_hostname = false
[[outputs.influxdb_v2]]
  namedrop = ["vehicle_info", "vehicle_state"]
  ## The URLs of the InfluxDB cluster nodes.
  ##
  ## Multiple URLs can be specified for a single cluster, only ONE of the
  ## urls will be written to each interval.
  ##   ex: urls = ["https://us-west-2-1.aws.cloud2.influxdata.com"]
  urls = ["http://192.168.16.2:8086"]

  ## Token for authentication.
  token = "jovyuwtCCN59l2X3LA8Qz35A2em6wrlihj2E2P_meF0qN-UEWwcOEq4ZRmTU7ZVMCD-IL7_YvAwTdsasW-OocQ=="

  ## Organization is the name of the organization you wish to write to; must exist.
  organization = "trunkport"

  ## Destination bucket to write into.
  bucket = "system_resource"

  ## The value of this tag will be used to determine the bucket.  If this
  ## tag is not set the 'bucket' option is used as the default.
  # bucket_tag = ""

  ## If true, the bucket tag will not be added to the metric.
  # exclude_bucket_tag = false

  ## Timeout for HTTP messages.
  # timeout = "5s"

  ## Additional HTTP headers
  # http_headers = {"X-Special-Header" = "Special-Value"}

  ## HTTP Proxy override, if unset values the standard proxy environment
  ## variables are consulted to determine which proxy, if any, should be used.
  # http_proxy = "http://corporate.proxy:3128"

  ## HTTP User-Agent
  # user_agent = "telegraf"

  ## Content-Encoding for write request body, can be set to "gzip" to
  ## compress body or "identity" to apply no encoding.
  # content_encoding = "gzip"

  ## Enable or disable uint support for writing uints influxdb 2.0.
  # influx_uint_support = false

  ## Optional TLS Config for use on HTTP connections.
  # tls_ca = "/etc/telegraf/ca.pem"
  # tls_cert = "/etc/telegraf/cert.pem"
  # tls_key = "/etc/telegraf/key.pem"
  ## Use TLS but skip chain & host verification
  # insecure_skip_verify = false
[[inputs.cpu]]
  ## Whether to report per-cpu stats or not
  percpu = true
  ## Whether to report total system cpu stats or not
  totalcpu = true
  ## If true, collect raw CPU time metrics
  collect_cpu_time = false
  ## If true, compute and report the sum of all non-idle CPU states
  report_active = false
[[inputs.disk]]
  ## By default stats will be gathered for all mount points.
  ## Set mount_points will restrict the stats to only the specified mount points.
  # mount_points = ["/"]
  ## Ignore mount points by filesystem type.
  ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"]
[[inputs.diskio]]
  ## By default, telegraf will gather stats for all devices including
  ## disk partitions.
  ## Setting devices will restrict the stats to the specified devices.
  # devices = ["sda", "sdb", "vd*"]
  ## Uncomment the following line if you need disk serial numbers.
  # skip_serial_number = false
  #
  ## On systems which support it, device metadata can be added in the form of
  ## tags.
  ## Currently only Linux is supported via udev properties. You can view
  ## available properties for a device by running:
  ## 'udevadm info -q property -n /dev/sda'
  ## Note: Most, but not all, udev properties can be accessed this way. Properties
  ## that are currently inaccessible include DEVTYPE, DEVNAME, and DEVPATH.
  # device_tags = ["ID_FS_TYPE", "ID_FS_USAGE"]
  #
  ## Using the same metadata source as device_tags, you can also customize the
  ## name of the device via templates.
  ## The 'name_templates' parameter is a list of templates to try and apply to
  ## the device. The template may contain variables in the form of '$PROPERTY' or
  ## '${PROPERTY}'. The first template which does not contain any variables not
  ## present for the device is used as the device name tag.
  ## The typical use case is for LVM volumes, to get the VG/LV name instead of
  ## the near-meaningless DM-0 name.
  # name_templates = ["$ID_FS_LABEL","$DM_VG_NAME/$DM_LV_NAME"]
[[inputs.mem]]
  # no configuration
[[inputs.net]]
  ## By default, telegraf gathers stats from any up interface (excluding loopback)
  ## Setting interfaces will tell it to gather these explicit interfaces,
  ## regardless of status.
  ##
  # interfaces = ["eth0"]
  ##
  ## On linux systems telegraf also collects protocol stats.
  ## Setting ignore_protocol_stats to true will skip reporting of protocol metrics.
  ##
  # ignore_protocol_stats = false
  ##
[[inputs.processes]]
  # no configuration
[[inputs.swap]]
  # no configuration
[[inputs.system]]
  ## Uncomment to remove deprecated metrics.
  # fielddrop = ["uptime_format"]


  [[processors.printer]]

  [[processors.starlark]]
namepass = ["vehicle_state"]  

#################################################################################


###################  telegraf.d / vehicle_info.conf ########################

#################################################################################

# Configuration for telegraf agent
[agent]
  ## Default data collection interval for all inputs
  interval = "10s"
  ## Rounds collection interval to 'interval'
  ## ie, if interval="10s" then always collect on :00, :10, :20, etc.
  round_interval = true

  ## Telegraf will send metrics to outputs in batches of at most
  ## metric_batch_size metrics.
  ## This controls the size of writes that Telegraf sends to output plugins.
  metric_batch_size = 1000

  ## Maximum number of unwritten metrics per output.  Increasing this value
  ## allows for longer periods of output downtime without dropping metrics at the
  ## cost of higher maximum memory usage.
  metric_buffer_limit = 10000

  ## Collection jitter is used to jitter the collection by a random amount.
  ## Each plugin will sleep for a random time within jitter before collecting.
  ## This can be used to avoid many plugins querying things like sysfs at the
  ## same time, which can have a measurable effect on the system.
  collection_jitter = "0s"

  ## Default flushing interval for all outputs. Maximum flush_interval will be
  ## flush_interval + flush_jitter
  flush_interval = "10s"
  ## Jitter the flush interval by a random amount. This is primarily to avoid
  ## large write spikes for users running a large number of telegraf instances.
  ## ie, a jitter of 5s and interval 10s means flushes will happen every 10-15s
  flush_jitter = "0s"

  ## By default or when set to "0s", precision will be set to the same
  ## timestamp order as the collection interval, with the maximum being 1s.
  ##   ie, when interval = "10s", precision will be "1s"
  ##       when interval = "250ms", precision will be "1ms"
  ## Precision will NOT be used for service inputs. It is up to each individual
  ## service input to set the timestamp at the appropriate precision.
  ## Valid time units are "ns", "us" (or "µs"), "ms", "s".
  precision = ""

  ## Log at debug level.
  # debug = false
  ## Log only error level messages.
  # quiet = false

  ## Log target controls the destination for logs and can be one of "file",
  ## "stderr" or, on Windows, "eventlog".  When set to "file", the output file
  ## is determined by the "logfile" setting.
  # logtarget = "file"

  ## Name of the file to be logged to when using the "file" logtarget.  If set to
  ## the empty string then logs are written to stderr.
  # logfile = ""

  ## The logfile will be rotated after the time interval specified.  When set
  ## to 0 no time based rotation is performed.  Logs are rotated only when
  ## written to, if there is no log activity rotation may be delayed.
  # logfile_rotation_interval = "0d"

  ## The logfile will be rotated when it becomes larger than the specified
  ## size.  When set to 0 no size based rotation is performed.
  # logfile_rotation_max_size = "0MB"

  ## Maximum number of rotated archives to keep, any older logs are deleted.
  ## If set to -1, no archives are removed.
  # logfile_rotation_max_archives = 5

  ## Pick a timezone to use when logging or type 'local' for local time.
  ## Example: America/Chicago
  # log_with_timezone = ""

  ## Override default hostname, if empty use os.Hostname()
  hostname = ""
  ## If set to true, do no set the "host" tag in the telegraf agent.
  omit_hostname = false
[[outputs.influxdb_v2]]
  namedrop = ["vehicle_info", "vehicle_state"]
  ## The URLs of the InfluxDB cluster nodes.
  ##
  ## Multiple URLs can be specified for a single cluster, only ONE of the
  ## urls will be written to each interval.
  ##   ex: urls = ["https://us-west-2-1.aws.cloud2.influxdata.com"]
  urls = ["http://192.168.16.2:8086"]

  ## Token for authentication.
  token = "jovyuwtCCN59l2X3LA8Qz35A2em6wrlihj2E2P_meF0qN-UEWwcOEq4ZRmTU7ZVMCD-IL7_YvAwTdsasW-OocQ=="

  ## Organization is the name of the organization you wish to write to; must exist.
  organization = "trunkport"

  ## Destination bucket to write into.
  bucket = "system_resource"

  ## The value of this tag will be used to determine the bucket.  If this
  ## tag is not set the 'bucket' option is used as the default.
  # bucket_tag = ""

  ## If true, the bucket tag will not be added to the metric.
  # exclude_bucket_tag = false

  ## Timeout for HTTP messages.
  # timeout = "5s"

  ## Additional HTTP headers
  # http_headers = {"X-Special-Header" = "Special-Value"}

  ## HTTP Proxy override, if unset values the standard proxy environment
  ## variables are consulted to determine which proxy, if any, should be used.
  # http_proxy = "http://corporate.proxy:3128"

  ## HTTP User-Agent
  # user_agent = "telegraf"

  ## Content-Encoding for write request body, can be set to "gzip" to
  ## compress body or "identity" to apply no encoding.
  # content_encoding = "gzip"

  ## Enable or disable uint support for writing uints influxdb 2.0.
  # influx_uint_support = false

  ## Optional TLS Config for use on HTTP connections.
  # tls_ca = "/etc/telegraf/ca.pem"
  # tls_cert = "/etc/telegraf/cert.pem"
  # tls_key = "/etc/telegraf/key.pem"
  ## Use TLS but skip chain & host verification
  # insecure_skip_verify = false
[[inputs.cpu]]
  ## Whether to report per-cpu stats or not
  percpu = true
  ## Whether to report total system cpu stats or not
  totalcpu = true
  ## If true, collect raw CPU time metrics
  collect_cpu_time = false
  ## If true, compute and report the sum of all non-idle CPU states
  report_active = false
[[inputs.disk]]
  ## By default stats will be gathered for all mount points.
  ## Set mount_points will restrict the stats to only the specified mount points.
  # mount_points = ["/"]
  ## Ignore mount points by filesystem type.
  ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"]
[[inputs.diskio]]
  ## By default, telegraf will gather stats for all devices including
  ## disk partitions.
  ## Setting devices will restrict the stats to the specified devices.
  # devices = ["sda", "sdb", "vd*"]
  ## Uncomment the following line if you need disk serial numbers.
  # skip_serial_number = false
  #
  ## On systems which support it, device metadata can be added in the form of
  ## tags.
  ## Currently only Linux is supported via udev properties. You can view
  ## available properties for a device by running:
  ## 'udevadm info -q property -n /dev/sda'
  ## Note: Most, but not all, udev properties can be accessed this way. Properties
  ## that are currently inaccessible include DEVTYPE, DEVNAME, and DEVPATH.
  # device_tags = ["ID_FS_TYPE", "ID_FS_USAGE"]
  #
  ## Using the same metadata source as device_tags, you can also customize the
  ## name of the device via templates.
  ## The 'name_templates' parameter is a list of templates to try and apply to
  ## the device. The template may contain variables in the form of '$PROPERTY' or
  ## '${PROPERTY}'. The first template which does not contain any variables not
  ## present for the device is used as the device name tag.
  ## The typical use case is for LVM volumes, to get the VG/LV name instead of
  ## the near-meaningless DM-0 name.
  # name_templates = ["$ID_FS_LABEL","$DM_VG_NAME/$DM_LV_NAME"]
[[inputs.mem]]
  # no configuration
[[inputs.net]]
  ## By default, telegraf gathers stats from any up interface (excluding loopback)
  ## Setting interfaces will tell it to gather these explicit interfaces,
  ## regardless of status.
  ##
  # interfaces = ["eth0"]
  ##
  ## On linux systems telegraf also collects protocol stats.
  ## Setting ignore_protocol_stats to true will skip reporting of protocol metrics.
  ##
  # ignore_protocol_stats = false
  ##
[[inputs.processes]]
  # no configuration
[[inputs.swap]]
  # no configuration
[[inputs.system]]
  ## Uncomment to remove deprecated metrics.
  # fielddrop = ["uptime_format"]


  [[processors.printer]]

  [[processors.starlark]]
namepass = ["vehicle_state"]



#################################################################################
#################  telegraf.d/vehicle_state.conf #######################################
#################################################################################

# Configuration for telegraf agent
[agent]
  ## Default data collection interval for all inputs
  interval = "10s"
  ## Rounds collection interval to 'interval'
  ## ie, if interval="10s" then always collect on :00, :10, :20, etc.
  round_interval = true

  ## Telegraf will send metrics to outputs in batches of at most
  ## metric_batch_size metrics.
  ## This controls the size of writes that Telegraf sends to output plugins.
  metric_batch_size = 1

  ## Maximum number of unwritten metrics per output.  Increasing this value
  ## allows for longer periods of output downtime without dropping metrics at the
  ## cost of higher maximum memory usage.
  metric_buffer_limit = 10000

  ## Collection jitter is used to jitter the collection by a random amount.
  ## Each plugin will sleep for a random time within jitter before collecting.
  ## This can be used to avoid many plugins querying things like sysfs at the
  ## same time, which can have a measurable effect on the system.
  collection_jitter = "0s"

  ## Default flushing interval for all outputs. Maximum flush_interval will be
  ## flush_interval + flush_jitter
  flush_interval = "10s"
  ## Jitter the flush interval by a random amount. This is primarily to avoid
  ## large write spikes for users running a large number of telegraf instances.
  ## ie, a jitter of 5s and interval 10s means flushes will happen every 10-15s
  flush_jitter = "10s"

  ## By default or when set to "0s", precision will be set to the same
  ## timestamp order as the collection interval, with the maximum being 1s.
  ##   ie, when interval = "10s", precision will be "1s"
  ##       when interval = "250ms", precision will be "1ms"
  ## Precision will NOT be used for service inputs. It is up to each individual
  ## service input to set the timestamp at the appropriate precision.
  ## Valid time units are "ns", "us" (or "µs"), "ms", "s".
  precision = ""

  ## Log at debug level.
  debug = true
  ## Log only error level messages.
  # quiet = false

  ## Log target controls the destination for logs and can be one of "file",
  ## "stderr" or, on Windows, "eventlog".  When set to "file", the output file
  ## is determined by the "logfile" setting.
  # logtarget = "file"

  ## Name of the file to be logged to when using the "file" logtarget.  If set to
  ## the empty string then logs are written to stderr.
  # logfile = ""

  ## The logfile will be rotated after the time interval specified.  When set
  ## to 0 no time based rotation is performed.  Logs are rotated only when
  ## written to, if there is no log activity rotation may be delayed.
  # logfile_rotation_interval = "0d"

  ## The logfile will be rotated when it becomes larger than the specified
  ## size.  When set to 0 no size based rotation is performed.
  # logfile_rotation_max_size = "0MB"

  ## Maximum number of rotated archives to keep, any older logs are deleted.
  ## If set to -1, no archives are removed.
  # logfile_rotation_max_archives = 5

  ## Pick a timezone to use when logging or type 'local' for local time.
  ## Example: America/Chicago
  # log_with_timezone = ""

  ## Override default hostname, if empty use os.Hostname()
  hostname = ""
  ## If set to true, do no set the "host" tag in the telegraf agent.
  omit_hostname = false
[[outputs.influxdb_v2]]
namedrop = ["vehicle_info","system"]
  ## The URLs of the InfluxDB cluster nodes.
  ##
  ## Multiple URLs can be specified for a single cluster, only ONE of the
  ## urls will be written to each interval.
  ##   ex: urls = ["https://us-west-2-1.aws.cloud2.influxdata.com"]
  urls = ["http://192.168.16.2:8086"]

  ## Token for authentication.
  token = "jovyuwtCCN59l2X3LA8Qz35A2em6wrlihj2E2P_meF0qN-UEWwcOEq4ZRmTU7ZVMCD-IL7_YvAwTdsasW-OocQ=="

  ## Organization is the name of the organization you wish to write to; must exist.
  organization = "trunkport"

  ## Destination bucket to write into.
  bucket = "vehicle_info"

  ## The value of this tag will be used to determine the bucket.  If this
  ## tag is not set the 'bucket' option is used as the default.
  # bucket_tag = ""

  ## If true, the bucket tag will not be added to the metric.
  # exclude_bucket_tag = false

  ## Timeout for HTTP messages.
  # timeout = "5s"

  ## Additional HTTP headers
  # http_headers = {"X-Special-Header" = "Special-Value"}

  ## HTTP Proxy override, if unset values the standard proxy environment
  ## variables are consulted to determine which proxy, if any, should be used.
  # http_proxy = "http://corporate.proxy:3128"

  ## HTTP User-Agent
  # user_agent = "telegraf"

  ## Content-Encoding for write request body, can be set to "gzip" to
  ## compress body or "identity" to apply no encoding.
  # content_encoding = "gzip"

  ## Enable or disable uint support for writing uints influxdb 2.0.
  # influx_uint_support = false

  ## Optional TLS Config for use on HTTP connections.
  # tls_ca = "/etc/telegraf/ca.pem"
  # tls_cert = "/etc/telegraf/cert.pem"
  # tls_key = "/etc/telegraf/key.pem"
  ## Use TLS but skip chain & host verification
  # insecure_skip_verify = false


###############################################################################
#                            INPUT PLUGINS                                    #
###############################################################################
[[inputs.mqtt_consumer]]
  ## Broker URLs for the MQTT server or cluster.  To connect to multiple
  ## clusters or standalone servers, use a seperate plugin instance.
  ##   example: servers = ["tcp://localhost:1883"]
  ##            servers = ["ssl://localhost:1883"]
  ##            servers = ["ws://localhost:1883"]
  ## servers = ["tcp://172.29.60.10:1883"]
  servers = ["tcp://172.17.0.2:1883"]

  ## Topics that will be subscribed to.
  topics = [
    "/realtime-monitor/upload-info/+"
  ]

  ## Data format to consume.
  ## Each data format has its own unique set of configuration options, read
  ## more about them here:
  ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
  data_format = "json"


  name_override = "vehicle_state"


###############################################################################
#                            Processor PLUGINS                                #
###############################################################################
#credit goes to @srebhan

[[processors.starlark]]
namepass = ["vehicle_info"]
  source = '''
def apply(metric):
  fields = ["wheel_angle", "wheel_odometer"]
  for f in fields:
    key_prefix = f+"_"
    key_len = len(key_prefix)
    keys = [x for x in metric.fields.keys() if x.startswith(key_prefix)]
    keys = sorted(keys, key=lambda x: int(x[key_len:]))
    values = [str(metric.fields.pop(k)) for k in keys]
    new_value = "["+', '.join(values)+"]"
    metric.fields[f] = new_value
  return metric
'''

[[processors.printer]]
49

I used a split line. Please copy it
#################################################################################
telegraf.d Is the configuration directory
#################################################################################

50

It may not be perfect， Please correct anything bad

######################## docker-compose  #######################
version: "2.2"
services:
  influxdb:
    image: influxdb:latest
    container_name: influxdb
    ports:
      - "8086:8086"

    volumes:
      # Data persistency
      # sudo mkdir -p ./influxdb/data
      - /Users/apple/vdss_volume/influxdb/data:/var/lib/influxdb2
      - ./etc/timezone:/etc/timezone:ro
      - /Users/apple/vdss_volume/localtime:/etc/localtime:ro

    expose:
      - "8083"
      - "8086"
      - "8090"

    deploy:
      resources:
        limits:
          cpus: '0.20'
          memory: 1024M



    restart: unless-stopped 

    logging:
      driver: "json-file"
      options:
        max-size: "100m"


  telegraf:
    image: telegraf:latest
    container_name: telegraf
    links:
      - influxdb
    command:
      telegraf -config /etc/telegraf/telegraf.conf --config-directory /etc/telegraf/telegraf.d
    volumes:
      - /Users/apple/vdss_volume/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf
      - /Users/apple/vdss_volume/telegraf/telegraf.d:/etc/telegraf/telegraf.d
      - /Users/apple/vdss_volume/localtime:/etc/localtime:ro
      - /Users/apple/vdss_volume/telegraf/logs:/var/log

    deploy:
      resources:
        limits:
          cpus: '0.20'
          memory: 1024M

    restart: unless-stopped

    logging:
      driver: "json-file"
      options:
        max-size: "100m"

networks:
  vdss:
    driver: bridge
51 
##################  mqtt msg ##########################
{
        "y": 1111111.354243171,
        "x": 1111111.9858582001,
        "wheel_speed": [0.0, 1.0, 2.0, 4.0],
        "gps_status": 50,
        "throttle": 0.0,
        "gear": 0,
        "planning_cmd_at": 1634607917.908871,
        "turn_light_status": 1,
        "global_path_at": None,
        "right_behind_at": 1634607917.858688,
        "wheel_angle": [
                    0.35431448844127544, 0.09063859006636221, 0.35431448844127544, 0.09063859006636221,
                    -0.08514534218357994, 0.35431448844127544, -0.2609292744335221, 0.1785305561913333
                        ],
        "name": "T501",
        "left_behind_at": 1634607917.8445928,
        "current_brake": 0.0,
        "vehicle_error_status": 0,
        "yaw": -1.3395460726545767,
        "left_front_at": 1634607917.8334768,
        "right_front_at": 1634607917.9152327,
        "brake": 0.0,
        "updated_at": 1635304171.318089,
        "auto_driver_status": 0,
        "odometer": 611.0,
        "emergency_stop_status": 0,
        "wheel_odometer": [9.9, 9.1, 9.5, 6.6000000000000005, 9.9, 10.100000000000001, 9.5, 7.9],
        "distance": 0.0, "gy": 957004.5313795683,
        "current_throttle": 0.0,
        "ogm_points": [{"type": 0, "x": -27.771453857421875, "y": 7.960874557495117},
                        {"type": 0, "x": -26.048484802246094, "y": 7.958527565002441},
                        {"type": 16, "x": 14.540033340454102, "y": -25.705089569091797}
                      ],
        "topics": {
            "/pnc_msgs/vehicle_cmd": {
                "secs": 1637906129,
                "nsecs": 790535501
            },
            "/pnc_msgs/planning_cmd": {
                "secs": 1637906129,
                "nsecs": 794717721
            },

        },
                }
53

Hi @loneWolf666,
Can you try this? Let me know when you have downloaded it and I will delete the link: telegraf-car.zip - Google Drive

54

Hi,@Jay_Clifford
Thank you very much for your great support for such a long time， :100: :100: :100:

Let me venture to say my idea. Although it has been realized, I always feel that mqtt data is broadcast to all configurations, and re filtering is not very elegant. In addition, telegraf 1 Is the X version handled the same way?

[[processors]]  Can you filter some JSON fields？？？？ :pray:
55

Hi @loneWolf666,
Yes we support both field and tag filtration: Configuring Telegraf | Telegraf 1.20 Documentation

56

Hi，@Jay_Clifford
If I want to collect non talking server resources through different telegraf configurations and store them in different data buckets, can I achieve this？ :hear_no_evil: :hear_no_evil:

57

Can you explain a little more about what you mean?

Yes through the same technique and structure we discussed in your previous example.

58

Hi，@Jay_Clifford
I consider deploying telegraf on multiple servers and adding telegraf to collect server resources Conf, how to store different machine resources in different influxdb data buckets

59

If multiple servers use the same configuration, each of the same indicators collected by the servers will have the same measurement, and the data will overlap each other

  namedrop = ["vehicle_info", "vehicle_state"]
  ## The URLs of the InfluxDB cluster nodes.
  ##
  ## Multiple URLs can be specified for a single cluster, only ONE of the
  ## urls will be written to each interval.
  ##   ex: urls = ["https://us-west-2-1.aws.cloud2.influxdata.com"]
  urls = ["http://192.168.16.2:8086"]

  ## Token for authentication.
  token = "Y3PWU-UlJoeHgKIkdPfiLVEY_jOkWeZupgjNrrefE8FxMPOZSp-BmfdZI8f6zassMcDb9w5zeN49heoTPzVTxw=="

  ## Organization is the name of the organization you wish to write to; must exist.
  organization = "trunkport"

  ## Destination bucket to write into.
  bucket = "system_resource"


[[inputs.cpu]]
  ## Whether to report per-cpu stats or not
  percpu = true
  ## Whether to report total system cpu stats or not
  totalcpu = true
  ## If true, collect raw CPU time metrics
  collect_cpu_time = false
  ## If true, compute and report the sum of all non-idle CPU states
  report_active = false
[[inputs.disk]]
  ## Ignore mount points by filesystem type.
  ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"]
[[inputs.diskio]]

[[inputs.mem]]

[[inputs.net]]

[[inputs.processes]]

[[inputs.swap]]
60

So there are two ways you could tackle this. Suppose we have servers A, B and C running Telegraf. Server D is running InfluxDB:

  1. Write all data to the same system bucket but add a unique tag. Telegraf automatically adds the host tag to your data to along as each host has a unique hostname then you can filter by this to get the results for each server. If you prefer to add your own unique identifier you can do it like this:
[global_tags]
  servername = "A"


[[inputs.cpu]]
  ## Whether to report per-cpu stats or not
  percpu = true
  ## Whether to report total system cpu stats or not
  totalcpu = true
  ## If true, collect raw CPU time metrics
  collect_cpu_time = false
  ## If true, compute and report the sum of all non-idle CPU states
  report_active = false
[[inputs.disk]]
  ## Ignore mount points by filesystem type.
  ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"]
[[inputs.diskio]]

[[inputs.mem]]

[[inputs.net]]

[[inputs.processes]]

[[inputs.swap]]

This would be the most efficient way if you plan to compare and query data between servers.
2. You could change the measurement name of each input plugin per server with a suffix/prefix. All measurements would still be stored in the same bucket just under different measurements:

[[inputs.cpu]]
  name_suffix = "_serverA"
  ## Whether to report per-cpu stats or not
  percpu = true
  ## Whether to report total system cpu stats or not
  totalcpu = true
  ## If true, collect raw CPU time metrics
  collect_cpu_time = false
  1. Lastly you could create a bucket for each server and just copy configs across to these servers. From there just change the output bucket name to the bucket you want your data to go into.
1 Like
61

@Jay_Clifford ，Thank you very much for your support, I use regular processing today，
I don’t know if it’s a good way

image
image1694×1526 308 KB

62

Hi，@Jay_Clifford
Can telegraf store different fields of JSON data monitored by mqtt in different measurements of the same influxdb data bucket?