Ingesting data to different retention policies by tag

1

In my system there are 1000s of devices that have a UUID

My series are the combination of one measurement “data” along with a TAG of UUID, this creates a unique series.

I’m using http_listener_v2 to capture data via telegraf

I want to be able to have different series in different retention policies and change this per uuid at will. I have found I can move data between policies as such:

SELECT * INTO "60d"."data" FROM "30d"."data" WHERE uuid='JRN7FM'

This works well for moving data between policies, however I am having a few issues with how I would ingest new data outside of the default.

One idea is to remove and create continuous queries to copy data from default retention policy (shortest) to whatever current policy the device has (in the case its not default shortest).

Can I have telegraf send data directly to a non-default policy, so a UUID that is on, say 60d, doesn’t have to first write to default, and then a continuous query pipe the data to 60d?

Is there another mechanism to accomplish this general goal I am overlooking?

Thank you,

2

You can, though you would need to maintain a list of the UUIDs in the Telegraf config, or possibly generate your config file from another source. It would look something like this:

[[outputs.influxdb]]
  urls = [ "http://localhost:8086" ]
  database = "data"
  retention_policy = "30d"
  [outputs.influxdb.tagpass]
    uuid = [
      "JRN7FM",
      "ABCDEF",
    ]

[[outputs.influxdb]]
  urls = [ "http://localhost:8086" ]
  database = "data"
  retention_policy = "60d"
  [outputs.influxdb.tagpass]
    uuid = [
      "GHIJKL",
      "MNOPQR",
    ]
1 Like