Renaming key values and writing to new bucket

InfluxDB 2
1

Hello, I have a collection of about two years of sensor data that I get via MQTT. I want to do some small changes to a few MQTT topic names to make them shorter.
I would like to change my key values so they would match the new topic names. How should I do this?

Currently I have something working in Flux but it is not efficient at all. I’m still very new in the flux language.

extractedA = from(bucket: "oldBucket")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "LocationA")
  |> filter(fn: (r) => r["source"] == "Water log")
  |> filter(fn: (r) => r["_field"] == "total m3 extracted A")
  |> set(key: "source", value: "WaterLog")
  |> set(key: "_field", value: "TotalExtractedA")
  
  extractedB = from(bucket: "oldBucket")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "LocationA")
  |> filter(fn: (r) => r["source"] == "Water log")
  |> filter(fn: (r) => r["_field"] == "total m3 extracted B")
  |> set(key: "source", value: "WaterLog")
  |> set(key: "_field", value: "TotalExtractedB")

  extractedA |> to(bucket: "newBucket")
  extractedB |> to(bucket: "newBucket")

What would be the most efficient way to do this? Can this be done in one query or should i avoid Flux at all? I want to change the “source” and _field key values to match my new MQTT topics.
Thanks!

2

Hello @AltopLuukG,
Are you using the MQTT telegraf plugin?
Honestly updating values in InfluxDB isn’t very efficient at all.
I suggest using telegraf to write the values correctly and until the old data expires. If this is an option.

I don’t see anything wrong with your script or a way to really optimize it further.
You could perform the task in smaller chuncks/time ranges. That might help.

3

Yes I am using the MQTT plugin for telegraf so new data intake won’t be a problem. I was hoping to change the old data so we can see everything in our dashboard charts. Would it be hard to query both the old and new data to display in our dashboard? If possible I would like to store the new data in a different bucket to keep everything clean. Our MQTT topic structure won’t change except for some topic names.