What is the proper way to move a measurement to a different bucket?

Fluxlang
1

I’m using Influx v2.0.1. I have a few measurements in a bucket that I need to change the retention policy for. I’m looking to move it to a different bucket. Its a fairly large amount of data. What is the proper way to make this move without running out of memory/disk?

from(bucket: "current_bucket")
|> range(start:-2y)
|> filter(fn: (r) => r["_measurement"] == "my_measurement")
|> to(bucket: "new_bucket", org: "myOrg")

Is there an easy way to tell how many records I have in a given measurement?

2

Hello @nsk1,
Welcome!
It depends on how much data you have in that measurement. How much data is it?
You might have to split it up in a series of smaller time periods.
You can also change the retention policy of your existing bucket.

You can also use the CLI to export data to a CSV and then write the CSVs to InfluxDB.

Here’s an example from a coworker:

#!/bin/bash

###
#
# This script will create daily annotated CSV files from a InfluxDB org using the Influx CLI.
# Notes: 
# 1) The InfluxDB credentials must be set in the influx CLI.
# 2) The annotated CSV files are saved to the current directory where the script is run.
# 
###
BUCKET=telegraf
NUM_DAYS=3
END_DATE=now # now or YYYY-MM-DD:00:00Z
###

cli_ver=`influx version`

if [[ $cli_ver != *"Influx CLI"* ]]; then
   echo "influx CLI not found."
   exit 1
fi

echo $cli_ver

if [ "${END_DATE}" == "now" ]; then
   end_date=`date +"%Y-%m-%dT%H:%M:00Z"`
else
   end_date=${END_DATE}
fi

echo "Processing ${NUM_DAYS} day(s) ending on ${end_date}..." 

for ((n=0; n < ${NUM_DAYS};n++));
do
   start_date=`date -d "${end_date} ${n} day ago" +"%Y-%m-%dT00:00:00Z"`
   start_date_short=`date -d "${start_date}" +"%Y-%m-%d"`
   echo ${BUCKET}_data_${start_date_short}.csv.gz

   if [[ -v stop_date ]]; then
      stop_date=`date -d "${start_date} + 1 day" +"%Y-%m-%dT00:00:00Z"`
   else
      stop_date=now\(\) 
   fi

   # output=`influx query -r "from(bucket: \"${BUCKET}\") |> range(start: ${start_date}, stop: ${stop_date}) |> drop(columns: [\"_start\", \"_stop\"])" | gzip > ${BUCKET}_data_${start_date_short}.csv.gz`
   output=`influx query -r "from(bucket: \"${BUCKET}\") |> range(start: ${start_date}, stop: ${stop_date}) |> aggregateWindow(every: 1h, fn: mean, createEmpty: false) |> to() |> filter(fn: (r) => false == true)" | ${BUCKET}_data_${start_date_short}.csv`
   if [[ $output != *"exceeded limited_query plan limit"* ]]; then
      echo "Error encountered."
      exit 1
   fi 
   sleep 10
done
3

Sorry, fairly new to flux here. How can I tell how much data is in a measurement?

I have a few measurements in that bucket that require a longer retention policy than the measurement I want to move.