Sure thing @kyled
Correct, it runs as it’s own process. It’s generally used to trigger alerts but the scripts involved can be used for data manipulation. You can run them side by side on the same server.
The influxDBOut node takes the results of your query and conversion and forwards it to a database, measurement and retention policy of your choosing.
So based on the default settings when Influx is installed I’m guessing you have a database with a retention policy of “autogen” so you would be querying “database.measurement.autogen” for example.
In the InfluxDBOut node you could specify the following
|influxDBOut()
.database('yourdatabase')
.retentionPolicy('converted_data')
.measurement('measurement')
.precision('s')
So a basic example using the win_disk measurement might be like this. This query gets the mean value of the raw data before writing back to a new RP.
batch
|query('SELECT mean("Free_Megabytes") AS "Free_Megabytes", mean("Percent_Free_Space") AS "Percent_Free_Space" FROM "yourdatabase"."autogen"."win_disk"')
.period(5m)
.every(5m)
.groupBy(time(5m), *)
|log()
|influxDBOut()
.database('yourdatabase')
.retentionPolicy('converted_data')
.measurement('win_disk')
.precision('s')
The above will run every 5 minutes, grabbing 5 minutes of data and then group the ouputted data in 5 minute chunks, obviously the timings are optional and can be changed to suit your requirement.
It then returns the converted data back to the same database, the same measurement but with the retention policy of “converted_data” leaving your original raw data intact.
The database, measurement and RP you write the data back to are optional also so you could write it to a separate database and measurement if you wanted to. I write it back to the same database and measurement, with the same field names but in a different retention policy because it makes handling my dashboards easier. I only need to change the retention policy in my dashboard to query raw or historical data.
If you haven’t created a new retention policy in influx before then log into the Influx CLI and run the following
create retention policy "converted_data" on "yourdatabase" duration 30d replication 1
show retention policies
^^^ this should show the default (autogen) and your new retention policy.
That creates a second RP on the database with a 30 day retention policy, which of course can be changed to 60d, 7d or however long you want. Replication 1 will suffice unless you are using a clustered InfluxDB environment.
Once the RP is created, you can modify the above query to fit your needs. Save that in a file called “convertdata.tick” and upload it to your server.
The following command will define the script for processing.
sudo kapacitor define convertdata -type batch -tick /path_to_tick_file -dbrp yourdatabase.autogen
Followed by
sudo kapacitor enable convertdata
Wait however long you set the query time/duration and check the results.
You can check in a couple of ways, if you are in the influx CLI you can query the data directly, but you will need to choose your new retention policy. So in the CLI it would be
use yourdatabase.converted_data
You can jump between RP’s by using the same command and specifying your other RP.
Followed by
show measurements
If all has worked then you will see your measurement there waiting for you to query.
You can also check whether Kapacitor has executed the task succesfully with the following two commands
kapacitor show convertdata
This will output a digraph of the script showing how many points have been processed and any errors if they exist.
You can also run
kapacitor watch convertdata
This will show the query in action, as it runs.
If you’re unsure about working with different retention policies in Influx then it might be worth having a browse through here
Hope that helps, explanations aren’t my strong point but if you get stuck let me know.
Phil
Edit: Just to point out, i don’t think it is possible to mix different aggregations in the query but if you’re just wanting to perform divisions, multiplications then you can drop the aggregation part from the query.
