Dear All,
We are designing a system to store data from sensors (who isn’t) and have some discussions how to store the metadata.
Our metadata consist of:
UNIT: degrees Celcius
LLIM: low limit of the range
HLIM: high limit of the range
Since this metadata is likely not to change much we would store them as tag keys like:
INSERT TI100,UNIT=degC,X_LLIM=0.0,X_HLIM=2000 X=40.
INSERT TI100,UNIT=degC,W_LLIM=0.0,W_HLIM=2000 W=40.
INSERT TI100,UNIT=%,Y_LLIM=0.0,Y_HLIM=100 Y=40.
Normally each measurement (TI100 in this case) can have up to 8 fields. (X actual value ; W setpoint ; Y: output value ; …)
Some of the fields don’t have extra metadata, like we also store boolean data so these don’t have a range.
Since this metadata is not changing (or minimal), do we need to send it every cycle when when we update our measurement (the X ) ?
This would be also extra load four our backend to retrieve this metadata every update
The next problem we might see is when we need to delete data. We have an external program which checks the size of the DB and if it reaches a certain size it starts to delete old data.
When we delete field data this goes quickly, but deleting tag data makes the query really slower. So we could just delete old field data to reduce the size, but this would make the database look a bit odd, since tag key data from the past will be stored forever.
Seeing this problem, someone proposed to have a separate InfluxDB database to store all this metadata. Would this be a better option?
Any views or suggestions would be appreciated
