[InfluxDB 2 OSS] - Restructure a measurement

InfluxDB 2
1

Hi all
I am trying to deploy the Influxdb solution in my company to track our production line data.
Working like a charm (nodered + influxdb 2 + grafana) but of course as I am not used to influxdb, I probably made non optimized choices.
First of all : I set up no tag on my measurement and now I am thinking about use some values as tag especially because I filter by some specific values and these values seems to be good candidate to be a tag.
I was wondering if I can create a new measurement from the current one and switch some field to tags
Is there any article or forum post explaining how to wisely choose the tags ?
thanks for your support

2

Hi @gmassart

I also use Node-RED + InfluxDB v2 + Grafana.

I found this webinar from Influxdb about Schema Design for IoT to be extremely helpful (even though it’s pretty old, it’s still relevant). Watch it all the way through, then think about your own data. Write down your field names & tag names, then go back and watch the video again to make sure it still makes sense.

Depending on your setup, you may have something like this:

Tags Examples
EquipID 404, 6A

and the possible fields:

Fields Type
air_temp integer or float?
process_temp integer or float?
rotational_speed integer or float?
1 Like
3

Thank you for the video which is in fact pretty old but instructive. Will watch that tomorrow.
I can imagine that setting too many tags can be counter productive too.

I basically record production line data with some tags coming to my minde such as :
-ProductionOrder
-ProductionNumber
-ProductName
…
I am thinking about putting tags for all PRESET that my automation colleague set up (pulling information from SIEMENS PLC through snap7 nodered node), information that does not change so often and is usually a source for filtering.

4

What are some examples of the above? Are there tens, hundreds of thousands of possible values?

Also, just to confirm, are you using Flux or InfluxQL?

Yes, you could essentially create a new database and convert your data using Node-RED and populate the correct tag data when doing the conversion.

5

The 3 examples are integers for the two first and a 5 alphanumeric for the third.
First are some kind of incremental, ProductionOrder it can change every 2/3 days and is a common key to filter values for statistical study.
ProductionNumber is an extension of ProductionOrder and change every 3 to 4 hours (could be more)
ProductName is selfexplanatory :slight_smile: we have on hundred of them probably but on one measurement it is more or less 20 of them (on measurement is a production line, some product are specific)
I use Flux.

I was looking for something similar to the following SQL but in FLUX
SELECT * INTO measurement2 FROM measurement1 transforming 3 field into 3 tags but keeping all other field the same (and keep all data)
Idea is to set up the new measurement with the tags and continue to record data into the new one (using the right tags setup with NODERED)

6

The thread below shows the way I transferred all of my old Influx data (v1.7, which was using InfluxQL and a very poor design for fieldnames, tags, etc.) into new Influx data (v2.3, which is using Flux with proper fieldnames, tags, etc.).

1 Like
7

Ok I get it, need to test if the amount of data is not too much and perhaps do a loop on the time range to split.
I know there is a way in flux to copy data to another measurement but I don’t know how to set tags depending on field value

8

How about using a more complex function node, such as:

msg.payload = msg.payload.map(function(value) {
    return {
        measurement: "SingleRotaryFurnaceZoneData",
        fields: {
            temperature: value.last,
        },
        tags: {
            MeasType: value.last > 1000 ? "high" : "normal",  // Example dynamic tag
            EquipNumber: "1",
            EquipZone: value.last > 800 ? "zone2" : "zone1"  // Example dynamic tag
        },
        timestamp: value.time
    };
});
return msg;
1 Like
9

Ok I get the idea but :

  • I have 40 fields, I want to keep 37
  • 3 fields need to become tags (keeping values of the field as tag value)

The function above with map() is interesting but I need to catch the tags before the function to set them into the msg.payload
So it means I need to pivot the data first so I can access column values and manage tags in that way will be easier.