InfluxDB Duplicates Problem

InfluxDB 2
,
1

Hello everyone! I am looking to remove duplicates in my influxDB database for a certain timerange. I have the TRAFFIC to a number of different locations as my influxDB data. The API I am using gives me the traffic data for those locations in realtime - meaning I have to keep calling the API every X period and store the traffic.
Right now, i am calling the API and storing the data every 30 seconds. What’s happening is people who are already in those locations are being added every time the data gets stored. I want to filter out all those duplicates in a 10 minutes time range (I am willing to count someone’s traffic TWICE if they stay for more than 10 minutes in a place). How would I go about doing this? I haven’t been able to figure out the process. I would really appreciate if someone could assist.

The traffic is for a virtual game. There are different locations in that virtual world and I am creating a traffic database for that virtual world, to find the highest visited places, etc.

So basically, what I am willing to do - is somehow filter check that how many individuals who stayed on the same location for less than 10 minutes (meaning there are multiple data points for that individual in that 10 minute data) and remove those data points.

2

Hello @R3Sist-SpM,
You could maybe:

  1. query for 10 min
  2. window data by every minute make sure to create empty windows with null and you can fill with a value (or maybe use null) but you might have to use fill()
  3. group data back together
  4. add a column with a value of 1 with map() function if the value isn’t null or your substitute value.
    map() function | Flux 0.x Documentation
    and then use statecount?
    stateCount() function | Flux 0.x Documentation

And see how many ppl reach a count less than 10 for the 10 min range?

3

@R3Sist-SpM,
let me know if that makes sense or if you want me to create an example with some dummy data.

4

Hello @Anaisdg , thanks a lot for your reply.
Yes, it would help a lot if you could show an example please.
Thank you!