Evaluation of thousands metrics

Hello,
I’m trying to evaluate thousands of metrics using a checkers, but my computer doesn’t count it. I tried tasks too.

PC: notebook with Core i5 (8 threads) and 16 GB RAM
I’m running influxdb in the docker (6 threads, 8 GB RAM is allowed).

Have you some idea where is problem?
Or influxdb can compute so many metrics?

Thanks!

Hello @Matous_Dvorak,
Can you please explain a little bit more about evaluate thousands of metrics using checkers, but your computer doesn’t count it?

What exactly have you tried?

Thanks

Hello @Anaisdg,
I have influxdb in docker, where I using telegraf for writing data (it is something about 1,5K metrics every seconds). I don’t have problem with writting data. I using checkers (thresholds alerts) for evaluation of these metrics. (I try evaluate every 15s) I import these alerts to influxdb, but after import all of them, I only see 1-8 alert from time evaluation in alert history. So I try tasks, split evaluation to 1/15 alerts every seconds, but everything had same result.

Hello @Matous_Dvorak,
Are you using an offset? Sometimes there’s a read/write conflict with checks. Can you share some of the Flux for tasks that aren’t working please?

It sounds like you might know a lot about the checks and notification system in InfluxDB but this could be useful to you:

2 Likes

Hello @Anaisdg ,
yes of course, I using offset. Yes, thank you. I think this help me. I only see units of evaluated metrics, sometimes loading fails and if I turn off container in docker and start it again, it do not load home page of influxdb.

My tasks look like this:

I think I have a problem with data filtering.

Hello @Matous_Dvorak,
I would try to filter by measurement, tag, and then field.
Are you using the official 2.0 image? Are you getting any errors when you can’t load the UI?
Your task looks good otherwise. Perhaps the task isn’t set to active/run when you start docker again?

Yes, official 2.0 image. I didn´t try it, because it is so many metrics. How I active task to run after docker again?

Hello @Matous_Dvorak,
You can list a task and see it’s status with:

curl -X GET \
  https://us-west-2-1.aws.cloud2.influxdata.com/api/v2/tasks?orgID=<my org ID>?name=<my task name>
  -H 'authorization: Token <my token>\

https://docs.influxdata.com/influxdb/v2.0/api/#operation/GetTasks

Then you can update it with and set it to active with:

curl -X GET \
   https://us-west-2-1.aws.cloud2.influxdata.com/api/v2/tasks/<taskID>
  -H 'authorization: Token <my token>\
  -d '{"status": "active"}'

https://docs.influxdata.com/influxdb/v2.0/api/#operation/PatchTasksID

1 Like

Hello @Anaisdg,
I start with using offset and optimization of writing data. It is better, my processor does not go on 100%, but on 50-60%. But my problem is not solved. Now I using deadman checks (8000 metrics).

Hello @Anaisdg,
I found this: Hardware sizing guidelines | InfluxDB OSS 1.8 Documentation, Have you something like that for influxdb v2.0?

@Matous_Dvorak, those guidelines apply to v2 OSS as well.

@scott is there a guide for v2 as well?

1 Like

Not yet. While these recommendations should apply to both 1.8 and 2.0, engineering is still working through benchmarks to provide hardware recommendations specific to 2.0.

1 Like

@Anaisdg, can I evaluate more then 1 metrics in one task?

Hello @Matous_Dvorak,
Yes you can.

1 Like

Hello @Anaisdg,
I can evaluate all metrics every 15s, but I must be faster. Another problem is that the evaluation must be configurable. So I decide to not using influxdb to evaluation.

Thank you for your help.