Read Performance Slow on Raw Data Queries (500,000 points @ 1 second resolution)

1

New to InfluxDB but familiar with TSDBs like OSI PI and Wonderware Historian.

Running some performance tests on InfluxDB with 5 tags that produce data points (numbers) every 1 second. Currently have roughly 500,000 points in the database over 2 day period. I am finding that the queries are slower than what I would expect for a small number of data points.It seems to take several minutes to retrieve ~500k points from the database (raw data query). Is there anything that can be done to improve this retrieval time for the raw data?

Note: The queries do get slightly better when I run functions with longer time intervals like “Group By time(10s)”. However, with such a small data set I would expect the raw data retrieval times to be much faster for ~500k points of data.

My schema is very simple:

Measurement: PLC
Tags: tagname, quality
Fields: value

Environment is running on Raspberry PI 3B+. Memory is good during performance test (well within the memory of unit). CPU fluctuates depending on read query and will max out periodically to 100%, but return to normal. Data store will be on the SD card on the unit.

Many thanks in advance for any insight you can bring to improving this performance.

2

I suspect this may be reading from SD card will be affecting performance.

Standard SDCards read at about 10MiB/s, with middle of the range hitting about 50MiB/s, and the top end being 90MiB/s.

Depending on what card you have, you’ll see be seeing speeds similar to this.

3

Thanks “Rawkode” for the suggestion. I will investigate this potential bottle neck with SD. However, I am still curious as to why the CPU gets to close to 100% on Influx task when retrieving. With a basically raw data request it shouldn’t be doing much post processing of the data before it sends it to the client. I am not sure why it takes some much CPU effort?