Take more time to visualize the data in Chronograf

InfluxDB 1 Chronograf
1

Hi,

I used the k6 load testing tool for executing the following load.

  • Transactions per second (TPS): 2500
  • Response payload size: 50 KB
  • Duration: 12 hours

Used TICK stack.

  • Influxdb v1.8.9
  • Chronograf v1.9.0
  • Telegraf v1.19.2

Used AWS c5.4xlarge Ubuntu instance. I create a simple graph by using the following query.

SELECT percentile(“value”, 95) AS “95_value” FROM “myk6db”.“autogen”.“http_req_duration” WHERE time > :dashboardTime: GROUP BY time(:interval:) FILL(none)

When I select 12 hour period to see the graph, it will take around 5 minutes to visualize the data.

I would be grateful if any of you can suggest a workaround for this issue.

Thanks,
Dilshan

2

Hello @Dilshan_Fernando,
How does the performance compare to 2.x?

3

How is your data structured? Having a high degree of cardinality can negatively impact performance, see this blog post for some more info about what that looks like.

I’m also curious about the TPS and response payload size you have indicated. Are there 2500 points for every second in the database? And is each point 50kb? That would be quite a bit of data, but I’m probably not understanding those load numbers completely.

4

@wbaker

I used the k6 as our performance testing tool. I’m working for an API platform team. There are a huge number of request traffic on our platform. Averagely it’s around 2500 per second.

I’m not doing any changes when the k6 tool sends data to InfluxDB. I’m using their existing sollution. Also, when comes to Grafana, I used their preconfigured dashboard.

5

Hi @Anaisdg,

sorry, I didn’t try it out with InfluxDB v2.0.

Also, I’m not sure whether k6 supports InfluxDB 2.0

6

I’m not familiar with k6 at all, but I checked it out briefly and from the looks of things you are able to create your own scripts for running tests. Is it possible that your script is generating a large amount of unique series that would lead to a large amount of cardinality? That would be my first guess as to why the query is taking so long to run. Maybe this isn’t an issue if k6 has a fixed output format.

Other than that, have you tried using a smaller time period for the query?

7

@wbaker ,

Yes, for a small time period (1 hour) this won’t be a big problem. Since k6 outputs, a large amount of data, and when the Data are visualized, influxDB takes time, which is expected.

FYI: How to minimize the size of the Output data - Converters & Integrations - k6 community forum

Best Regards,
Dilshan