I use a possible very inefficient way to store data in InfluxDB 1.2
- Python 3.3
- Pandas DataFrame
- daily sales of articles sold in stores)
- I need to add tags because the data needs to be filtered in Grafana by article ID and store ID
- I loop over stores and articles and use
write_points()
from influxdb import DataFrameClient
influx_client = DataFrameClient(...)
# Loop over stores
for store in stores:
# Loop over articles for article in articles: # Select data df_influx = df.loc[(df['store_id'] == store) & df['article_id'] == article] # Write data to InfluxDB influx_client.write_points(df_influx, 'measurement', tags={'tag_store':store, 'tag_article':article})
Problem:
- thousands of InfluxDB API calls
- loop takes very long
- I can’t write the whole DataFrame and subsequently add tags because article-store combinations would be overwritten with daily timestamps
Is there a way to increase efficiency of writing daily data with tags?