Hi there. I’m new to InfluxDB, and I’m trying to determine if it would be a good fit for my use-case, storing trade data from cryptocurrency exchanges.
Here are two examples of the kind of data I’d like to store:
And here’s an example of how I’m currently storing this in InfluxDB:
trade,exchange=polo,base=eth,quote=gnt,side=buy price=0.00197600,volume=1.46874329 1496698706000 trade,exchange=liqu,base=eth,quote=gnt,side=sell price=0.00209707,volume=91.00652896 1496701451000
Once I have the trade data I want to reduce it to candlestick data (open, high, low, close).
I have two concerns:
- How do I prevent adding duplicate trades? This data will be regularly scraped and thrown at the DB, so I think I will need a way to “upsert” trades based on some kind of unique ID (maybe an MD5 hash). I’ve found this issue on Github, but being new to InfluxDB the solution there isn’t clear to me.
- With how I’m currently storing this data, I think trades with the same tags and timestamp will be overwritten. So if there were two trades on the same exchange with the same base/quote that happened at the exact same time, only one of them would be stored. Is that right? And if so, how could I get around that?
Thanks for reading.