Strategies for Abstracting InfluxDB Reads & Writes in Web Apps

In our applications we perform quite a bit of sensor data collection. In some cases sensors are impractical, so we collect certain sparse measurements with purely manual techniques and handheld instruments. Given that all our sensor readings are in Influx, naturally we want to store manual measurements there as well.

To make data entry easily available and simple we rely on web application frameworks to build those CRUD-style interfaces with a variety of user friendly extras in the UI. Generally, web app frameworks assume relational databases and provide ORMs and abstraction layers to make it easy to map data storage to the user actions. To our knowledge, InfluxDB is not supported by any of these technologies. Because of the assumptions baked into web app frameworks we’ve had to “wire in” InfluxDB library calls in somewhat ugly ways. I’d like to do this in a smarter way.

I’m curious if anyone has already come up with an elegant solution to this problem? Is there a framework out there that already natively supports InfluxDB? Is there some middleware that is able to translate InfluxDB reads and writes to a traditional SQL style for these integrations? Any thoughts on how to best incorporate a ‘raw’ InfluxDB connection into an otherwise basic CRUD-style web app?

Hello @mkarlesky,
Thanks for your question. This is a great question and crossing that divide is the challenge. I think @noahcrowley is working on something related to this.
Alternatively, have you checked out source and sink functions yet?

1 Like

Take a look at this talk from InfluxDays on using Flux to for data enrichment. It might help!

@katy and @Anaisdg, thank you both for your prompt responses.

I’ve certainly considered how Flux’s SQL abilities might be used to solve our problem. On the surface it seems promising. Unfortunately, as far as I can tell this won’t help us meet our need.

Web App ↔︎ Framework ↔︎ SQL Database

The essential issue is that our web applications need to offer classic CRUD abilities (Create, Read, Update, and Delete) for our manual measurement data records. The available frameworks map those actions to SQL-style databases quite nicely so that there’s very little to be considered in the way of database mechanics. If we want to use existing frameworks (we do) then as far as we can see we need some way for InfluxDB to pose as a SQL database in order for the web app framework features to be fully and properly utilized.

Flux’s SQL abilities are wonderful but do not appear to help us. requires source data to already be in InfuxDB (our apps are, in part, trying to Create data in InfluxDB). sql.from() might be able to help us in continually pulling timestamped data from a SQL database and writing it to InfluxDB, but it’s unlikely to help us with the needed Update and Delete actions in that pipeline. Deleting and updating records in InfluxDB works much differently than in a classic SQL database context.

I am very curious to know what @noahcrowley might be working on that is relevant to our need.