Failing data writes with influxdb_client python

InfluxDB 2
1

Hi,

I’m wrote a python script which reads MQTT messages, parses the data and pushes the output to influxdb V2 OSS. All of this is done on a 1 second interval. The issue I have is that data is pushed to influx only the first message cycle, from then the writes are not arriving in influx but I don’t get errors from the influx write client. New mqtt message are clearly received and the updated line protocols are also created so that cannot be the problem. The influxclient keeps “writing” and debug logging always shows “response 204”. Can anyone assist?

Code overview:

while True:
    # 1. Waint untill paho MQTT client receives new messages and set flag false
    while no_received_flag:
        time.sleep(0.1)
    # 2. Parse messages and extract data  
    # 3. Create list "lps" with line protocol strings of all data points (112 in total)
    # 4. Write data to influx
    try:
        write_options = WriteOptions(
                                batch_size=500,
                                flush_interval=500)

        with client.write_api(write_options=write_options) as write_api:
            write_api.write(bucket=database['bucket'], record=lps, write_precision=WritePrecision.MS)
            
    except InfluxDBError as e:
        print(e)

Some details:
• the entire program runs in a while true loop for cycling processing based on time interval (1s)
• Influx line protocols are created and collected in a large list, and it sends all line protocols in one go (112 in total)
• I tried changing the write options as such but no changes
write_options = WriteOptions(
batch_size=500,
flush_interval=500)

Any idea makes the write_api stop after one write?

2

Hello @SamR,
First might I ask why you’ve decided to use Python instead of Telegraf and the MQTT input plugin:

Here’s an example of using it:

Hmm i don’t think its a batching issue.
I wonder if your series are the same for multiple lines? If your series is the same, you’ll overwrite your data. Have you checked to see if this could be causing the problem?