Different Time Results Causing Issues

Hi,

I’m very new to InfluxDB, and programming in general, but I have come across my first problem that has been a little stumped.

I will try and explain what I am doing.

I have an InfluxDB that I use for collecting the “blinks” from my electricity meter, this uses a Pi 1B and a bit of Python - nothing too clever but represented a significant learning step for me.

This script occasionally stops working, bound to be my buggy code so I wrote a little bit of Python that checks to see if the last update was more than 5 mins ago.

This works 99% of the time, it crashes occasionally with a date error when I try to convert the date string from the result set into a DateTime type so I can do comparisons against it.

As far as I can tell 99.9% of the time the date returned is in this format;

2020-06-09T13:07:36.723Z

VERY occasionally, I get it in this format

2020-06-09T10:43:28Z

Originally my scripts weren’t coded to handle that and crashed, I have since addressed this and these changes will not cause issues. But…

Is there any potential explanation for why this happens

Is anyone able to share any thoughts to help me understand it, or shall I just ignore it :slight_smile:

The query I am using is SELECT last(“blink”) FROM “leccy”.“autogen”.“house”

As I said above this is a new journey for me, I have some history in RDMS and tend to try and use InfluxDB with that mindset which I know can be wrong.

Thanks in advance.

Jon

I have an InfluxDB that I use for collecting the “blinks” from my
electricity meter, this uses a Pi 1B and a bit of Python - nothing too
clever but represented a significant learning step for me.

This script occasionally stops working, bound to be my buggy code so I
wrote a little bit of Python that checks to see if the last update was
more than 5 mins ago.

Have you considered using “monit” to restart the script if it falls over?

This works 99% of the time, it crashes occasionally with a date error when
I try to convert the date string from the result set into a DateTime type
so I can do comparisons against it.

Why not just use Epoch-format timestamps from the start?

As far as I can tell 99.9% of the time the date returned is in this format;

2020-06-09T13:07:36.723Z

VERY occasionally, I get it in this format

2020-06-09T10:43:28Z

Is there any potential explanation for why this happens

My guess is that you get the second format when the seconds part of the
timestamp is exact (no decimals).

In other words 2020-06-09T10:43:28Z is 2020-06-09T10:43:28.000Z

Without seeing the code which generates such a timestamp I can’t be sure, but
that’s what I suspect is going on.

For what it’s worth, my recommendation is to use Epoch timestamps such as
1591708056.723 in all of your code and calculations, and only convert to
something (slightly more) human-readable such as 2020-06-09T13:07:36.723Z when
you need to.

Regards,

Antony.

Thanks for your suggestions, I’ll need to do some research on monit and Epoch dates, I get why that could be a great idea just not used it in Python before.

I did think that maybe there was a data entry exactly on 00 seconds etc, but alas no;

2020-06-09T10:43:01.219Z
2020-06-09T10:43:03.647Z
2020-06-09T10:43:06.113Z
2020-06-09T10:43:08.555Z
2020-06-09T10:43:11.012Z
2020-06-09T10:43:13.464Z
2020-06-09T10:43:15.923Z
2020-06-09T10:43:18.355Z
2020-06-09T10:43:20.802Z
2020-06-09T10:43:23.239Z
2020-06-09T10:43:25.643Z
2020-06-09T10:43:28.000Z
2020-06-09T10:43:30.388Z
2020-06-09T10:43:32.731Z
2020-06-09T10:43:35.084Z
2020-06-09T10:43:37.493Z
2020-06-09T10:43:39.836Z
2020-06-09T10:43:42.198Z
2020-06-09T10:43:44.546Z
2020-06-09T10:43:46.899Z
2020-06-09T10:43:49.274Z
2020-06-09T10:43:51.646Z
2020-06-09T10:43:54.023Z
2020-06-09T10:43:56.398Z
2020-06-09T10:43:58.786Z

When you say “share the code that generates the timestamp”, do you mean the code that’s inserting the data?

Thanks,

J

I did think that maybe there was a data entry exactly on 00 seconds etc,
but alas no;

I didn’t mean zero seconds, I meant a seconds value with zero decimals, such
as (from your list):

2020-06-09T10:43:28.000Z

Therefore this clearly isn’t the problem.

When you say “share the code that generates the timestamp”, do you mean the
code that’s inserting the data?

I don’t know. I mean whatever code is creating the timestamp which sometimes
has decimal seconds and occasionally doesn’t.

That might be the code that’s inserting the data; it might be something
earlier in the process; I can’t tell without knowing more about what you’ve
built.

Antony.

Actually, that’s the exact line that caused the problem, me staring at the numbers couldn’t see it but you’ve pulled it right out. So it is therefore very simple if it has an entry with all zeros as MS, it doesn’t return those values in this query.

SELECT last(“blink”) FROM “leccy”.“autogen”.“house”

I think, as you’ve mentioned before, I need to use Epoch times to avoid any issues like this.

J

BTW, just wanted to pop back and say thank you for the monit advise, really like it and the lightweight approach.

:slight_smile:

You’re most welcome. I like it for similar reasons.

Antony.