Batch Processing vs. Stream Processing: What’s the Difference?

Stream tasks fail for a particular alerting application that seems like it would be common: Alerting on a stateDuration triggered by an infrequently sampled data point such as a Boolean that is written to the database only on change. Let’s say you have a condition like, “Alert me whenever this Boolean has been true for over 10 seconds” If the data points that Boolean depends on are only written on change, or at a frequency less than the resolution you want to alert at, then the alert will never trip. The Boolean will become true, causing the stateDuration node with a unit of 1s to emit “duration=0” and then since the data is not sampled after this until the next change, the alert based on a condition like “duration > 10” won’t fire at the actual 10 second mark, but only when the next data point comes in after 10 seconds. If that data happens to reset the Boolean to false, no alert will happen.

In this case I just have to use a batch task and query at a frequency higher than the resolution I want to alert at.

I have written about this issue here on the forum and here as an open issue on github and received no response about the correct way to work around this subtle but significant limitation of stream tasks and the stateDuration node.