# Kapacitor stream processing multiple fields

**URL:** <https://community.influxdata.com/t/kapacitor-stream-processing-multiple-fields/3240>\
**Category:** Kapacitor\
**Tags:** kapacitor\
**Created:** [November 29, 2017, 2:02pm UTC](https://community.influxdata.com/t/kapacitor-stream-processing-multiple-fields/3240 "2017-11-29T14:02:35Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![mantzas](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/mantzas/32/1656_2.png) [@mantzas](https://community.influxdata.com/u/mantzas)\
**Post date:** [November 29, 2017, 2:02pm UTC](https://community.influxdata.com/t/kapacitor-stream-processing-multiple-fields/3240/1 "2017-11-29T14:02:35Z")

</div>

Hi,

i am trying to do a stream process for downsampling as follows

```
var period = 1s
var interval = 1s

var data = batch
|from()
    .database('db')
    .retentionPolicy('autogen')
    .measurement('ms1')
    .groupBy(*)
|eval(lambda: "latency1" - "latency2")
    .as('latency_diff')        
|mean('latency_diff')
    .as('latency_diff')
|mean('latency1')
    .as('latency1')
|mean('latency2')
    .as('latency2')
|window()
    .period(period)
    .every(interval)
|influxDBOut()
    .database('db_cold')
    .retentionPolicy('autogen')
    .measurement('ms1')
    .precision('s')   

```

but it does not work. using kapacitor cli i have found that some of the mean calcs return errors.

is the above possible?

---

<div class="post-metadata">

**Author:** ![philb](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/philb/32/1805_2.png) [@philb](https://community.influxdata.com/u/philb)\
**Post date:** [November 30, 2017, 9:40am UTC](https://community.influxdata.com/t/kapacitor-stream-processing-multiple-fields/3240/2 "2017-11-30T09:40:49Z")

</div>

Hi,

Do you get an error message when defining the TICK script in Kapacitor?  
Firstly i think you need to change the line

var data = _batch_ to var data = _stream_

I think you might need to stream each value you want to measure separately and then join them. I’m not 100% sure though.

the link below might help.  
[joining data](https://docs.influxdata.com/kapacitor/v1.3/guides/join_backfill/)

---

<div class="post-metadata">

**Author:** ![mantzas](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/mantzas/32/1656_2.png) [@mantzas](https://community.influxdata.com/u/mantzas)\
**Post date:** [November 30, 2017, 9:52am UTC](https://community.influxdata.com/t/kapacitor-stream-processing-multiple-fields/3240/3 "2017-11-30T09:52:59Z")

</div>

Hi,

the stream/batch was a type, it is stream.  
do you suggest that multiple values can only be handled with multiple stream that are then joined in the end?

---

<div class="post-metadata">

**Author:** ![philb](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/philb/32/1805_2.png) [@philb](https://community.influxdata.com/u/philb)\
**Post date:** [November 30, 2017, 10:29am UTC](https://community.influxdata.com/t/kapacitor-stream-processing-multiple-fields/3240/4 "2017-11-30T10:29:18Z")

</div>

Based on the little knowledge i have i think that would be the way to go.

I’m relatively new to TICK but from what i read about the join node i think that would work.

---

<div class="post-metadata">

**Author:** ![mantzas](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/mantzas/32/1656_2.png) [@mantzas](https://community.influxdata.com/u/mantzas)\
**Post date:** [November 30, 2017, 10:48am UTC](https://community.influxdata.com/t/kapacitor-stream-processing-multiple-fields/3240/5 "2017-11-30T10:48:30Z")

</div>

it works but i have to create a stream for every filed i want to join in the end.  
if like, i have, you have 3 fields you would need 3 streams and join them in order to get the data in db.

---

<div class="post-metadata">

**Author:** ![philb](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/philb/32/1805_2.png) [@philb](https://community.influxdata.com/u/philb)\
**Post date:** [November 30, 2017, 2:46pm UTC](https://community.influxdata.com/t/kapacitor-stream-processing-multiple-fields/3240/6 "2017-11-30T14:46:16Z")

</div>

Hi,

Yep that’s how i understand it. AFAIK you stream them, do any aggregations on them that you want to do and store the value of that. Then you join them together to build the alert. i think it joins them back together based on the UNIX time stamp in the database.

Also, I’m not sure if you’re using telegraf but your script is looking for data every second but i think the default for telegraf itself is 10s so you might end up with points where there is no data. That might actually cause it to not work if there is no data. Or you might get false alarms.

There might be an easier or better way to do this but i haven’t found it yet. I don’t do much streaming based on multiple measurements. A majority of scripts i write i’m told which measurement i need to alert on and use that.

If you can define and enable your script in Kapacitor then a good tool to use is ‘kapacitor show task\_name’ once it is running - That should give you an idea as to where abouts your script gets to before it stops working. I’ve found it quite useful over the last few days.

---

<div class="post-metadata">

**Author:** ![mantzas](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/mantzas/32/1656_2.png) [@mantzas](https://community.influxdata.com/u/mantzas)\
**Post date:** [November 30, 2017, 4:02pm UTC](https://community.influxdata.com/t/kapacitor-stream-processing-multiple-fields/3240/7 "2017-11-30T16:02:19Z")

</div>

Hi,

kapacitor show, showed me that the script did not work due to errors.  
the metric are custom directly from the application and do not use telegraf.

---

<div class="post-metadata">

**Author:** ![mantzas](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/mantzas/32/1656_2.png) [@mantzas](https://community.influxdata.com/u/mantzas)\
**Post date:** [November 30, 2017, 4:52pm UTC](https://community.influxdata.com/t/kapacitor-stream-processing-multiple-fields/3240/8 "2017-11-30T16:52:16Z")

</div>

after watching today’s webinar about advanced kapacitor the problem is solved the following way:

```
var period = 1s
var interval = 1s

var data = stream
|from()
    .database('db')
    .retentionPolicy('autogen')
    .measurement('ms1')
    .groupBy(*)
|window()
.period(period)
.every(interval)

var diff = data |eval(lambda: "latency1" - "latency2")
           .as('latency_diff')        
          |mean('latency_diff')
          .as('latency_diff')

var latency1 = data | mean('latency1')
                .as('value')

var latency2 = data|mean('latency2')
                .as('value')

diff 
| join(latency1,latency2)
        .as('lat1', 'lat2')
|influxDBOut()
    .database('db_cold')
    .retentionPolicy('autogen')
    .measurement('ms1')
    .precision('s')

```

thanks to Michael DeSa

---

<div class="post-metadata">

**Author:** ![philb](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/philb/32/1805_2.png) [@philb](https://community.influxdata.com/u/philb)\
**Post date:** [November 30, 2017, 4:58pm UTC](https://community.influxdata.com/t/kapacitor-stream-processing-multiple-fields/3240/9 "2017-11-30T16:58:37Z")

</div>

Ah good news! glad you worked it out. Nice of you to post the solution as well!

---

<div class="post-metadata">

**Author:** ![mantzas](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/mantzas/32/1656_2.png) [@mantzas](https://community.influxdata.com/u/mantzas)\
**Post date:** [November 30, 2017, 5:52pm UTC](https://community.influxdata.com/t/kapacitor-stream-processing-multiple-fields/3240/10 "2017-11-30T17:52:45Z")

</div>

i thought it might interest you as well. thanks for your input

---

<div class="post-metadata">

**Author:** ![philb](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/philb/32/1805_2.png) [@philb](https://community.influxdata.com/u/philb)\
**Post date:** [December 1, 2017, 9:25am UTC](https://community.influxdata.com/t/kapacitor-stream-processing-multiple-fields/3240/11 "2017-12-01T09:25:48Z")

</div>

It’s appreciated, as it happens I’m working on something to do with latency so it could prove helpful.

No worries.
