# Telegraf from 1.18 to 1.30

**URL:** <https://community.influxdata.com/t/telegraf-from-1-18-to-1-30/34680>\
**Category:** Telegraf\
**Tags:** telegraf\
**Created:** [June 24, 2024, 9:29am UTC](https://community.influxdata.com/t/telegraf-from-1-18-to-1-30/34680 "2024-06-24T09:29:17Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![BHARATH\_KUMAR](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/bharath_kumar/32/15817_2.png) [@BHARATH\_KUMAR](https://community.influxdata.com/u/BHARATH_KUMAR)\
**Post date:** [June 24, 2024, 9:29am UTC](https://community.influxdata.com/t/telegraf-from-1-18-to-1-30/34680/1 "2024-06-24T09:29:17Z")

</div>

Hello All,

I have a docker telegraf 1.18 and it’s working fine. Whenever a new file uploads into the s3 bucket it triggers the lambda function and it triggers the kinesis and telegraf and update the data into the influxdb.

Now i changed the telegraf to 1.30 and I changed the few telegraf config files to run the telegraf container:

My config files are as below:

4g-agent-telegraf.conf:  
[agent]  
interval = “10s”  
debug = false  
metric\_buffer\_limit = 100000  
metric\_batch\_size = 8000  
omit\_hostname = true  
logtarget = “file”  
logfile = “/etc/telegraf/log/telegraf.log”  
logfile\_rotation\_max\_size = “10MB”  
logfile\_rotation\_max\_archives = 3

4g-input-telegraf.conf:  
[[inputs.kinesis\_consumer]]  
region = “${REGION}”  
max\_undelivered\_messages = 500000 #2,00,000 #20,00,000  
#interval = “1s” #6s  
access\_key =“${ACCESS\_KEY}”  
secret\_key = “${SECRET\_KEY}”  
streamname = “chronos\_4g\_cu\_om”  
shard\_iterator\_type = “LATEST”  
data\_format = “csv”  
csv\_header\_row\_count = 1  
csv\_tag\_columns = [“column names”]

processor-telegraf.conf:  
[[processors.converter]]  
[processors.converter.tags]  
measurement = [“type”]  
[processors.converter.fields]  
tag = [“BcMacId”,“operatorName”,“eNBName”,“PlmnIdentity”,“cellName”,“eNodeBId”,“NoOfController”]

telegraf container logs:  
2024-06-24T07:19:28Z I! Starting Telegraf 1.30.0 brought to you by InfluxData the makers of InfluxDB  
2024-06-24T07:19:28Z I! Available plugins: 233 inputs, 9 aggregators, 31 processors, 24 parsers, 60 outputs, 6 secret-stores  
2024-06-24T07:19:28Z I! Loaded inputs: kinesis\_consumer (4x)  
2024-06-24T07:19:28Z I! Loaded aggregators:  
2024-06-24T07:19:28Z I! Loaded processors: converter  
2024-06-24T07:19:28Z I! Loaded secretstores:  
2024-06-24T07:19:28Z I! Loaded outputs: file (2x) influxdb\_v2 (6x)  
2024-06-24T07:19:28Z I! Tags enabled:  
2024-06-24T07:19:28Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:“”, Flush Interval:10s

 ![image](https://us1.discourse-cdn.com/flex023/uploads/influxdata/original/2X/c/c82b39cfb9e6eeef1af7c77cbc9e9feb82097931.png)

In the cloud watch I am able to see the lambda function triggered and data pushed to stream. But the telegraf not able to read the data.

Any config which I missed?

Thanks in Advance.

---

<div class="post-metadata">

**Author:** ![BHARATH\_KUMAR](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/bharath_kumar/32/15817_2.png) [@BHARATH\_KUMAR](https://community.influxdata.com/u/BHARATH_KUMAR)\
**Post date:** [June 25, 2024, 5:46am UTC](https://community.influxdata.com/t/telegraf-from-1-18-to-1-30/34680/2 "2024-06-25T05:46:14Z")

</div>

Can anyone help here?

---

<div class="post-metadata">

**Author:** ![scott](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/scott/32/16492_2.png) [@scott](https://community.influxdata.com/u/scott)\
**Post date:** [June 25, 2024, 2:43pm UTC](https://community.influxdata.com/t/telegraf-from-1-18-to-1-30/34680/3 "2024-06-25T14:43:25Z")

</div>

@BHARATH_KUMAR Is it that Telegraf isn’t able to read the data or is it not able to write it to InfluxDB. I can see that outputs are enabled, but there weren’t any outputs in the configuration files you provided.

---

<div class="post-metadata">

**Author:** ![BHARATH\_KUMAR](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/bharath_kumar/32/15817_2.png) [@BHARATH\_KUMAR](https://community.influxdata.com/u/BHARATH_KUMAR)\
**Post date:** [June 27, 2024, 11:51am UTC](https://community.influxdata.com/t/telegraf-from-1-18-to-1-30/34680/4 "2024-06-27T11:51:53Z")

</div>

@scott  
From kinesis I am able to see the message data pushed to stream… Later telegraf is not able to read and I am not seeing any error in telegraf logs

---

<div class="post-metadata">

**Author:** ![BHARATH\_KUMAR](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/bharath_kumar/32/15817_2.png) [@BHARATH\_KUMAR](https://community.influxdata.com/u/BHARATH_KUMAR)\
**Post date:** [June 27, 2024, 12:13pm UTC](https://community.influxdata.com/t/telegraf-from-1-18-to-1-30/34680/5 "2024-06-27T12:13:22Z")

</div>

[[outputs.influxdb\_v2]]

## Test bucket

urls = [“[http://influxdb:8086](http://influxdb:8086)”]  
tagexclude = [“data\_type”,“device\_type”,“host”,“operator”]  
namedrop = [“exe\_\*”]  
fieldexclude = [“TimeInUTCOffset”,“time”,“date”,“NodeCellId”]  
token = “${INFLUXDBTOKEN}”  
organization = “CS”  
bucket = “$BUCKET”  
flush\_interval = “10s”  
metric\_buffer\_limit = 120000  
[outputs.influxdb\_v2.tagpass]  
operator = [“CS”]  
data\_type = [“test”]

[[outputs.file]]  
files = [“stdout”]

The above is the output-telegraf conf file.

I am not able to see the data in influxdb and on the docker logs as well

Could anyone please help me?

Thanks in Advance

---

<div class="post-metadata">

**Author:** ![jpowers](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/jpowers/32/8569_2.png) [@jpowers](https://community.influxdata.com/u/jpowers)\
**Post date:** [June 27, 2024, 2:44pm UTC](https://community.influxdata.com/t/telegraf-from-1-18-to-1-30/34680/6 "2024-06-27T14:44:24Z")

</div>

fwiw I’ve seen folks have issues with kinesis\_consumer before, but never heard back from users as to what they found out to say one way or another:

- [Kinesis consumer not working from 1.20.3 onwards · Issue #13853 · influxdata/telegraf · GitHub](https://github.com/influxdata/telegraf/issues/13853)
- [Kinesis input consumer plugin not working after update · Issue #10197 · influxdata/telegraf · GitHub](https://github.com/influxdata/telegraf/issues/10197)

The theory was a configuration issue, but I don’t know what to point at.

---

<div class="post-metadata">

**Author:** ![BHARATH\_KUMAR](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/bharath_kumar/32/15817_2.png) [@BHARATH\_KUMAR](https://community.influxdata.com/u/BHARATH_KUMAR)\
**Post date:** [June 27, 2024, 2:51pm UTC](https://community.influxdata.com/t/telegraf-from-1-18-to-1-30/34680/7 "2024-06-27T14:51:08Z")

</div>

Thanks for your reply… Let me go through this doc.

---

<div class="post-metadata">

**Author:** ![BHARATH\_KUMAR](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/bharath_kumar/32/15817_2.png) [@BHARATH\_KUMAR](https://community.influxdata.com/u/BHARATH_KUMAR)\
**Post date:** [June 27, 2024, 2:54pm UTC](https://community.influxdata.com/t/telegraf-from-1-18-to-1-30/34680/8 "2024-06-27T14:54:56Z")

</div>

Actually they are getting some info in the logs but I am not getting any logs from the telegraf after uploading some files.

---

<div class="post-metadata">

**Author:** ![BHARATH\_KUMAR](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/bharath_kumar/32/15817_2.png) [@BHARATH\_KUMAR](https://community.influxdata.com/u/BHARATH_KUMAR)\
**Post date:** [July 9, 2024, 5:43am UTC](https://community.influxdata.com/t/telegraf-from-1-18-to-1-30/34680/9 "2024-07-09T05:43:22Z")

</div>

Hello All,

I tried changing the input configuration as below:

[[inputs.kinesis\_consumer]]

# ## Amazon REGION of kinesis endpoint.

region = “${REGION}”  
max\_undelivered\_messages = 500000 #2,00,000 #20,00,000  
#interval = “1s” #6s  
access\_key =“${ACCESS\_KEY}”  
secret\_key = “${SECRET\_KEY}”  
role\_arn = “${EC2\_PROFILE}”  
streamname = “$stream\_name”  
shard\_iterator\_type = “LATEST”  
data\_format = “csv”

## merge = “override”

```
csv_header_row_count = 1

```

csv\_tag\_columns = [“ver”,“node”,“operatorName”,“type”,“data\_type”,“cellId”,“node\_cellId”,“FriendlyName”,“NoOfController”,“SwVersion”,“Band”,“City”,“CustomerName”,“State”,“VenueType”,“HwVersion”,“USID”,“Region”,“Market”,“Transport”,“Venue”,“device\_type”,“DASType”,“NuetralHostName”,“Deployment”]  
csv\_timestamp\_format = “2006/01/02T15:04:05”

This is the kinesis input plugin configuration. Earlier for 1.18 version i used profile=“{$EC2\_PROFILE}”  
where we created a role for EC2 and it worked for it.. Later for 1.30 version with the same config I am getting the error as below:

[telegraf] Error running agent: starting input inputs.kinesis\_consumer: failed to get shared config profile,

Later I commented the profile and added the role\_arn where I passed the role\_arn as kinesis stream arn which it worked in one setup and again with the same config it didn’t worked for another setup.

Can anyone help where I went wrong?

output.conf:

[[outputs.influxdb\_v2]]

## Test bucket

urls = [“[http://influxdb:8086](http://influxdb:8086)”]  
tagexclude = [“data\_type”,“device\_type”,“host”,“operator”]  
namedrop = [“exe\_\*”]  
fieldexclude = [“TimeInUTCOffset”,“time”,“date”,“NodeCellId”]  
token = “${INFLUXDBTOKEN}”  
organization = “CS”  
bucket = “$bucket\_name”  
flush\_interval = “10s”  
metric\_buffer\_limit = 120000

# [outputs.influxdb\_v2.tagpass]

## operator = [“CS”]

data\_type = [“test”]

And I want to understand why I am not able to add profile as it is throwing error in 1.30 but not in 1.18 version

Thanks in Advance.

---

<div class="post-metadata">

**Author:** ![BHARATH\_KUMAR](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/bharath_kumar/32/15817_2.png) [@BHARATH\_KUMAR](https://community.influxdata.com/u/BHARATH_KUMAR)\
**Post date:** [July 10, 2024, 5:06am UTC](https://community.influxdata.com/t/telegraf-from-1-18-to-1-30/34680/10 "2024-07-10T05:06:14Z")

</div>

Could anyone help me in this?

---

<div class="post-metadata">

**Author:** ![jpowers](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/jpowers/32/8569_2.png) [@jpowers](https://community.influxdata.com/u/jpowers)\
**Post date:** [July 10, 2024, 1:45pm UTC](https://community.influxdata.com/t/telegraf-from-1-18-to-1-30/34680/11 "2024-07-10T13:45:25Z")

</div>

> shared config profile

As far as I know, there were no intentional changes to remove support.

v1.18 was released in July of 2020, that means there is almost 4 years of changes between the two versions you are trying to move to. It doesn’t make it easy to determine what might have changed.

> [telegraf] Error running agent: starting input inputs.kinesis\_consumer: failed to get shared config profile,

The biggest change I can think of is the move to the v2 AWS SDK. Looking at the SDK itself and that error message it appears to show up when in fact the config does not actually exist: [aws-sdk-go-v2/config/config.go at 0e07cc82b25692dce8f68e0b5bd0d0c5cdbcd279 · aws/aws-sdk-go-v2 · GitHub](https://github.com/aws/aws-sdk-go-v2/blob/0e07cc82b25692dce8f68e0b5bd0d0c5cdbcd279/config/config.go#L218)

Searching the web I found a couple posts about this error as well. It seems a number of issues due to anything from quotes, extra params and more:

- [amazon web services - Error: configuring Terraform AWS Provider: failed to get shared config profile, \<profile\> - Stack Overflow](https://stackoverflow.com/questions/75740175/error-configuring-terraform-aws-provider-failed-to-get-shared-config-profile)
- [amazon web services - Terraform: can't authenticate to aws provider using shared config file or static variables - Stack Overflow](https://stackoverflow.com/questions/60475880/terraform-cant-authenticate-to-aws-provider-using-shared-config-file-or-static)