# Influxdb high CPU usage

**URL:** https://community.influxdata.com/t/influxdb-high-cpu-usage/976
**Category:** Systems
**Created:** [May 18, 2017, 6:26pm UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976 "2017-05-18T18:26:51Z")
**Posts on this page:** 18
**Page:** 1

<div class="post-metadata">

### Author: ![gcyre](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/gcyre/32/3845_2.png) [@gcyre](https://community.influxdata.com/u/gcyre)
#### Post date: [May 18, 2017, 6:26pm UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/1 "2017-05-18T18:26:51Z")

</div>

We’ve been running influxdb in production for about 6-8months and in dealing with some memory issues I noticed the CPU usage has been rising steadily this week to the point where performance on our Grafana graphs is being impacted. According to htop cpu usage is always over 80% and load average is constantly over 7

The server is a Dell with dual quad core, 128GB RAM, 3.2TB disk. There are 9 databases, the 2 largest databases have ~200K series each, the rest are small with less than 5K series.

The server itself is dedicated to influx but does have telegraf and kapacitor (no tick scripts running) services installed.

I read somewhere high cpu could be caused by garbage collection but don’t know what metrics to look at

any help would be appreciated.

thanks  
Garry

---

<div class="post-metadata">

### Author: ![jackzampolin](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/jackzampolin/32/5_2.png) [@jackzampolin](https://community.influxdata.com/u/jackzampolin)
#### Post date: [May 20, 2017, 12:58am UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/2 "2017-05-20T00:58:44Z")

</div>

Can you pull a CPU profile from the instance and post the results here?:

```auto
for i in block goroutine heap profile; do curl -sL -o influxdb-debug-$i.txt "http://localhost:8086/debug/pprof/$i?debug=1"; done && for i in $(seq 6); do curl -sL -o influxdb-debug-vars-${i}0s.txt "http://localhost:8086/debug/vars"; sleep 10; done && tar -cvzf influxdb-debug.$(hostname).$(date +%s).tar.gz influxdb-debug-*.txt && rm influxdb-debug-*.txt

```

---

<div class="post-metadata">

### Author: ![gcyre](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/gcyre/32/3845_2.png) [@gcyre](https://community.influxdata.com/u/gcyre)
#### Post date: [May 20, 2017, 1:50am UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/3 "2017-05-20T01:50:35Z")

</div>

thanks for the reply, I’m getting an error uploading the file. Unauthorized file ext.

thanks  
Garry

---

<div class="post-metadata">

### Author: ![jackzampolin](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/jackzampolin/32/5_2.png) [@jackzampolin](https://community.influxdata.com/u/jackzampolin)
#### Post date: [May 20, 2017, 5:25pm UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/4 "2017-05-20T17:25:45Z")

</div>

@gcyre Sorry about that! I’ve added `.txt`, `tar.gz` and `gz` to the allowed extensions. Can you try again?

---

<div class="post-metadata">

### Author: ![gcyre](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/gcyre/32/3845_2.png) [@gcyre](https://community.influxdata.com/u/gcyre)
#### Post date: [May 20, 2017, 5:40pm UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/5 "2017-05-20T17:40:56Z")

</div>

thanks @jackzampolin, now I’m getting a file size limit error. I’ve broken the file up into 2 files[influxdb-debug.pc-influxdb-004.1495244369.tar.gz](https://discoursecommunity.s3-us-west-1.amazonaws.com/original/1X/8436096dbc58ea15f4c54209a42a33447a8e6ee6.gz) (303.4 KB)

---

<div class="post-metadata">

### Author: ![gcyre](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/gcyre/32/3845_2.png) [@gcyre](https://community.influxdata.com/u/gcyre)
#### Post date: [May 20, 2017, 5:42pm UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/6 "2017-05-20T17:42:50Z")

</div>

Second file cannot be uploaded, still getting file size error

---

<div class="post-metadata">

### Author: ![gcyre](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/gcyre/32/3845_2.png) [@gcyre](https://community.influxdata.com/u/gcyre)
#### Post date: [May 24, 2017, 1:00pm UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/7 "2017-05-24T13:00:38Z")

</div>

@jackzampolin did you have an opportunity to look at these?

thanks

---

<div class="post-metadata">

### Author: ![gcyre](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/gcyre/32/3845_2.png) [@gcyre](https://community.influxdata.com/u/gcyre)
#### Post date: [June 1, 2017, 6:47pm UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/8 "2017-06-01T18:47:12Z")

</div>

@jackzampolin do you have any suggestions?

---

<div class="post-metadata">

### Author: ![jackzampolin](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/jackzampolin/32/5_2.png) [@jackzampolin](https://community.influxdata.com/u/jackzampolin)
#### Post date: [June 1, 2017, 7:01pm UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/9 "2017-06-01T19:01:59Z")

</div>

@gcyre Just noticed I had a message in our private conversation I haven’t sent you. Can you repost the google drive link here? I have to get one of our core engineers to come take a peek at this.

Sorry for the slow response here. This fell off of my radar.

---

<div class="post-metadata">

### Author: ![gcyre](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/gcyre/32/3845_2.png) [@gcyre](https://community.influxdata.com/u/gcyre)
#### Post date: [June 1, 2017, 7:45pm UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/10 "2017-06-01T19:45:28Z")

</div>

@jackzampolin no worries, appriciate your time looking into this. Here’s the link

> **[Google Drive: Sign-in](https://accounts.google.com/v3/signin/identifier?dsh=S1158062668%3A1674792552296468&continue=https%3A%2F%2Fdrive.google.com%2Ffile%2Fd%2F0Bw0v620SnsMUNWNXaHlBRUUtRUE%2Fedit&followup=https%3A%2F%2Fdrive.google.com%2Ffile%2Fd%2F0Bw0v620SnsMUNWNXaHlBRUUtRUE%2Fedit&passive=1209600&service=wise&flowName=WebLiteSignIn&flowEntry=ServiceLogin&ifkv=AWnogHffKRtMo4KvmKjFim2SIfodCiLC0fcXL98LyTrMOAkx9wolvHyHnN8AlK8dX04DbZuFZVKagw)**
>
> Access Google Drive with a Google account (for personal use) or Google Workspace account (for business use).

thanks  
Garry

---

<div class="post-metadata">

### Author: ![gcyre](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/gcyre/32/3845_2.png) [@gcyre](https://community.influxdata.com/u/gcyre)
#### Post date: [June 13, 2017, 1:37pm UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/11 "2017-06-13T13:37:29Z")

</div>

@jackzampolin have you had a chance to look at this?

thanks  
Garry

---

<div class="post-metadata">

### Author: ![ryan](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/ryan/32/26_2.png) [@ryan](https://community.influxdata.com/u/ryan)
#### Post date: [June 13, 2017, 2:46pm UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/12 "2017-06-13T14:46:56Z")

</div>

Sorry about the back and forth but we can’t read a profile without knowing the software version that you are running.

Offhand, I can think of a few reasons for seeing the server slow down.

1. You may be low on physical memory and the process swaps or can’t maintain an efficient cache of hot data. You can see this by looking at available memory at the process / OS level.
2. You may be running queries that scale to the number of series being read; as you read more series, the queries grow slower. You could run the queries manually and time them to understand their basic performance, perhaps?
3. You are writing data in a way that causes adverse compaction behavior (for example backfilling with new data or overwriting (updating) existing data that has been compacted).
4. You are sending data inefficiently. For example, you write a lot of old points or write only a few points per batch.

Also of note is that InfluxDB will do file I/O per database, per retention policy. If you have multiple hot databases, this can be expensive.

Completing the information for this bug reporting template (from GitHub/influxdb issues) would be helpful in trying to figure out what’s happening:

### Bug report

**System info:** [Include InfluxDB version, operating system name, and other relevant details]

**Steps to reproduce:**

1. [First Step]
2. [Second Step]
3. [and so on…]

**Expected behavior:** [What you expected to happen]

**Actual behavior:** [What actually happened]

**Additional info:** [Include gist of relevant config, logs, etc.]

Also, if this is an issue of for performance, locking, etc the following commands are useful to create debug information for the team.

```auto
curl -o profiles.tar.gz "http://localhost:8086/debug/pprof/all?cpu=true"

curl -o vars.txt "http://localhost:8086/debug/vars"
iostat -xd 1 30 > iostat.txt

```

**Please note** It will take at least 30 seconds for the first cURL command above to return a response.  
This is because it will run a CPU profile as part of its information gathering, which takes 30 seconds to collect.  
Ideally you should run these commands when you’re experiencing problems, so we can capture the state of the system at that time.

If you’re concerned about running a CPU profile (which only has a small, temporary impact on performance), then you can set `?cpu=false` or omit `?cpu=true` altogether.

Please run those if possible and link them from a [gist](http://gist.github.com) or simply attach them as a comment to the issue.

---

<div class="post-metadata">

### Author: ![gcyre](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/gcyre/32/3845_2.png) [@gcyre](https://community.influxdata.com/u/gcyre)
#### Post date: [June 13, 2017, 7:57pm UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/13 "2017-06-13T19:57:13Z")

</div>

[influxdb-debug.1497383714.tar.gz](https://discoursecommunity.s3-us-west-1.amazonaws.com/original/1X/383f4b384c991bef72710173c1e12be261f22805.gz) (2.7 MB)

[iostat.txt](https://discoursecommunity.s3-us-west-1.amazonaws.com/original/1X/eb52dac129d694f35ca04cd7027e873213ce4a47.txt) (7.2 KB)  
[vars.txt](https://discoursecommunity.s3-us-west-1.amazonaws.com/original/1X/6f95ebacf5cb4bdff6209aedbb3316c7d38b3048.txt) (801.1 KB)

---

<div class="post-metadata">

### Author: ![gcyre](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/gcyre/32/3845_2.png) [@gcyre](https://community.influxdata.com/u/gcyre)
#### Post date: [June 13, 2017, 7:57pm UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/14 "2017-06-13T19:57:29Z")

</div>

hi @ryan

Thanks for the response, here’s the relevant information, I received an error when running ‘curl -o profiles.tar.gz “[http://localhost:8086/debug/pprof/all?cpu=true](http://localhost:8086/debug/pprof/all?cpu=true)”’ (Unknown profile: all). I ran the command that Jack asked earlier in the thread.[influxdb-debug.1497383714.tar.gz](https://discoursecommunity.s3-us-west-1.amazonaws.com/original/1X/383f4b384c991bef72710173c1e12be261f22805.gz) (2.7 MB)

System Information:  
InfluxDB version: InfluxDB v1.1.1 (git: master e47cf1f2e83a02443d7115c54f838be8ee959644)  
OS version: Linux 3.19.0-68-generic #76~14.04.1-Ubuntu  
CPU: 2 X Intel Xeon E5-2609 (4 cores per CPU)  
RAM: 128GB  
Disks: 8x 500GB SAS

Databases:  
10 including \_internal  
Series: 8 databases 3-5K series  
1 database ~250K series  
1 database ~500K series

Actual behaviour  
CPU usage increased from 30% average to 70% average, increase happened on May 18 and has stayed steady at 70% since. InfluxDB service has been restarted a couple of times.

---

<div class="post-metadata">

### Author: ![ryan](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/ryan/32/26_2.png) [@ryan](https://community.influxdata.com/u/ryan)
#### Post date: [June 14, 2017, 8:22pm UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/15 "2017-06-14T20:22:15Z")

</div>

Thanks.

InfluxDB version: InfluxDB v1.1.1 (git: master e47cf1f2e83a02443d7115c54f838be8ee959644) is a pretty old version.

There are few memory leak and performance fixes in 1.1.2, 1.1.4. And substantial performance improvements on the 1.2.x branch as well.

Does anything prevent you from upgrading?

---

<div class="post-metadata">

### Author: ![gcyre](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/gcyre/32/3845_2.png) [@gcyre](https://community.influxdata.com/u/gcyre)
#### Post date: [June 15, 2017, 1:51pm UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/16 "2017-06-15T13:51:36Z")

</div>

Thanks Ryan,

I’ll look at upgrading to 1.4.2

Garry

---

<div class="post-metadata">

### Author: ![gcyre](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/gcyre/32/3845_2.png) [@gcyre](https://community.influxdata.com/u/gcyre)
#### Post date: [June 19, 2017, 12:57pm UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/17 "2017-06-19T12:57:05Z")

</div>

hi @ryan

I upgraded Influx to 1.2.4 over the weekend, CPU usage has dropped slightly and has been stable. Are there any stats I can look at to let me know the system is healthy?

thanks  
Garry

---

<div class="post-metadata">

### Author: ![quan](https://sea1.discourse-cdn.com/flex023/user_avatar/community.influxdata.com/quan/32/1876_2.png) [@quan](https://community.influxdata.com/u/quan)
#### Post date: [October 31, 2017, 10:26am UTC](https://community.influxdata.com/t/influxdb-high-cpu-usage/976/18 "2017-10-31T10:26:33Z")

</div>

I’m running InfluxDB v1.3.7 on an ARM embedded computer (BeagleBone). InfluxDB is the highest CPU usage. Can you please look into the debug log to see what wrong in my installation? [influxdb-debug.phen.1509444957.tar.gz](http://agriconnect.vn/media/influx/influxdb-debug.phen.1509444957.tar.gz)
