High CPU and RAM consume while writing

Store
1

Dear team,

I’m using InfluxDB to store website ranking (SERP data), I can say it’s alot.

The idea is when you do the search with “keyword research”, we will have a lot of website in the resuls.

We will store first 100 results into SERP.

Here’s the data schema:

{
schema: [
    {
        measurement: 'keyword_daily',
        fields: {
            domain: Influx.FieldType.STRING,
            link: Influx.FieldType.STRING,
            title: Influx.FieldType.STRING,
            desc: Influx.FieldType.STRING,
            keyword: Influx.FieldType.STRING,

            position: Influx.FieldType.INTEGER,
            keyword_id: Influx.FieldType.INTEGER,
            campaign_id: Influx.FieldType.INTEGER,
            unique_key: Influx.FieldType.STRING,
        },
        tags: []
    }
]
}

Here’s the sample data: JSON Editor Online: JSON editor, JSON formatter, query JSON

Curently, I have 10 bot that run frequently to crawl google data, after that, it will save to InfluxDb via InfluxDB-PHP SDK.

The problem: I can see that the RAM usage and the CPU consume is at the top, here’s the graph to say it all.

CPU Usage:

26-11_02_51
26-11_02_511638×1217 77 KB

RAM Usage: screenshot-ca.ovh.com-2019.09.26-11_03_45 - Nimbus Capture

I don’t know why/what is the problem.

My installation specs:

  • 32Gb RAM 4 core 8 threads
  • Centos 7
  • InfluxDB 1.7.8

Thank you for your time!