I am trying to figure out how to find out how much data I am sending to my influxdb so that I can project the growth, I have used this query although I am not sure I quiet understand the figures, I would like to have it in Mbytes
> select sum(writeReqBytes) from httpd where time > now() - 1d group by time(1d)
name: httpd
time sum
---- ---
1492473600000000000 23469208528260229
1492560000000000000 1588509235646268
@lanox You should be able to see this with du -h /var/lib/influxdb. You could also install telegraf on the instance with the disk plugin enabled. This information is not in internal by default. The ingest statistics will be very incorrect due to the compression we apply once the data is in the database.
@jackzampolin I wish it was that simple, We are paying for cloud instance and I do not have access to the server only DB it self. I can’t seem to get disk information from Internal DB or I am doing something wrong.
The measurement you’re querying for in your example is giving you how much data you’ve processed from the httpd endpoint over your timeframe, but doesn’t have a direct correlation to what’s stored on disk for that data (to Jack’s point). Since the backend is compressing the data quite extensively, it won’t give you insight into what’s actually consuming your disk.
That being said, you can look at the tsm1_filestore metrics from the _internal table if you enable collection on the database level itself. These may not directly correlate to disk consumption, but they may get you closer to the measurements you’re after.
> use _internal
Using database _internal
> show measurements
name: measurements
name
----
database
httpd
queryExecutor
runtime
shard
subscriber
tsm1_cache
tsm1_engine
tsm1_filestore
tsm1_wal
write
> show field keys from tsm1_filestore
name: tsm1_filestore
fieldKey fieldType
-------- ---------
diskBytes integer
numFiles integer
> show tag keys from tsm1_filestore
name: tsm1_filestore
tagKey
------
database
engine
hostname
id
path
retentionPolicy
walPath
> select mean(diskBytes) from tsm1_filestore where time > now() - 1m group by time(10s),"database",path
name: tsm1_filestore
tags: database=_internal, path=/data/influxdb-data/metrics/_internal/monitor/55
time mean
---- ----
1492731790000000000
1492731800000000000 3.889556e+06
1492731810000000000 3.889556e+06
1492731820000000000 3.889556e+06
1492731830000000000 3.889556e+06
1492731840000000000 3.889556e+06
1492731850000000000 3.889556e+06
name: tsm1_filestore
tags: database=_internal, path=/data/influxdb-data/metrics/_internal/monitor/57
time mean
---- ----
1492731790000000000
1492731800000000000 3.777064e+06
1492731810000000000 3.777064e+06
1492731820000000000 3.777064e+06
1492731830000000000 3.777064e+06
1492731840000000000 3.777064e+06
1492731850000000000 3.777064e+06
...
My snippet of code only shows data from the _internal database itself, but adding the "database" to group by will do the needful and split along all shards on on dbs for your host
