I want to collect:
- Server KPIs (cpu/ram/disk/etc);
- Web Access Logs (method/path/status/etc);
- Services up-status (serviceX is running, serviceY is down, etc);
I am already collecting Server KPIs into an InfluxDB server similar to:
InfluxDB --> database_xyz ----> measurement_CPU ------> tag layer: "web" tag server: "webserver-1" value cpu_idle: 93 value cpu_something: 4 ----> measurement_RAM ------> tag layer: "web" tag server: "webserver-1" value RAM_Free: 65536
Now I want to also collect, say, the Web Access Logs but I’m unsure how exactly InfluxDB is meant to be structured.
1) Should we add everything to the same measurement but filtered by Tags?
2) Should we create a different Measurement per … type of log (ex: ServerKPIs, WebAccessLogs) or per type of sub-log (ex: ServerKPI-CPU, ServerKPI-RAM)?
3) Should we create a different Database altogether per type of log?
4) Should we create a different InfluxDB server per type of log?
This just feels so clunky and ugly… However, looks to me completely feasible and I can’t think of any technical drawbacks. Can this give me the best flexibility/ease to in the future process the data (ex: CPU used per Request/s)?
Feels pretty standard to me, which is why I originally went with this structure. Am I losing any kind of ability to combine data?
Seems quite reasonable to separate such disparate logs, after all, they really are different logs. Would I lose ability to combine logs (ex: CPU used per Request/s)?
If I just create a new InfluxDB server (same machine), later scaling the system to remove bottlenecks should be quite easy, by moving each InfluxDB service into its own dedicated machine. Would I still be able to combine logs, in the likes of Grafana?