I want to store access logs of my HTTP Server with InfluxDB.
My expectations are:
1- List last 1 hour, last 1 days, last 30 days access logs
2- Get last 1 hour, last 1 days, last 30 days unique visitor counts
3- Some of these request has been blocked by waf, i will tag these with “blocked”. List which logs are blocked
Is this suitable for me?
I want to store access logs of my HTTP Server with InfluxDB.
My expectations are:
1- List last 1 hour, last 1 days, last 30 days access logs
When you say “access” logs, what precisely do you mean? I can think of at
least two possible things of interest:
a) accesses to your web server, independently of which pages / images / other
items are being accessed
b) accesses to your web server, broken down by site / page / item
2- Get last 1 hour, last 1 days, last 30 days unique visitor counts
What format is your raw data in, and do you have a method for parsing
“visitor” into a field usable by InfluxDB (or any other DB for that matter)?
3- Some of these request has been blocked by waf,
Er, what’s a waf?
i will tag these with “blocked”. List which logs are blocked
Is this suitable for me?
I would ask:
i) it’s all very well saying you’re considering InfluxDB as a data store, but
what do you plan to use as the visualisation tool to take the data out of your
data store and display it in some meaningful way?
ii) do you have something in mind for how you would like to see that processed
data - for example “list last 1 day’s access logs” - do you really just mean a
list of what got accessed during the past 24 hours, or are you looking for
some sort of summary / collation of that data?
iii) have you looked at existing web server log analysis tools, and identified
shortcomings of them which mean you feel it’s better to create your own?
Antony.
