• Logs should be pro­tect­ed like backups. 
  • Remote
  • Mul­ti­ple versions
• Logs should be read-only
• Logs should be aggra­gat­ed dynam­i­cal­ly when necessary.
• Logs should be avail­able also after an attack.
• Logs should con­tain use­ful data 
  • A web serv­er logs which has only the request IPs of the fire­wall before it, is not so useful.
• The log for­mat should be ver­sa­tile and robust (e.g. JSONL ist bet­ter than CSV files)
• The time­stamps should be in the same for­mat and the same time­zone or con­tain also the timezone.

Ques­tions, who should be answered easily:

• Find every­thing what A did.
• Find every­thing releat­ed to X.
• Find events which are not like the others.

Tips for analyzing log files

Search a log and count the hosts:

cat log | jq -c '.host' | sort | uniq -c | sort -n

To see more about the host “Apache”

cat log | grep "Apache" | jq

Search in gzipped logs:

zgrep log.gz
zcat log.gz | grep "Apache"

Con­vert time­stamps on the fly:

cat log | jq -c '.ts|=strftime("%Y-%m-%d %H:%M:%S %z")' | jq