Update of "logfmt1"
Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Artifact ID: | b50bde087462897cce397825b6c409d3a170baf90ae85e7a7758aa3b39432e6d |
---|---|
Page Name: | logfmt1 |
Date: | 2020-12-16 10:36:46 |
Original User: | mario |
Mimetype: | text/x-markdown |
Next | 642634a9c85aa607bcdf9173257e33f35d56f6b26c071157ee525d663390af9a |
logfmt1 handles *.log.fmt
files to transform LogFormat / placeholder
strings to regular expressions (named capture groups). Currently just comes
with rules for Apache definitions. It bundles a logex
and update-logfmt
to create/rewrite *.log.fmt
files globally.
{
"class": "apache combined",
"record": "%h %l %u %t \"%r\" %>s %b",
}
It's basically meant for universal log parsing, whilst reducing manual configuration or the restrain on basic log variants. It originated in modseccfg. This Python package is mostly a stub. You should preferrably install the system package:
apt install python3-logfmt1
This will yield the proper /usr/share/logfmt/
structure and the run-parts
wrapper update-logfmt
. The grok placeholders are supported, but remain
untested.
logfmt1
To craft a regex:
import logfmt1, json
fmt = json.load(open("/.../access.log.fmt", "r"))
rx = logfmt1.regex(fmt)
rx = logfmt1.rx2re(rx) # turn into Python regex
Or with plain old guesswork / presuming a standard log format:
rx = logfmt1.regex({"class": "apache combined"})
Though that's of course not the intended use case, and hinges on predefined formats in /usr/share/logfmt/.
logex
Very crudementary extractor for log files:
logex .../access.log --tab @host @date +id
Which of course handles the .fmt
implicitly.
update-logfmt
The Python package does bundle a run-parts wrapper, but just the apache
collector, and a local Python copy of the format database. It should discover
all *.log
files nonetheless and pair them with .fmt
declarations.