Log parsing is a curse, because each application has its own format, and oftentimes configurable fields at that. Various attempts at standardizing logs have failed, or are bound to. Logging services and database storage are largely just symptomatic kludges, with JSON logs and not-quite-JSON formats held back by inertia.

Instead logfmt1 aims to have descriptors for each log file, in order to make them parseable. You can't attempt anything but guesswork until you know what's in a file.

So the idea is to have a *.fmt next to each *.log file, with a descriptor such as:

{
   "class": "apache combined",
   "record": "%h %l %u %t \"%r\" %>s %b"
}

Notably the "record" field should be the most current format string that the application itself uses. In order to resolve the placeholders, an application reference is kept in "class". Which allows combining the format string with placeholder field definitions from the global .fmt database (/usr/share/logfmt) database.

common classes

If the *.log itself is just JSON, then the .log.fmt would specify it simply as:

{
   "class": "json appmoniker"
}

For widely-known formats such as the "combined" Apache logs, the class might also suffice. But you can't rely on the user not having modified the LogFormat within the Apache settings then.

additional fields

The *.log.fmt itself might declare definitions such as aliases and more specific/custom placeholders.

{
   "class": "apache cust3",
   "record": "%a %h %{iso}t '%r' %s",
   "fields": {
       "%{iso}t": { "id": "datetime", "rx": "..." }
   },
   "alias": {
       "iso8601": "datetime",
   }
}

Which ought to be joined and override any global fmt definitions. Though such user customizations are more likely to be applied there anyway. Care should be taken by update-logfmt or applications to not jettison user-customized *.log.fmt options.