解析 json 日志（Parsing json logs）

https://www.elastic.co/guide/en/beats/filebeat/8.7/running-on-kubernetes.html#_parsing_json_logs

It is common case when collecting logs from workloads running on Kubernetes that these applications are logging in json format. In these case, special handling can be applied so as to parse these json logs properly and decode them into fields. Bellow there are provided 2 different ways of configuring filebeat’s autodiscover so as to identify and parse json logs. We will use an example of one Pod with 2 containers where only one of these logs in json format.

在从运行在Kubernetes上的工作负载收集日志时，这些应用程序记录的内容通常都是json格式。在这种情况下，可以应用特殊处理，以正确解析这些json日志并将其解码为字段。下面提供了两种不同的方法来配置Filebeat的自动发现，以识别和解析json日志。我们将使用一个Pod示例，其中有2个容器，仅一个以json格式记录日志。

示例日志（选自 elastic.co）

{"type":"log","@timestamp":"2020-11-16T14:30:13+00:00","tags":["warning","plugins","licensing"],"pid":7,"message":"License information could not be obtained from Elasticsearch due to Error: No Living connections error"}

1、Using json.* options with templates.（使用带模版的 json.* 选项）

filebeat.autodiscover:
  providers:
      - type: kubernetes
        node: ${NODE_NAME}
        templates:
          - condition:
              contains:
                kubernetes.container.name: "no-json-logging"
            config:
              - type: container
                paths:
                  - "/var/log/containers/*-${data.kubernetes.container.id}.log"
          - condition:
              contains:
                kubernetes.container.name: "json-logging"
            config:
              - type: container
                paths:
                  - "/var/log/containers/*-${data.kubernetes.container.id}.log"
                json.keys_under_root: true
                json.add_error_key: true
                json.message_key: message

2、Using json.* options with hints.（使用带提示的 json.* 选项）

Key part here is to properly annotate the Pod to only parse logs of the correct container as json logs. In this, annotation should be constructed like this:

这里的关键部分是正确注释Pod，以仅解析正确容器的JSON日志。在此方面，注释应该像这样构建：

co.elastic.logs.<container_name>/json.keys_under_root: "true"

自动发现配置：

filebeat.autodiscover:
  providers:
    - type: kubernetes
      node: ${NODE_NAME}
      hints.enabled: true
      hints.default_config:
        type: container
        paths:
          - /var/log/containers/*${data.kubernetes.container.id}.log

然后正确注解 Pod：

annotations:
    co.elastic.logs.json-logging/json.keys_under_root: "true"
    co.elastic.logs.json-logging/json.add_error_key: "true"
    co.elastic.logs.json-logging/json.message_key: "message"

JZTXT

filebeat篇章——解析 json 日志

解析 json 日志（Parsing json logs）