Logging with filebeat

If needed filebeat logging can be activated on the devops stack to send the logs of chosen pods to any of the available outputs.

Prerequisites

  • Devops Stack version > 0.46.0

Activate filebeat

The deployment of filebeat is part of the loki-stack chart and needs to be set in the app_of_apps_values_overrides in your terraform modules :

 app_of_apps_values_overrides =  <<EOT
---
loki-stack:
  filebeat:
    enabled: true
    filebeatConfig:
      filebeat.yml: |
        filebeat.autodiscover:
          providers:
            - type: kubernetes
              hints.enabled: true
              hints.default_config.enabled: false
  EOT
}
Only the pods with the annotation co.elastic.logs/enabled set to "true" will be ingested.

You can also filter the lines with the include_lines annotation, for example :

podAnnotations:
  co.elastic.logs/enabled: "true"
  co.elastic.logs/include_lines: "[INFO].*"

Then you need a valid configuration for filebeat to send the logs to an output, see some examples below.

Configuration examples

Kafka with credentials :

 app_of_apps_values_overrides =  <<EOT
---
loki-stack:
  filebeat:
    enabled: true
    filebeatConfig:
      filebeat.yml: |
        filebeat.autodiscover:
          providers:
            - type: kubernetes
              hints.enabled: true
              hints.default_config.enabled: false
        output.file:
          enabled: false
        output.kafka:
          topic: "devopstack"
          hosts:
             - "kafka-broker-0.logs.test.com"
             - "kafka-broker-1.logs.test.com"
          username: "user"
          password: "secret"
          sasl.mechanism: "SCRAM-SHA-512"
          ssl.enabled: true
  EOT
}

Kafka with TLS client certificates :

  app_of_apps_values_overrides =  <<EOT
---
loki-stack:
  filebeat:
    enabled: true
    filebeatConfig:
      filebeat.yml: |
        filebeat.autodiscover:
          providers:
            - type: kubernetes
              hints.enabled: true
              hints.default_config.enabled: false
        output.file:
          enabled: false
        output.kafka:
          topic: "devopstack"
          hosts:
            - "kafka-broker-0.logs.test.com:1234"
            - "kafka-broker-1.logs.test.com:1234"
            - "kafka-broker-2.logs.test.com:1234"
          ssl.enabled: true
          ssl.certificate_authorities: "/usr/share/filebeat/certs/ca.crt"
          ssl.certificate: "/usr/share/filebeat/certs/tls.crt"
          ssl.key: "/usr/share/filebeat/certs/tls.key"
    secretMounts:
      - name: filebeat-certificates
        secretName: filebeat-certificates
        path: /usr/share/filebeat/certs
  EOT
}
If you are bootstrapping a devops stack you can inject the certificates and key directly in the values, for example: 
      filebeat.yml: |
          [...]
          ssl.certificate: "/usr/share/filebeat/tls.crt"
          ssl.key: "/usr/share/filebeat/tls.key"
      tls.crt: |
       -----BEGIN CERTIFICATE-----
       MIIE...
       -----END CERTIFICATE-----
      tls.key: |
       -----BEGIN PRIVATE KEY-----
       MIIE..
       -----END PRIVATE KEY-----
  EOT
}

Deploy specific version :

  app_of_apps_values_overrides =  <<EOT
---
loki-stack:
  filebeat:
    enabled: true
    image: 'docker.elastic.co/beats/filebeat'
    imageTag: '7.15.0'
  EOT
}