Fluent Bit v4.0.0 Release Announcement

The Fluentbit team is excited to announce the release of Fluent Bit v4.0.0, the latest version of our fast and lightweight telemetry agent designed for Linux, BSD, macOS, and Windows systems.

As a graduated project under the Cloud Native Computing Foundation (CNCF) umbrella of Fluentd, Fluent Bit continues to evolve, offering enhanced features and improvements to meet the growing demands of cloud and containerized environments.

What is Fluent Bit ?

Fluent Bit serves as a versatile telemetry agent capable of collecting logs, metrics, and traces from various sources, processing them with filters, and forwarding them to multiple destinations. With a strong emphasis on performance and low resource consumption, it has become the preferred choice for many organizations seeking efficient data processing solutions. This major release introduces several significant enhancements aimed at improving log processing capabilities, security, and extensibility. Below are the key highlights of Fluent Bit v4.0.0:

Key Features and Enhancements

Conditional Log Processing

Fluent Bit v4.0.0 introduces a powerful new feature that allows processors to conditionally modify logs based on specific field values. This enhancement provides granular control over log processing, enabling users to apply a variety of conditional operators such as and, or, in, gt, lt, regex, and more. For example, users can now insert a priority field with a value of high into log records where the log level is error.

Example Configuration:

pipeline:
  processors:
    logs:
      - name: content_modifier
        match: '*'
        condition:
          operator: AND
          rules:
            - field: "$log[\"level\"]"
              operator: eq
              value: "error"
        action: insert
        context: log_body
        key: priority
        value: high

Trace Sampling Processor

This release introduces a new trace sampling processor with a pluggable architecture, supporting various sampling strategies and backends. It includes both head sampling (probabilistic) and tail sampling with conditions, allowing for more efficient and targeted trace data collection.

Head Sampling (Probabilistic): Enables probabilistic sampling of traces based on a specified sampling percentage.

Tail Sampling (Conditional): Evaluates traces based on various conditions before making a sampling decision. Available conditions include:

• Status Code
• Latency
• String Attribute
• Numeric Attribute
• Boolean Attribute
• Span Count
• Trace State

Example Configuration:

pipeline:
  inputs:
    - name: opentelemetry
      port: 4318
      processors:
        traces:
          - name: sampling
            type: tail
            sampling_settings:
              decision_wait: 2s
            conditions:
              - type: boolean_attribute
                key: "user.logged"
                value: false

Enhanced Security Mechanisms

Fluent Bit v4.0.0 brings several security enhancements to provide users with greater control and flexibility:

• TLS Configuration Enhancements: Users can now specify minimum and maximum allowed TLS versions (tls.min_version and tls.max_version) and define acceptable ciphers (tls.ciphers) to strengthen communication security.
• Environment Variable File Reference: Fluent Bit now supports referencing internal variables from the file system, facilitating secure handling of sensitive data like secrets or tokens. Secrets or bearer tokens can be stored in files and referenced in configuration files using the file:// prefix.

Example Configuration:

env:
  FLUSH_INTERVAL: 1
  TOKEN: file://mysecret.txt

service:
  flush: ${FLUSH_INTERVAL}
  log_level: info

pipeline:
  inputs:
    - name: dummy
      dummy: '{"message": "Hello, World!", "secret": "${TOKEN}"}'

  outputs:
    - name: http
      match: '*'
      host: example.com
      port: 443
      uri: /ingest
      format: json
      tls: on
      header: Bearer ${TOKEN}

Experimental Features

Fluent Bit v4.0.0 includes experimental support for plugins written in the Zig programming language. This new capability allows developers to write high-performance plugins leveraging Zig’s modern features and safety guarantees. Note that experimental features are disabled by default but can be enabled at build time.

Plugin Improvements

Input Plugins

OpenTelemetry:

• Added JSON trace support.
• Enforced 16MB max gRPC message size.
• eBPF: Fixed skeleton compilation.
• Kubernetes Events: SQLite DB cleanup fixes.
• HTTP: Added gzip and zstd compression support.

Filters & Processors

• Kubernetes Filter: Added owner_references metadata.
• Sampling Processor: New tail/head sampling logic.
• Labels Processor: Dynamic buffer fixes.

Output Plugins

• OpenTelemetry: zstd compression support.
• HTTP: Fixed compressed payload handling.
• Loki: Structured metadata key support.
• Azure Kusto: Managed identity authentication.

Upgrading Notes

For users upgrading from previous versions, it is essential to review the Upgrading Notes section of our documentation to ensure a smooth transition.

Contributors

We extend our gratitude to the numerous contributors who have played a vital role in this release through coding, bug reporting, troubleshooting, documentation, and more. Your efforts are invaluable to the continued success of Fluent Bit.

Join FluentD Community

We invite you to join our growing community. Connect with us through the following channels:

• GitHub: http://github.com/fluent/fluent-bit
• Slack: http://slack.fluentd.org
• Twitter: @fluentbit

For more details and to download Fluent Bit v4.0.0, visit our official website.