How can we use the deployment name in fluentBit to generate a unique output folder for the logs of each module dynamically?

edit

One way to use the deployment name in fluentBit to generate a unique output folder for the logs of each module dynamically is by using the Kubernetes metadata filter plugin in combination with the fluentBit flexible output plugin.

First, enable the Kubernetes metadata filter plugin in your fluentBit configuration file:

[FILTER]
    Name                kubernetes
    Match               *

This plugin will add Kubernetes metadata such as the deployment name, namespace, and pod name to each log record.

Next, use the metadata fields in the flexible output plugin's path option to generate a dynamic output folder for each module based on the deployment name:

[OUTPUT]
    Name                flex
    Match               *
    Format              json
    Path                /var/log/fluent-bit/{{.kubernetes.namespace_name}}/{{.kubernetes.deployment_name}}.log

In this example, the output folder will be /var/log/fluent-bit/<namespace>/<deployment>.log. This will create a separate log file for each deployment in the namespace.

You can adjust the output path to fit your specific logging needs. This approach can help you organize your logs by deployment and module, making it easier to manage and analyze your application logs.