Revision history [back]

To direct all log messages from the celery tasks to a single file, you can use the following steps:

1. Initialize a logging instance in your celery application:

import logging

app = Celery('my_task')
logger = logging.getLogger(__name__)

1. Set the log level and the log file path in the configuration:

app.conf.update(
    task_default_queue='default',
    task_default_exchange='task_exchange',
    task_default_routing_key='task',
    worker_redirect_stdouts_level='INFO',
    task_track_started=True,
    task_publish_retry=True,
    task_publish_retry_policy={
        'max_retries': 3,
        'interval_start': 0,
        'interval_step': 0.2,
        'interval_max': 0.2,
        's': 0,
    },
    worker_pool_restarts=True,
    worker_max_restart_freq=10,
    worker_hijack_root_logger=True,
    CELERYD_HIJACK_ROOT_LOGGER=False,
    CELERYD_REDIRECT_STDOUTS=False,
)


@app.task(bind=True)
def my_task(self, *args, **kwargs):
    logger.addHandler(logging.FileHandler('/path/to/your/logfile.log'))
    logger.setLevel(logging.INFO)

    try:
        # your task code here
        pass

    except Exception as exc:
        logger.error("Task failed: {}".format(exc), exc_info=True)
        raise self.retry(exc=exc)

1. In the code for each task, add a logging handler that writes to the log file you specified in the configuration.

logger.addHandler(logging.FileHandler('/path/to/your/logfile.log'))
logger.setLevel(logging.INFO)

With these steps in place, all log messages from your celery tasks will be directed to the specified log file.