Setting up log rotation for Mesos agents

 ⋅ 2 min read

Setting up log rotation for Mesos agents

Continuing with my series of Mesos posts, I wanted to show how to correctly configure log rotation for your Mesos agent sandboxes. The official documentation on Mesos logging leaves much to be desired in terms of real-world examples, so here's how I did it successfully.

Steps to set up log rotation

  1. Set the --container-logger Agent flag to org_apache_mesos_LogrotateContainerLogger.
  2. Set the --modules Agent flag to file:///path/your-custom-config.json
  3. Save your logrotate to /path/your-custom-config.json.


In my production setup, I prefer using environment variables as they're easier to override and separate from run scripts. For any Mesos flag, you can always use an environment variable of the form MESOS_SOME_FLAG in lieu of a flag of the form --some-flag.

# Default environment variables for Mesos.
# See
cat <<EOF | sudo tee /etc/default/mesos


Once logrotation is enabled, write your logrotate configuration for both stdout and stderr streams in the modules JSON file. Note that the logrotate directives need to be separated by a \n, as JSON strings do not support newlines.


The below configuration sets up to keep the last 4 logs each of size up to 25 MB, compresses rotated logs and doesn't email old logs to any address.

# Default log rotation for all agent machines.
cat <<EOF | sudo tee /etc/mesos-agent-settings.json
  "libraries": [{
    "file": "/usr/lib/",
    "modules": [{
      "name": "org_apache_mesos_LogrotateContainerLogger",
      "parameters": [{
        "key": "logrotate_stdout_options",
        "value": "rotate 4\nsize 25M\nmissingok\nnotifempty\ncompress\ndelaycompress\nnomail\n"
      }, {
        "key": "logrotate_stderr_options",
        "value": "rotate 4\nsize 25M\nmissingok\nnotifempty\ncompress\ndelaycompress\nnomail\n"