Skip to main content

Log Rotation and Cleanup

find is the standard tool for enforcing file retention policies — compressing old logs, deleting ancient ones, and cleaning up empty directories left behind.

The Three-Stage Retention Pattern​

Most retention policies have three stages:

Stage 1: Compress  →  logs older than 7 days  →  gzip them
Stage 2: Archive   →  logs older than 30 days →  move to cold storage
Stage 3: Delete    →  logs older than 90 days →  remove permanently

Stage 1: Compress Logs Older Than 7 Days​

# Find uncompressed logs older than 7 days and gzip them
find /var/log/nginx -type f -name "*.log" ! -name "*.gz" -mtime +7 \
  -exec gzip {} +

The + at the end batches all matched files into a single gzip call — far more efficient than one process per file.

Stage 2: Delete Compressed Logs Older Than 30 Days​

find /var/log/nginx -type f -name "*.gz" -mtime +30 -delete

Stage 3: Remove Empty Directories Left Behind​

After rotating, parent date-named directories may be empty:

find /var/log/nginx -mindepth 1 -type d -empty -delete

-mindepth 1 ensures we never delete the /var/log/nginx root itself.

Full Maintenance Script (Cron-Ready)​

#!/bin/bash
# /usr/local/bin/log-retention.sh
# Runs via: 0 2 * * * /usr/local/bin/log-retention.sh

LOG_DIR="/var/opt/myapp/logs"

# Safety check
if [[ ! -d "$LOG_DIR" ]]; then
  echo "ERROR: log dir not found: $LOG_DIR" >&2
  exit 1
fi

# 1. Dry run first (comment out in production after testing)
echo "=== Would compress (>7d) ==="
find "$LOG_DIR" -type f -name "*.log" ! -name "*.gz" -mtime +7 -print

echo "=== Would delete (>30d) ==="
find "$LOG_DIR" -type f -name "*.gz" -mtime +30 -print

# 2. Actual operations (uncomment for production)
# find "$LOG_DIR" -type f -name "*.log" ! -name "*.gz" -mtime +7 -exec gzip {} +
# find "$LOG_DIR" -type f -name "*.gz" -mtime +30 -delete
# find "$LOG_DIR" -mindepth 1 -type d -empty -delete

Warning: Do Not Delete Open Files​

If app.log is open by a running process, deleting it with find -delete will remove the directory entry but the process will keep writing to the open file descriptor. The disk space is not freed until the process is restarted.

# Check which processes have a log file open
lsof /var/log/myapp/app.log

# Safe pattern: only target rotated files (with date in name)
find /var/log/myapp -type f -name "app.log.[0-9]*" -mtime +7 -delete