Skip to content

At authentication

PAM Backdoors: Linux Persistence via Auth Modules

How attackers backdoor Linux PAM with pam_exec, rogue modules, pam_permit and a patched pam_unix.so, and how to verify PAM stacks and module checksums.

MITRE ATT&CK
T1556.003
Tactics
Persistence, Credential access, Defense evasion
Runs
At authentication
Set up by
root
Locations
  • /etc/pam.d/*
  • /etc/pam.conf
  • /usr/share/pam-configs/*
  • /usr/lib/x86_64-linux-gnu/security/*.so
  • /usr/lib64/security/*.so

How it works

Programs that authenticate users through PAM (login, su, sudo, sshd with UsePAM yes, passwd, cron, display managers, screen lockers, polkit) call libpam with a service name. libpam reads /etc/pam.d/<service>, or /etc/pam.d/other when there is no such file; /etc/pam.conf is only used when /etc/pam.d does not exist. The file is read at each authentication, so an edit applies to the next login with no reload.

Each line is type control module [arguments]. The core of Debian's common-auth:

auth  [success=1 default=ignore]  pam_unix.so nullok
auth  requisite                   pam_deny.so
auth  required                    pam_permit.so

The types are auth (prove identity), account (is access allowed now), password (change credentials) and session (set up and tear down the session). The control decides what a result does: required, requisite, sufficient (success ends the stack if nothing required has failed), optional, include, substack, or the [value=action] syntax. A leading - on the type hides the error when a module is missing.

A module name without a slash is loaded from the system folder: /usr/lib/x86_64-linux-gnu/security/ on Debian and Ubuntu (/lib/x86_64-linux-gnu/security/ before merged /usr), /usr/lib64/security/ on RHEL and Fedora. A full path loads a shared object from anywhere. Modules run inside the calling process, as root for sshd, su, sudo and login, and auth modules see the password in clear text.

Debian and Ubuntu share stacks through @include common-auth and its siblings, generated by pam-auth-update from profiles in /usr/share/pam-configs/. RHEL 8+ and Fedora include system-auth and password-auth, which authselect generates under /etc/authselect/ and links into /etc/pam.d/.

Where it lives

PathWhatWho can write it
/etc/pam.d/<service>Per-service stacksroot, packages
/etc/pam.d/common-* (Debian, Ubuntu)Shared stacks generated by pam-auth-updateroot
/usr/share/pam-configs/ (Debian, Ubuntu)Profiles merged into common-*packages, root
/etc/authselect/ (RHEL, Fedora)Generated system-auth, password-auth, linked from /etc/pam.droot (authselect)
/usr/lib/pam.d/Vendor stacks where Linux-PAM is built with a vendor folder (openSUSE)packages
/etc/pam.confSingle-file format, ignored when /etc/pam.d existsroot
/usr/lib/x86_64-linux-gnu/security/*.so, /usr/lib64/security/*.soThe modulespackages
/etc/security/*.confModule settings (access.conf, pam_env.conf, limits.conf)root

How attackers use it

Patch pam_unix.so

The best-known PAM backdoor: a rebuilt pam_unix.so that accepts one hard-coded password for every account, often while writing each real password to a file. The Skidmap cryptomining malware installs such a module. Nothing changes in /etc/pam.d; only the module's checksum gives it away:

$ dpkg -V libpam-modules
??5??????   /lib/x86_64-linux-gnu/security/pam_unix.so

Load an extra module

auth  sufficient  /usr/local/lib/.cache/pam_auth_sys.so

The same works with a new file dropped into the module folder and named without a path. Placed first as sufficient, the module's own check (a magic password) is enough to log in as anyone.

pam_exec with expose_authtok

auth  optional  pam_exec.so quiet expose_authtok /usr/local/sbin/pam-audit
#!/bin/sh
# pam_exec writes the password to stdin and sets PAM_USER, PAM_RHOST, PAM_SERVICE
read -r pw
printf '%s %s %s %s\n' "$PAM_SERVICE" "$PAM_RHOST" "$PAM_USER" "$pw" >> /var/tmp/.pam.log

Every password typed for sudo, su or SSH lands in the file. A variant uses sufficient with a script that exits 0 for a magic password. On a session line, pam_exec runs a program as root at every login: persistence without the password grab.

pam_permit

auth  sufficient  pam_permit.so

At the top of common-auth or /etc/pam.d/sshd, any password works for every account.

Blend into generated files

A profile dropped into /usr/share/pam-configs/ and enabled with pam-auth-update puts the line into the generated common-* files, where it looks like part of the distribution's setup. On RHEL and Fedora, authselect check reports hand edits to the files it manages, so intruders favour the per-service files (sshd, su, sudo), which authselect does not manage.

Hook libpam without touching PAM

A library in /etc/ld.so.preload can hook pam_authenticate or pam_get_item in every process, leaving PAM files and modules clean.

How to check a host

On a live host

grep -rnE 'pam_exec|pam_permit|expose_authtok' /etc/pam.d/ /etc/pam.conf 2>/dev/null
grep -rnE '^\s*-?(auth|account|password|session)\s+(\[[^]]*\]|\S+)\s+/' /etc/pam.d/   # modules by full path
ls -la --time-style=full-iso /etc/pam.d/ /usr/share/pam-configs/ 2>/dev/null
dpkg -V libpam-modules libpam-modules-bin libpam-runtime    # Debian, Ubuntu
rpm -V pam; authselect check                               # RHEL, Fedora
for f in /lib/x86_64-linux-gnu/security/*.so /usr/lib/x86_64-linux-gnu/security/*.so; do
  [ -e "$f" ] || continue; p=${f#/usr}
  dpkg -S "$p" >/dev/null 2>&1 || dpkg -S "/usr$p" >/dev/null 2>&1 || echo "unowned: $f"
done
rpm -qf /usr/lib64/security/*.so | grep 'not owned'       # RHEL, Fedora
sudo grep -h '/security/' /proc/[0-9]*/maps 2>/dev/null | awk '{print $6, $7}' | sort -u

The loop tries both /lib and /usr/lib spellings because dpkg may record either on merged-/usr systems. The last command lists PAM modules mapped into running processes, including deleted files still in memory. dpkg's md5sums sit on the host and root can rewrite them: on a host you distrust, compare with a fresh copy of the package (apt-get download libpam-modules, then dpkg-deb -x).

Offline or from a collection

  • Read every file in /etc/pam.d, follow @include, include and substack, and list each module with its path. Start with the top of common-auth, system-auth, password-auth, sshd, su, sudo and login.
  • Hash every .so in the security folders and compare with the package database:
grep -h 'security/pam_unix.so' var/lib/dpkg/info/libpam-modules*.md5sums
md5sum lib/x86_64-linux-gnu/security/pam_unix.so usr/lib/x86_64-linux-gnu/security/pam_unix.so 2>/dev/null
  • On RPM systems, run rpm --root <mount> -V pam.
  • A module whose time does not match its package's install in /var/log/dpkg.log or /var/log/dnf.rpm.log, or a .so that no package lists, needs a closer look; strings often shows a hard-coded password or a log path.

Linux Persistence does both offline: the collection command archives /etc, the Debian and Ubuntu module folders and the dpkg database.

What Linux Persistence flags

Each PAM service file is one entry (pam: <file>) showing its stack and the lines that raised flags.

  • pam_exec runs command (high): a line loads pam_exec.so; its arguments are shown.
  • Module loaded from path (high): a module given by full path outside the system security folders (under /lib, /lib64, /usr/lib, /usr/lib64, or their multiarch <triplet> subfolders).
  • pam_permit accepts anyone (high): an auth line with pam_permit.so as sufficient, or with success=done, success=ok or a success=N jump.
  • The arguments also go through the command heuristics, for example Runs from a temporary folder or Runs from a hidden folder.
  • Package layer for stack files: an unchanged packaged file has its flags shown as info. An edited one gets Differs from package package (medium); a file no package installed gets Not installed by any package (medium), except Debian's generated common-* files. Changed date marks files modified within 7 days of the collection.
  • Modules: every .so in a security folder of the collection is hashed and compared with dpkg's md5sums. A changed module gets Differs from package package (high), one no package owns gets Not installed by any package (high). Unchanged modules produce no entry.

Limits: module checks need the dpkg database, so RPM hosts get no module verdict (the collection command still archives /usr/lib64/security for your own checks). Each stack file is judged alone: includes are not followed, symbolic links are not read (RHEL's system-auth and password-auth point into /etc/authselect/, which is not parsed), and neither is /usr/lib/pam.d. Weakened options (nullok, pam_wheel.so trust, a removed pam_deny.so, a reordered stack) are not flagged, modules are compared by checksum only, and hooks loaded through ld.so.preload are covered by the LD_PRELOAD checks instead.

Cleaning up

  1. Preserve first: cp -a /etc/pam.d, the profiles and any suspect .so into your evidence folder, hash them, and save /proc/<pid>/maps of sshd and other long-running processes. Keep the module: its strings give the backdoor password and the log file that scopes the credential theft.
  2. Keep a root shell open while editing PAM: a broken auth stack can lock everyone out, root included. Test with a second login before closing the first.
  3. Restore modules from the package (apt install --reinstall libpam-modules, dnf reinstall pam) and remove unowned ones. Remove rogue lines; on Debian, delete the rogue profile and rerun pam-auth-update; on RHEL, authselect check until it reports a clean configuration.
  4. Restart services that keep PAM loaded (display managers, cron, other long-running daemons), or reboot.
  5. Rotate every password typed on the host since the change: SSH, su, sudo and console logins. A password log found on disk tells you whose. If the module accepted a magic password, every account was open: check the other doors, sudoers, sshd and new local accounts.

References

See also

On Linux Forensics

The artifacts behind this technique, in the Linux Forensics reference: