Skip to content

At login

Local Accounts: Linux Persistence via Rogue Users

How attackers add UID 0 users, empty passwords, shells on service accounts and root-equivalent groups on Linux, and how to audit passwd, shadow and group.

Tactics
Persistence, Privilege escalation, Defense evasion
Runs
At login
Set up by
root
Locations
  • /etc/passwd
  • /etc/shadow
  • /etc/group
  • /etc/gshadow

How it works

Linux identifies users by number. /etc/passwd maps names to numbers, one line per account: name:password:UID:GID:GECOS:home:shell. The kernel never sees the name, so any line with UID 0 is root, whatever it is called. The second field is normally x, meaning the hash is in /etc/shadow; if it holds a hash, or is empty, that value is used directly.

/etc/shadow holds name:hash:lastchg:min:max:warn:inactive:expire:. lastchg is the day of the last password change, counted in days since 1 January 1970. A hash starting with ! or * can never match (locked, or no password login). An empty field means no password at all, which pam_unix accepts when its nullok option is set, as it is in Debian and Ubuntu's default common-auth.

/etc/group lists name:password:GID:members. The members field holds supplementary members only; an account's primary group is the GID in passwd. /etc/gshadow holds group passwords and group administrators.

System accounts sit below UID_MIN: /etc/login.defs sets SYS_UID_MAX 999 and UID_MIN 1000 on current Debian, Ubuntu, RHEL and Fedora (RHEL and CentOS 6 and older used 500). They normally have /usr/sbin/nologin or /bin/false as shell; Debian's postgres account, with /bin/bash, is a common exception.

The shadow tools (useradd, usermod, passwd, chpasswd, and adduser on top of them) lock the files, keep the previous version as /etc/passwd-, /etc/shadow-, /etc/group- and /etc/gshadow-, and log to /var/log/auth.log (Debian, Ubuntu) or /var/log/secure (RHEL, Fedora). Writing to the files directly with echo >> or sed logs nothing, which is why intruders do it.

Some groups are root in practice. sudo (Debian, Ubuntu) and wheel (RHEL, Fedora) members get full sudo through the default sudoers. docker membership gives root through the Docker socket, lxd through privileged containers, disk through raw access to block devices, and shadow reads every password hash.

Accounts can also come from LDAP, SSSD or systemd through /etc/nsswitch.conf; this page covers the local files.

Where it lives

PathWhatWho can write it
/etc/passwdAccounts: UID, GID, home, shellroot
/etc/shadowPassword hashes, last change, expiryroot (readable by group shadow on Debian)
/etc/group, /etc/gshadowGroups and their supplementary membersroot
/etc/passwd-, /etc/shadow-, /etc/group-, /etc/gshadow-Previous versions kept by the shadow toolsroot
/etc/login.defsUID ranges, password aging defaultsroot
/etc/nsswitch.confWhich sources provide accountsroot
/usr/lib/sysusers.d/*.conf, /etc/sysusers.d/systemd-sysusers definitions of system accountspackages, root
/usr/share/base-passwd/passwd.master, group.masterDebian's reference list of static system accountspackages

How attackers use it

A second root

echo 'sysupd:x:0:0:System Update:/root:/bin/bash' >> /etc/passwd
echo 'sysupd:$6$<salt>$<hash>:20410:0:99999:7:::' >> /etc/shadow

Or with the hash in passwd itself, so shadow is never touched:

sysupd:$6$<salt>$<hash>:0:0::/root:/bin/bash

Logging in as sysupd is logging in as root, and the logs show sysupd. Names that imitate system accounts (systemd-sync, _apt2) get the same effect with less attention.

No password

passwd -d backup
usermod -s /bin/bash backup

Where nullok is set, su backup then asks nothing. Over SSH it also needs PermitEmptyPasswords yes in sshd_config.

A service account that can log in

usermod -s /bin/bash www-data
mkdir -p /var/www/.ssh && cat key.pub >> /var/www/.ssh/authorized_keys

Service accounts are rarely reviewed and their homes are not under /home. The key half of this is covered in SSH authorized keys.

Root through a group

usermod -aG docker deploy
# later, as deploy:
docker run --rm -it -v /:/host alpine chroot /host /bin/sh

The same works by adding the account to sudo or wheel, or by a sudoers rule that names it.

Reviving a dormant account

Setting a new password on an old, forgotten account (passwd oracle, chpasswd) gives access under a name that already looks legitimate. In the files, the only trace is the lastchg day in shadow.

How to check a host

On a live host

awk -F: '$3 == 0' /etc/passwd                                   # every UID 0 account
getent passwd | awk -F: '$3 == 0'                               # same, LDAP/SSSD included
awk -F: '$2 != "x" {print $1}' /etc/passwd                      # hash or empty field in passwd
sudo awk -F: '$2 == "" {print $1}' /etc/shadow                  # no password
awk -F: '$3 > 0 && $3 < 1000 && $7 !~ /(nologin|false|sync|shutdown|halt)$/' /etc/passwd
getent group sudo wheel admin docker lxd disk shadow
sudo awk -F: '$3 ~ /^[0-9]+$/ {print $3, $1}' /etc/shadow | sort -n | tail   # latest changes
date -u -d @$((20410 * 86400)) +%F                              # day number to date
sudo pwck -r; sudo grpck -r                                     # read-only consistency checks
sudo diff /etc/passwd- /etc/passwd; sudo diff /etc/shadow- /etc/shadow
grep -E 'useradd|usermod|userdel|groupadd|gpasswd|chpasswd|password changed' /var/log/auth.log

Use /var/log/secure on RHEL and Fedora, or the journal. On Debian and Ubuntu, update-passwd --dry-run --verbose compares the static system accounts with base-passwd's master list.

Offline or from a collection

  • Read passwd, shadow, group and gshadow and their - backups from the image. A difference between a file and its backup that no log line explains was made by hand.
  • A file's mtime is the last change to any account in it; shadow's lastchg dates each password change. Account creation has no timestamp in these files: look for useradd lines in the auth logs and the birth time of the home folder.
  • For each suspect account, read its home (.ssh/authorized_keys, shell history), its crontab in /var/spool/cron and any sudoers rule naming it.
  • No package owns these files. Compare with passwd.master on Debian, the sysusers.d definitions, or a known-good image of the same build.

Linux Persistence reads passwd, shadow and group from a collection. It keeps only whether a password is set and when it changed, never the hashes.

What Linux Persistence flags

One entry per account that raises a flag, with its UID, GID, home, shell, password state and last change, and one entry per membership of a watched group. The checks need /etc/passwd in the collection, and /etc/shadow for the password ones.

  • Second UID 0 account (high): UID 0 under any name other than root.
  • No password (high): the shadow hash field is empty and the shell is a login shell (sh, bash, dash, zsh, ksh, csh, tcsh, fish).
  • System account with a login shell (medium): UID 1 to 999, except 65534 (nobody), with one of those shells.
  • Password changed date (medium within about two days of the collection, low within 30 days): shadow's lastchg compared with the newest file time in the collection. Locked accounts are skipped.
  • Member of group (medium): a listed member of docker, lxd, disk or shadow.
  • Member of group (info): a listed member of sudo, wheel, admin or root, for review.

Account entries get no package or recency flags. Limits: the 1000 threshold is fixed, so on old hosts with UID_MIN 500 real users show as system accounts. The second field of /etc/passwd is read: an empty one (no password, and shadow is then ignored) triggers No password like an empty shadow field; a hash written there instead of x counts as a password set and is not flagged on its own. Primary groups (the GID in passwd), gshadow, and accounts from LDAP or SSSD are not checked. Expected cases do fire: Debian's postgres with a shell, Ubuntu cloud images putting the default user in lxd, developers in docker.

Cleaning up

  1. Preserve the four files and their - backups with cp -a, hash them, and export the auth log lines for the accounts involved.
  2. Disable before deleting: usermod -L -e 1 -s /usr/sbin/nologin <name> locks the password, expires the account (which also stops SSH key logins through PAM) and removes the shell. End its sessions.
  3. A second UID 0 account needs care: its processes are root's, so pkill -u <name> would kill root's processes and userdel refuses because the UID is in use. Remove its lines with vipw and vipw -s (and vigr, vigr -s for groups).
  4. Remove group memberships with gpasswd -d <name> docker, and give service accounts back their nologin shell (usermod -s /usr/sbin/nologin www-data).
  5. Look for the rest of the setup: keys in the account's home, sudoers rules, its crontab and systemd user units, and the PermitEmptyPasswords or PAM changes that made an empty password usable.
  6. Rotate: passwords the intruder set are known to them. If the shadow group or file was exposed, treat every hash as cracked and change all local passwords.

References

See also

On Linux Forensics

The artifacts behind this technique, in the Linux Forensics reference: