At SSH login
SSH Authorized Keys: Linux Persistence via authorized_keys
How attackers plant SSH keys, forced commands and ~/.ssh/rc scripts to keep access to Linux hosts, and how to find rogue authorized_keys entries.
- MITRE ATT&CK
- T1098.004
- Tactics
- Persistence
- Runs
- At SSH login
- Set up by
- Users and root
- Locations
- ~/.ssh/authorized_keys
- ~/.ssh/authorized_keys2
- /root/.ssh/authorized_keys
- ~/.ssh/rc
- ~/.ssh/environment
- /etc/ssh/sshrc
How it works
When a client offers a public key, sshd looks up the target account and reads the files named by AuthorizedKeysFile, relative to that account's home. The OpenSSH default is .ssh/authorized_keys .ssh/authorized_keys2. Debian and Ubuntu leave the line commented out, so both files count; RHEL and Fedora set AuthorizedKeysFile .ssh/authorized_keys, so authorized_keys2 is ignored there. If the client proves it holds the private half of any listed key, the login succeeds without a password.
Each line is [options] type base64-key [comment]. The comment is free text that sshd never checks. Options restrict or extend what the key may do:
| Option | Effect |
|---|---|
command="…" | Runs this instead of the shell or the requested command; the client's command is in SSH_ORIGINAL_COMMAND |
environment="NAME=value" | Sets a variable for the session, only if PermitUserEnvironment allows it |
from="pattern-list" | Accepts the key only from matching addresses or host names |
no-pty, no-port-forwarding, restrict | Narrow what the session may do |
cert-authority | Trusts certificates signed by this key rather than the key itself |
no-user-rc | Skips ~/.ssh/rc for this key |
With StrictModes yes (the default), sshd ignores the file when it, ~/.ssh or the home folder is writable by group or others. A planted key that "does not work" often fails this check.
After authentication and before the shell or command starts, sshd runs ~/.ssh/rc with sh if it exists, otherwise /etc/ssh/sshrc. PermitUserRC no or the no-user-rc option disables the per-user script. ~/.ssh/environment is read only when PermitUserEnvironment is not no, and no is the default.
Keys survive password changes, and with UsePAM yes (the default on Debian, Ubuntu, RHEL and Fedora) locking the password with passwd -l does not stop a key login: sshd checks the ! lock itself only when PAM is off. On cloud images, cloud-init writes the metadata keys into the default user's file at first boot.
Where it lives
| Path | What | Who can write it |
|---|---|---|
~/.ssh/authorized_keys | Keys for that account | The account, root |
~/.ssh/authorized_keys2 | Legacy second file, read by default except where AuthorizedKeysFile drops it | The account, root |
/root/.ssh/authorized_keys | Root logins, allowed with keys under the default PermitRootLogin prohibit-password | root |
/var/lib/<svc>/.ssh/, /var/www/.ssh/, /srv/<svc>/.ssh/ | Keys for service accounts whose home is there | The service account, root |
Path set by AuthorizedKeysFile, e.g. /etc/ssh/authorized_keys/%u | Central key files on some hardened hosts | root |
~/.ssh/rc | Script run at every SSH login of that account | The account, root |
/etc/ssh/sshrc | System script, run when the user has no ~/.ssh/rc | root |
~/.ssh/environment | Variables set at every SSH login | The account, root |
No package ships an authorized_keys file: every key was put there by a person, cloud-init or configuration management.
How attackers use it
Append a key
Cryptomining groups such as TeamTNT add their own key to root's file, and unauthenticated Redis servers have long been abused by pointing CONFIG SET dir at /root/.ssh and saving a key as the database file, which leaves binary junk around the key line.
mkdir -p /root/.ssh && chmod 700 /root/.ssh
echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIK3x...Qz root@backup01' >> /root/.ssh/authorized_keys
chattr +i /root/.ssh/authorized_keys
The immutable attribute makes a later cleanup fail with "Operation not permitted".
Copy a legitimate comment
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGb1...kP alice@laptop
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM9q...7w alice@laptop
A reviewer reading comments sees the same owner twice.
Hide where nobody looks
Keys in authorized_keys2, or under /var/www/.ssh/ after giving www-data a shell (see local accounts), escape scripts that only read /home/*/.ssh/authorized_keys.
Forced command on someone else's key
Prepending command= to an administrator's existing key runs a payload at each of their logins and still hands them a shell:
command="/var/tmp/.x/upd >/dev/null 2>&1 & exec $SHELL -l" ssh-ed25519 AAAAC3Nza...Lq admin@jump01
The giveaway: scp, sftp and git over SSH to that account stop working, because the forced command ignores SSH_ORIGINAL_COMMAND.
Session environment
Once PermitUserEnvironment yes is set in sshd_config, a key option or ~/.ssh/environment can preload a library into the user's shell:
environment="LD_PRELOAD=/home/alice/.cache/libnss.so" ssh-ed25519 AAAAC3Nza...Lq alice@laptop
A login script
# ~/.ssh/rc: runs with sh before the shell at every SSH login; must not write to stdout
nohup /dev/shm/.k >/dev/null 2>&1 &
How to check a host
On a live host
List every key file sshd would read, for every account, with owner, mode, times, attributes and fingerprints:
getent passwd | while IFS=: read -r user _ uid _ _ home shell; do
for f in "$home/.ssh/authorized_keys" "$home/.ssh/authorized_keys2"; do
[ -f "$f" ] || continue
echo "== $user (uid $uid, $shell) $f"
stat -c 'mtime %y ctime %z %U:%G %a' "$f"
lsattr "$f" 2>/dev/null
ssh-keygen -lf "$f"
done
done
Then check what sshd uses, find key files outside homes, and match fingerprints with logins:
sudo sshd -T | grep -Ei '^(authorizedkeys|trustedusercakeys|permituserenvironment|permituserrc)'
sudo find / /home -xdev \( -name 'authorized_keys*' -o -path '*/.ssh/rc' -o -path '*/.ssh/environment' \) 2>/dev/null
ls -l /etc/ssh/sshrc 2>/dev/null
grep 'Accepted publickey' /var/log/auth.log # /var/log/secure on RHEL; or journalctl -u ssh -u sshd
Each Accepted publickey for root from 203.0.113.7 port 50122 ssh2: ED25519 SHA256:… line names the fingerprint used: match it with ssh-keygen -lf to learn which key opened which session, from where.
Offline or from a collection
- Read every
.ssh/authorized_keys*under/root,/home/*and the homes/etc/passwdlists for service accounts (/var/lib,/srv,/var/www,/opt), plus whateverAuthorizedKeysFilepoints to. - No package owns these files, so
dpkg -Vandrpm -Vcannot help. Compare with your inventory: configuration management, cloud metadata, the image's defaults. - Compare file mtime and ctime with the first
Accepted publickeyline of each fingerprint in rotated auth logs and the journal. - Binary junk around a key line (a Redis dump), or a file inside
/root/.sshowned by a service account, points to an exploit writing the file.
Linux Persistence does this offline from the collection command, which archives /root/.ssh, /home/*/.ssh and /var/lib/*/.ssh.
What Linux Persistence flags
Each key line becomes one entry named after its comment (or its fingerprint when there is none), with the key type, the SHA256 fingerprint in the format sshd logs, the options, the from= restriction (any when absent) and the forced command.
- Key opens a root login (low): the key sits in root's
authorized_keysorauthorized_keys2. - Key sets environment variables (medium): the line has an
environment=option. - Weak or obsolete key type (info): the key is
ssh-dss(DSA), or anssh-rsakey under 2048 bits (the size is read from the key itself). - Same key for accounts (low): one fingerprint opens several accounts on the host.
- Another key uses the comment “comment” (medium): two different keys on the host share a comment. Both lines are flagged, the legitimate one included, and a user who generated a new key without changing the comment triggers it too.
- Changed date (low): the key file was modified within 7 days before the newest file in the collection.
- Forced commands go through the command heuristics, so a
command=can raise Reverse or bind shell, Downloads and runs code, Decodes and runs a payload, Runs from a temporary folder, Runs from a hidden folder or Detaches from the session, and program also runs from mechanisms when the same program is started by another mechanism. ~/.ssh/rcand/etc/ssh/sshrcraise SSH rc script (medium) and~/.ssh/environmentraises SSH environment file (low); their content goes through the same heuristics, so anLD_PRELOAD=line adds Sets LD_PRELOAD. These entries are listed under the sshd mechanism.
Limits: the tool reads files, not sshd's effective settings. It does not check permissions or immutable attributes, does not follow a custom AuthorizedKeysFile (the setting is flagged on the sshd side), cannot see what an AuthorizedKeysCommand returns, and only places key files under /root, /home/*, /var/lib/*, /srv/*, /var/www and /var/spool/*.
Cleaning up
- Preserve first: copy each file with
cp -a(keeps times),sha256sumit, save thestatoutput and fingerprints, and export the auth log lines for each rogue fingerprint. - If
lsattrshowsi, record it, then clear it withchattr -i. - Remove only the rogue lines; do not delete a file that also holds legitimate keys. Remove a rogue
~/.ssh/rc,~/.ssh/environmentor/etc/ssh/sshrc. - End the sessions that used the key:
who,ss -tnp state established '( sport = :ssh )'and the auth log give the processes to stop. - Look for the same key or payload elsewhere: every account's two key files,
AuthorizedKeysFile,AuthorizedKeysCommandandTrustedUserCAKeysin sshd_config, cloud instance metadata (cloud-init can put a key back), configuration management, and other hosts. - To block an account while you investigate, expire it with
usermod -e 1 <user>. Anologinshell alone still allows port forwarding withssh -N. - Rotate: private keys stored on the host (
~/.ssh/id_*) may have been copied;known_hostsand shell history show where they lead. Replace them everywhere they are trusted.
References
See also
Related techniques
On Linux Forensics
The artifacts behind this technique, in the Linux Forensics reference: