Skip to content

Each new shell

Shell History Tampering: Linux Anti-Forensics

How intruders stop or wipe shell history on Linux (.bash_history linked to /dev/null, HISTFILE unset), why it travels with persistence, and how to check.

MITRE ATT&CK
T1070.003
Tactics
Defense evasion
Runs
Each new shell
Set up by
Users and root
Locations
  • ~/.bash_history
  • ~/.zsh_history
  • /root/.bash_history
  • ~/.bashrc
  • /etc/profile.d/*.sh
  • /etc/environment

How it works

History tampering is not persistence: nothing runs again because of it. It is what intruders do in the same session where they plant persistence, so that the commands that planted it are not recorded. A history file linked to /dev/null, though, persists in its own way: every later session of that account goes unrecorded too.

bash keeps the commands of an interactive shell in memory and writes them to $HISTFILE (default ~/.bash_history) when the shell exits, or on history -a / history -w. Without shopt -s histappend the file is overwritten, otherwise appended to, and then truncated to $HISTFILESIZE lines. If HISTFILE is unset or the file cannot be written, nothing is saved. The variables that matter:

VariableEffect
HISTFILEWhere history is written; unset means nowhere
HISTSIZECommands kept in memory; 0 keeps none
HISTFILESIZELines kept in the file; 0 empties it
HISTCONTROLignorespace (or ignoreboth) drops commands typed with a leading space
HISTIGNOREPatterns of commands never saved
HISTTIMEFORMATWhen set, a #<epoch> timestamp is written before each command

Defaults differ: Debian and Ubuntu's /etc/skel/.bashrc sets HISTCONTROL=ignoreboth, HISTSIZE=1000, HISTFILESIZE=2000 and histappend, so on those systems a leading space already hides a command without any tampering. RHEL's /etc/profile sets HISTSIZE=1000. Neither sets HISTTIMEFORMAT, so most history files carry no timestamps and the file's modification time is the only date.

zsh saves nothing unless HISTFILE and SAVEHIST are set, usually in ~/.zshrc, so a missing ~/.zsh_history is not suspicious by itself. ksh writes ~/.sh_history, fish ~/.local/share/fish/fish_history. Other programs keep their own: ~/.python_history, ~/.mysql_history, ~/.psql_history, ~/.lesshst, ~/.viminfo, and ~/.wget-hsts, which records hosts that sent wget an HSTS header.

Where it lives

PathWhatWho can write it
~/.bash_history, /root/.bash_historybash history filethe user, root
~/.zsh_history (or the HISTFILE set in .zshrc)zsh historythe user, root
~/.sh_history, ~/.local/share/fish/fish_historyksh and fish historythe user, root
~/.bashrc, ~/.profile, ~/.zshrcPer-user history settingsthe user, root
/etc/profile, /etc/profile.d/*.sh, /etc/bash.bashrc, /etc/bashrcSystem-wide history settingsroot
/etc/environmentVariables for every login through pam_env, HISTFILE includedroot
/var/lib/<service>/.bash_history, /srv/<service>/.bash_historyHistory of service accounts with a shellthe account, root

How attackers use it

Linking the history file to /dev/null

The most durable form: bash writes to the link, the kernel discards the data, and the file looks present.

ln -sf /dev/null ~/.bash_history
for h in /root /home/*; do ln -sf /dev/null "$h/.bash_history"; done   # as root, every account

Switching history off for one session

unset HISTFILE                 # nothing is written at exit
export HISTFILE=/dev/null      # same, and inherited by child shells
export HISTSIZE=0              # nothing kept in memory
set +o history                 # stop recording in this shell
 cat /etc/shadow               # leading space: not saved under ignorespace
kill -9 $$                     # exit without saving

Making it permanent in startup files

# appended to ~/.bashrc, /etc/bash.bashrc or a file in /etc/profile.d/
export HISTFILE=/dev/null
# /etc/environment: every account, every login
HISTFILE=/dev/null

See shell startup files for everything else those files can carry.

Wiping or editing what is already there

history -c; history -w                 # empty the list, then overwrite the file with it
cat /dev/null > ~/.bash_history
sed -i '/203\.0\.113\.9/d' ~/.bash_history   # remove only the incriminating lines

These lines are common in intrusion scripts; TeamTNT's scripts, for example, are publicly documented clearing history with history -c.

How to check a host

On a live host

ls -la /root/.*history /home/*/.*history 2>/dev/null    # 'l' and '-> /dev/null' = link; size 0 = emptied
find /root /home /var/lib /srv -maxdepth 2 -name '.*history' \( -type l -o -size 0 \) \
  -printf '%M %u %t %p -> %l\n' 2>/dev/null
lsattr /root/.bash_history /home/*/.bash_history 2>/dev/null
grep -nE 'HISTFILE|HISTSIZE|HISTFILESIZE|HISTCONTROL|HISTIGNORE|set \+o history|history -c' \
  /etc/profile /etc/profile.d/* /etc/bash.bashrc /etc/bashrc /etc/environment \
  /root/.bashrc /root/.profile /home/*/.bashrc /home/*/.profile /home/*/.zshrc 2>/dev/null

Then compare activity with what was recorded. last -F and lastlog show logins; an account that logged in after its history file last changed, or whose file is empty, did not get recorded. /proc/<pid>/environ of a running shell shows the environment it started with, such as an inherited HISTFILE=/dev/null, but not an unset typed later.

History is not the only record. sudo logs every command line it runs to auth.log, secure or the journal; auditd keeps EXECVE records where rules are configured (ausearch -m EXECVE -i); pam_tty_audit logs keystrokes for the accounts it covers (aureport --tty); and ~/.viminfo, ~/.lesshst and ~/.wget-hsts often survive a wiped history.

Offline, from a collection

Tar archives keep symbolic links: tar -tvzf persist-host.tar.gz | grep -E '_history( -> |$)' shows each history file, links as l… lines ending in -> /dev/null. On a mounted image, find <mnt>/home <mnt>/root -name '.*history' -type l does the same; the link's own times (stat) date the change. Read the real history files for gaps and for the last commands before a gap. The collection command archives .*_history from /root and /home/*, and Linux Persistence reports the links offline.

What Linux Persistence flags

The tool does not read history contents. It reports two things.

When ~/.bash_history, ~/.zsh_history, ~/.history or ~/.sh_history of root, a /home account or a service account home is a symbolic link in the collection, it becomes an entry under that account, mechanism "anti-forensics", with the link's own time from the archive:

  • History sent to /dev/null (high): the link points to /dev/null.
  • History redirected to file (medium): the link points anywhere else.

These entries get no package or recency checks.

History switched off in commands and scripts

Turns shell history off (medium) is one of the shared command heuristics, so it fires in any file the tool parses (shell startup files, /etc/environment, crontabs, systemd units, rc.local, MOTD scripts, autostart Exec lines) on HISTFILE=/dev/null, unset HISTFILE, HISTSIZE=0, set +o history or history -c. In a dotfile identical to /etc/skel, or an unchanged packaged file, it is shown at info.

Limits: links survive only in tar collections; zip archives and dropped folders carry no link information. Regular history files are skipped, so emptied, truncated or edited files, missing files, HISTFILESIZE=0, HISTCONTROL and HISTIGNORE tricks, kill -9 $$, file attributes and other programs' history are not detected. No flag does not mean the history is complete.

Cleaning up

  1. Preserve first. Record the link before touching it: ls -l and stat (its times say when it was created), plus lsattr. Copy every real history file with cp -a and hash it.
  2. Restore recording. Remove the link (rm ~/.bash_history deletes the link, not /dev/null) and create a normal file with the right owner: install -m 600 -o alice -g alice /dev/null /home/alice/.bash_history. Remove the HISTFILE, HISTSIZE and set +o history lines from startup files and /etc/environment, and clear any chattr attribute.
  3. Find what was hidden. The time of the link or the edit marks the intrusion window. Pivot to auth logs, sudo records, the journal, auditd, and the persistence planted around that time: cron, systemd units, keys, accounts. Check whether logs were trimmed too, for example through logrotate.
  4. Rotate the credentials of every account whose history was disabled: you cannot know what was run with them. For the future, ship auditd or pam_tty_audit records off the host, where a user cannot switch them off.

References

See also

On Linux Forensics

The artifacts behind this technique, in the Linux Forensics reference: