Balabit - Finding and removing tainted files

category: solutionz · date: 2018-12-31 · updated: 2026-10-03 · author: LALA

Balabit SCB Solution · Config document · referenced from Operations, upgrades and troubleshooting

noteThe operation how-to that repeats the first half of this procedure says "All outputs from commands are only examples": the list of files is the one found on that occasion, not a list to delete blindly. Run the check and remove only what it reports. The rm commands run in the boot firmware against the persistent area of the core firmware; there is no undo.

After an upgrade the SCB can report its firmware as "TAINTED": some files that belong to the firmware differ from what the firmware shipped. This command set found those files and removed them, from the boot shell. The second half is what the vendor's support sent on another occasion, for two leftover files, with the two check commands for the future.

Item	Value
Where	SSH console of the SCB as `root`, menu "Shells", "1. Boot shell"
Which appliance	Not named in the notes
When	After an upgrade; the notes do not date it
Symptom	Firmware status "TAINTED" (Firmware is tainted) in System Monitor and under Basic Settings > System > Version details
Check commands	`xcbclient self xcb_check_core_files`, `xcbclient self xcb_check_boot_files`
Source	Troubleshooting notes, sections "TAINTED FIRMWARE - remove unneeded files" and the support's answer "TAINTED FIRMWARE"; operation how-to, section "Remove tainted files"

The commands

Check that the shell is in the boot firmware; both the notes and the support say the procedure runs there. The paths below are the boot firmware's view of the core firmware's persistent area.

bash

$ cat /etc/firmware-type

output 1 line

boot

Ask the appliance which files of the core firmware are changed.

bash

$ xcbclient self xcb_check_core_files

output 6 lines

/mnt/drbd/private/root/run/lighttpd.pid
/mnt/drbd/private/root/etc/ssh/sshd_config
/mnt/drbd/private/root/etc/ssh/ssh_config
/mnt/drbd/private/root/etc/lighttpd/lighttpd.conf
/mnt/drbd/private/root/etc/init/ssh.conf
/mnt/drbd/private/root/usr/sbin/sshd

Remove exactly those files, one rm each.

bash

$ rm /mnt/drbd/private/root/run/lighttpd.pid
$ rm /mnt/drbd/private/root/etc/ssh/sshd_config
$ rm /mnt/drbd/private/root/etc/ssh/ssh_config
$ rm /mnt/drbd/private/root/etc/lighttpd/lighttpd.conf
$ rm /mnt/drbd/private/root/etc/init/ssh.conf
$ rm /mnt/drbd/private/root/usr/sbin/sshd

On another occasion a debug bundle had gone to the vendor's support. Its answer named two different files, "not needed for normal operation of SCB; possibly they were created automatically during the upgrade", and said it was safe to remove them "from boor firmware" (boot firmware). Same shell.

bash

$ rm /mnt/drbd/private/root/dev/.blkid.tab
$ rm /mnt/drbd/private/root/dev/.blkid.tab.old

The support then advised to log out, log back in and look at System Monitor again, if necessary after emptying the browser's cache, or to open Basic Settings > System > Version details, where no warning means the firmware is fine. For the next time it gave the two check commands, one per firmware, to run from the boot shell; the comments are the support's.

bash

$ xcbclient self xcb_check_boot_files # check tainted files in boot firmware
$ xcbclient self xcb_check_core_files # check tainted files in core firmware

Line	What it means
`/mnt/drbd/private/root/`	As I understand it, the part of the DRBD-replicated disk where the core firmware keeps the files that differ from the read-only firmware image. A file listed here would then override the firmware's own copy, and removing it makes the firmware's version visible again. The notes record no trouble with SSH afterwards, but they do not record a test either
`run/lighttpd.pid`	A process ID file of the web server that, as I understand it, should not have survived a restart
`etc/lighttpd/lighttpd.conf`	A web server configuration that differed from the firmware's. After the upgrade from 4 to 5 the web server looked for PHP 5; see Web server fix after the upgrade from 4 to 5
`etc/ssh/…`, `etc/init/ssh.conf`, `usr/sbin/sshd`	SSH server files of the core firmware; the notes do not say how they came to differ
`dev/.blkid.tab`, `dev/.blkid.tab.old`	The cache of the `blkid` tool, which the support thought was created during the upgrade and not removed
`xcb_check_boot_files`	The same check for the boot firmware, which the notes never needed

Because the files sit on the DRBD device, I assume the removal reached the slave node through the normal replication and had to be done on the master only; the notes do not say on which node it was done.

Checked against One Identity Safeguard for Privileged Sessions 9.0

As built	Today
"TAINTED" in System Monitor, alert `xcbFirmwareTainted`	The firmware status in the web interface shows the core and boot firmware as "Corrupted" (the integrity check failed) or "Tainted" ("you have modified a file of the firmware locally"); the alert is now `xcbFirmwareError`
`xcbclient self xcb_check_core_files`, `xcb_check_boot_files`	Not in the 9.0 guide; the SCB 5 guide did not document them either, only `xcbclient self xcb_do_reboot`

The check commands came from the vendor's support and are in neither the SCB 5 nor the SPS 9.0 guide; before using them on a current release I would ask the support again.