Over the past while I’d been receiving alarms from vCenter claiming vague vCenter server health issues. I decided to look into the issue, and by running ‘df -h’ I noticed that the root partition was completely filled. After doing some research, I discovered that the root password expired, and the audit log is filled with errors, causing the disk to fill up. This is what I did to resolve the issue (as to note, after finding this, it was true across every single vcenter server that I managed).
SSH into the vcenter server (you may have to enable the SSH service, and allow SSH server in the vcenter firewall). At the CLI run to validate:
shell.set --enabled true shell cd /var/log/audit ls -lah | grep audit
If you are running into the issue, you first must free up some disk space to resolve the immediate issue of no disk space. I simply deleted some audit files:
rm -rf audit.log*
Now at this point, you have 2 options to resolve the issue. The less secure option is to set the root password to never expire.
chage -I -1 -m 0 -M 99999 -E -1 root
The more secure option is to simply reset the root password:
chage -l root service-control --stop --all service-control --start --all