Open unlinked inodes

Common problem when someone removes a file that a process is using. The  command simply unlinks files, it doesn't actually remove them, so if a process is still accessing it, then free space will not return.

To work out the processes that are hogging space:-

host01:/var/tmp# lsof +aL1 /usr/local COMMAND   PID   USER   FD   TYPE DEVICE   SIZE/OFF NLINK   NODE NAME ns-httpd  919 nobody    5u  VREG  85,18 1263031805     0 329450 /usr/local (/dev/md/dsk/d18) ns-httpd  919 nobody   11u  VREG  85,18  624088105     0 329264 /usr/local (/dev/md/dsk/d18) ns-httpd 2321 nobody    5u  VREG  85,18   35870416     0 447248 /usr/local (/dev/md/dsk/d18)

Now you have that information, the important columns are the PID and the FD handle number. Because the file no longer has a link within the normal filesystem, we have to access it through the proc interface that the process is using. This can be found in: /proc/ /fd/

host01:/proc/919/fd# ls 0   1    10   11   12   2    260  3    4    5    6    7    8    9

host01:/proc/919/fd# cat /dev/null > 5

The last cat command should erase the contents of the deleted log, thereby freeing up space. BEWARE, doing this will still cause the logs to be written to nowhere, and will be unavailable. If you *really* had to, you could cat the file descriptor to a real file...

A nice way to find the unlinked files if you don't have lsof installed:-

find /proc/*/fd -links 0 -type f