Sep 19, 2018

How to free space from deleted but still reserved (open) files

A file in Linux is a pointer to an inode, which contains the file data (permissions, owner and where its actual content lives on the disk). Deleting the file removes the link, but not the inode itself – if another process has it open, the inode isn't released for writing until that process is done with it.

lsof | grep deleted

to list those deleted files.

mysqld     6355     mysql   10uW     REG               8,17    268435456         18 /var/lib/mysql/ib_logfile1 (deleted)

The important columns are the second one, which gives you the PID of the process that has the file open (6355), and the fourth one, which gives you the file descriptor (10). Now, we go look in /proc, where there will still be a reference to the inode, from which you can copy the file back out:

ls -l /proc/6355/fd/10
lrwx------ 1 root root 64 Mar 12 15:50 /proc/6355/fd/10 -> /var/lib/mysql/ib_logfile1 (deleted)

now, just cat /dev/null into the fd

cat /dev/null > /proc/6355/fd/10
Note that the inode is still open, but now it's 0 length

See also:

How to recover Deleted Files With lsof