Skip to content

A faster way to delete millions of files in a directory⚓︎

Overview⚓︎

The post, from an unknown author (http://linuxnote.net/jianingy) is itself an overview of a Quora Post where people discuss various methods of deleting hundreds of thousands, or millions, of files on Linux systems as quickly as possible.

The author proceeds to do perform two sets of tests of the five methods discussed in the Quora post to see which may be faster.

Benchmark 1⚓︎

Method # Of Files Deletion Time
rsync -a –delete empty/ s1/ 1000000 6m50.638s
find s2/ -type f -delete 1000000 87m38.826s
find s3/ -type f | xargs -L 100 rm 1000000 83m36.851s
find s4/ -type f | xargs -L 100 -P 100 rm 1000000 78m4.658s
rm -rf s5 1000000 80m33.434s

Benchmark 2⚓︎

Command Elapsed System Time %CPU cs1 (Vol/Invol)
rsync -a –delete empty/ a 10.60 1.31 95 106/22
find b/ -type f -delete 28.51 14.46 52 14849/11
find c/ -type f | xargs -L 100 rm2 41.69 20.60 54 37048/15074
find d/ -type f | xargs -L 100 -P 100 rm2 34.32 27.82 89 929897/21720
rm -rf f 31.29 14.80 47 15134/11

(The # of files is 1000000. Each of them has 0 size.)


Source⚓︎