A faster way to delete millions of files in a directory⚓︎
Overview⚓︎
The post, from an unknown author (http://linuxnote.net/jianingy) is itself an overview of a Quora Post where people discuss various methods of deleting hundreds of thousands, or millions, of files on Linux systems as quickly as possible.
The author proceeds to do perform two sets of tests of the five methods discussed in the Quora post to see which may be faster.
Benchmark 1⚓︎
| Method | # Of Files | Deletion Time |
|---|---|---|
| rsync -a –delete empty/ s1/ | 1000000 | 6m50.638s |
| find s2/ -type f -delete | 1000000 | 87m38.826s |
| find s3/ -type f | xargs -L 100 rm | 1000000 | 83m36.851s |
| find s4/ -type f | xargs -L 100 -P 100 rm | 1000000 | 78m4.658s |
| rm -rf s5 | 1000000 | 80m33.434s |
Benchmark 2⚓︎
| Command | Elapsed | System Time | %CPU | cs1 (Vol/Invol) |
|---|---|---|---|---|
| rsync -a –delete empty/ a | 10.60 | 1.31 | 95 | 106/22 |
| find b/ -type f -delete | 28.51 | 14.46 | 52 | 14849/11 |
| find c/ -type f | xargs -L 100 rm2 | 41.69 | 20.60 | 54 | 37048/15074 |
| find d/ -type f | xargs -L 100 -P 100 rm2 | 34.32 | 27.82 | 89 | 929897/21720 |
| rm -rf f | 31.29 | 14.80 | 47 | 15134/11 |
(The # of files is 1000000. Each of them has 0 size.)
