东莞市住房建设局网站首页,怎么看网站被惩罚,有没有便宜的注册代理,汉口网站推广公司昨天#xff0c;我看到一个非常有趣的删除一个目录下的海量文件的方法。这个方法来自http://www.quora.com/How-can-someone-rapidly-delete-400-000-files里的Zhenyu Lee。 他没有使用find 或 xargs#xff0c;他很有创意的利用了rsync的强大功能#xff0c;使用rsync –de…昨天我看到一个非常有趣的删除一个目录下的海量文件的方法。这个方法来自http://www.quora.com/How-can-someone-rapidly-delete-400-000-files里的Zhenyu Lee。 他没有使用find 或 xargs他很有创意的利用了rsync的强大功能使用rsync –delete将目标文件夹以一个空文件夹来替换。之后我做了一个实验来比较各种方法。让我吃惊的是Lee的方法要比其它的快的多。下面就是我的测评。 环境 CPU: Intel(R) Core(TM)2 Duo CPU E8400 3.00GHzMEM: 4GHD: ST3250318AS: 250G/7200RPMMethod# Of FilesDeletion Timersync -a –delete empty/ s1/10000006m50.638sfind s2/ -type f -delete100000087m38.826sfind s3/ -type f | xargs -L 100 rm100000083m36.851sfind s4/ -type f | xargs -L 100 -P 100 rm100000078m4.658srm -rf s5100000080m33.434s 使用 –delete 和 –exclude你可以选择性删除符合条件的文件。还有一点当你需要保留这个目录做其它用处时这种方法是再适合不过了。 重新测评 几天前Keith-Winstein在回复Quora上的这个帖子时说我之前的测评无法复制因为操作的时间持续的太久。我澄清一下这些数据过大可能是因为我的计算机在过去的几年里做的事太多测评中可能存在一些文件系统错误。但我不确定是这些原因。现在好了我弄了一天比较新的计算机把测评再做一次。这次我使用/usr/bin/time它能提供更详细的信息。下面就是新的结果。 (每次都是1000000个文件。每个文件的体积都是0。) CommandElapsedSystem Time%CPUcs (Vol/Invol)rsync -a –delete empty/ a10.601.3195106/22find b/ -type f -delete28.5114.465214849/11find c/ -type f | xargs -L 100 rm41.6920.605437048/15074find d/ -type f | xargs -L 100 -P 100 rm34.3227.8289929897/21720rm -rf f31.2914.804715134/11 原始输出 # method 1
~/test $ /usr/bin/time -v rsync -a --delete empty/ a/Command being timed: rsync -a --delete empty/ a/User time (seconds): 1.31System time (seconds): 10.60Percent of CPU this job got: 95%Elapsed (wall clock) time (h:mm:ss or m:ss): 0:12.42Average shared text size (kbytes): 0Average unshared data size (kbytes): 0Average stack size (kbytes): 0Average total size (kbytes): 0Maximum resident set size (kbytes): 0Average resident set size (kbytes): 0Major (requiring I/O) page faults: 0Minor (reclaiming a frame) page faults: 24378Voluntary context switches: 106Involuntary context switches: 22Swaps: 0File system inputs: 0File system outputs: 0Socket messages sent: 0Socket messages received: 0Signals delivered: 0Page size (bytes): 4096Exit status: 0# method 2Command being timed: find b/ -type f -deleteUser time (seconds): 0.41System time (seconds): 14.46Percent of CPU this job got: 52%Elapsed (wall clock) time (h:mm:ss or m:ss): 0:28.51Average shared text size (kbytes): 0Average unshared data size (kbytes): 0Average stack size (kbytes): 0Average total size (kbytes): 0Maximum resident set size (kbytes): 0Average resident set size (kbytes): 0Major (requiring I/O) page faults: 0Minor (reclaiming a frame) page faults: 11749Voluntary context switches: 14849Involuntary context switches: 11Swaps: 0File system inputs: 0File system outputs: 0Socket messages sent: 0Socket messages received: 0Signals delivered: 0Page size (bytes): 4096Exit status: 0
# method 3
find c/ -type f | xargs -L 100 rm
~/test $ /usr/bin/time -v ./delete.shCommand being timed: ./delete.shUser time (seconds): 2.06System time (seconds): 20.60Percent of CPU this job got: 54%Elapsed (wall clock) time (h:mm:ss or m:ss): 0:41.69Average shared text size (kbytes): 0Average unshared data size (kbytes): 0Average stack size (kbytes): 0Average total size (kbytes): 0Maximum resident set size (kbytes): 0Average resident set size (kbytes): 0Major (requiring I/O) page faults: 0Minor (reclaiming a frame) page faults: 1764225Voluntary context switches: 37048Involuntary context switches: 15074Swaps: 0File system inputs: 0File system outputs: 0Socket messages sent: 0Socket messages received: 0Signals delivered: 0Page size (bytes): 4096Exit status: 0# method 4
find d/ -type f | xargs -L 100 -P 100 rm
~/test $ /usr/bin/time -v ./delete.shCommand being timed: ./delete.shUser time (seconds): 2.86System time (seconds): 27.82Percent of CPU this job got: 89%Elapsed (wall clock) time (h:mm:ss or m:ss): 0:34.32Average shared text size (kbytes): 0Average unshared data size (kbytes): 0Average stack size (kbytes): 0Average total size (kbytes): 0Maximum resident set size (kbytes): 0Average resident set size (kbytes): 0Major (requiring I/O) page faults: 0Minor (reclaiming a frame) page faults: 1764278Voluntary context switches: 929897Involuntary context switches: 21720Swaps: 0File system inputs: 0File system outputs: 0Socket messages sent: 0Socket messages received: 0Signals delivered: 0Page size (bytes): 4096Exit status: 0# method 5
~/test $ /usr/bin/time -v rm -rf fCommand being timed: rm -rf fUser time (seconds): 0.20System time (seconds): 14.80Percent of CPU this job got: 47%Elapsed (wall clock) time (h:mm:ss or m:ss): 0:31.29Average shared text size (kbytes): 0Average unshared data size (kbytes): 0Average stack size (kbytes): 0Average total size (kbytes): 0Maximum resident set size (kbytes): 0Average resident set size (kbytes): 0Major (requiring I/O) page faults: 0Minor (reclaiming a frame) page faults: 176Voluntary context switches: 15134Involuntary context switches: 11Swaps: 0File system inputs: 0File system outputs: 0Socket messages sent: 0Socket messages received: 0Signals delivered: 0Page size (bytes): 4096Exit status: 0 我真的十分好奇为什么Lee的方法要比其它的快竟然比rm -rf也要快。如果有人知道请写在下面非常感谢。 [英文原文 A faster way to delete millions of files in a directory ]转载于:https://www.cnblogs.com/hubing/p/3194378.html
