How To Move 20000 Files?

Asked by Michael Fletcher

I have a folder with 20,000 images in it. I tried to move them using a command like "mv /f1/*.png /f2" but I got an error that I had too many arguments in the list.

How do you move that many files?

Question information

Language:
English Edit question
Status:
Solved
For:
Ubuntu Edit question
Assignee:
No assignee Edit question
Solved by:
Eamonn Sullivan
Solved:
Last query:
Last reply:
Revision history for this message
Eamonn Sullivan (eamonn-sullivan) said :
#1

you can just move the directories.

mv f1 f2

Revision history for this message
Eamonn Sullivan (eamonn-sullivan) said :
#2

And if that doesn't work for you (i.e., you want to move 20,000 files out a directory with 40,000) you can try using more complicated wildcards to move the files in chunks. Try [a-m]*.png to move files beginning with 'a' through 'm' and then [n-z]*.png for the rest of the alphabet, or similar combinations. * matches any string of characters, ? matches any single character and square brackets ([ ]) enclose characters you want to match:

[abc]??.doc would match awk.doc, bat.doc and can.doc, but not dog.doc or apple.doc.

Finally, I believe Nautilus (the file manager in Gnome) can handle this. Navigate to the directory, type CTRL-A to select all the files and then drag them to the destination.

Revision history for this message
Dean Sas (dsas) said :
#3

You can also use the "xargs" program or even write a small script

for file in `ls f1`; do mv $file f2; done

Revision history for this message
Luca Falavigna (dktrkranz) said :
#4

If you need to move a specific file type only, you can use a variant of Dean's script:
for file in `find -name "*.png"` do mv $file f2; done

Revision history for this message
Ubuntu User (anotherubuntuuser) said :
#5

This is a classic unix problem. I think you can do it with a tar copy followed by a remove. You will need to have enough space on the disk to hold two copies of the data for a short period of time.

To fix it, use tar in a pipeline to tar up standard input and untar it on the other end of the pipe.

1. change into the directory of the first pictures:

cd /f1

2. Tar up the pictures in that directory to standard output and pipe standard output to a different directory where another copy of tar will untar them and then remove the orignals ONLY DO THIS IF YOU ARE 100% SURE THAT YOU WILL NOT LOSE DATA:

 tar -cf - ./*.png | (cd /f2; tar -xvf -); rm /f1/*.png

That should do it. You SHOULD LEAVE OFF the rm bit until you have confirmed that all the pictures are copied into place (I would suggest it)

Test before trying this with a smaller set and always have good backups. I wouldn't try moving that much data until I was sure that I had good backups.

We usually use a variation on this to move large directories from one volume to another. We DON'T put the rm command at the end, because we like to verify that everything is in its new location before wiping it out, so USE WITH EXTREME CAUTION. I just added it to show how it could be done.

Another SAFER way to do this is in Nautilus itself.

Open up Nautilus. Click on Places -> Computer. Double click Filesystem and then doubleclick on the f1 folder.

Now, to select only the png files (all 20,000 of them), select Edit --> Select Pattern and type

*.png

in the Select Pattern dialog. Then click ok and all the png files will be selected. Now choose Edit --> Cut and then use nautilus to go to /f2.

Once there, choose Edit --> Paste and all 20,000 images will be moved to the new location.

If these comments answered your question, please consider closing this ticket.

Good Luck

Jim Jones

Revision history for this message
Michael Fletcher (mikefletcher) said :
#6

Thank you very much, that helps!