Slow file COPY() to destination with many files existing

I’ve ran into odd problem. I have a folder with 20000 images (jpgs). When I use COPY() in CW11 and I try to copy a file from some folder to this folder with 20k images, it takes 10 secs+ to copy a single file (2mb file). If folder has 200 images instead of 20000, it takes a blink of a second to copy.

Why would COPY() have an issue like that? As if it would be reading whole directory prior to finish copying or something? Is there a workaround this?

Thank you all.

Bostjan

That just sounds like normal copy. even in other OS and windows explorer you’ll see similar behaviour. The copy needs to find a blank spot in the directory and that involves reading and searching all entries, which after a while could end up scattered all over the disk. that the directory, not the file. Then there’s all the allocation etc. That’s why a copy of a few big files is way quicker than lots of little ones.

Well, if I manually select 5 files, copy them and paste them in windows explorer into that folder, it’s a matter of a second.I tried using CopyFileA inside my app, same result as COPY(), perhaps CW uses that anyway as internal COPY() command.

Bostjan,

is the app code-signed? Perhaps real-time monitoring slows down the process? Especially on Windows 11 it can make a difference when all files (including DLLs) are code-signed.

Friedrich

Yes, it is code signed. You mean it’s actually a disadvantage to have it codesigned in this case?

No, it’s definitely an advantage to have it code-signed (including all called DLLs).

Friedrich

To copy many/large files as fast as possible run RoboCopy. It has many many many option switches and features. One of which /MT uses multiple threads (the default is 8 so no need to set it). Also it does not copy files that have not changed (optional and default). It also can write a Log file.

2 Likes

I will give it a go. Though in my case I only copy up to 10 small files into a folder with 20000 files.

There some other topics here discussing COPY. I pasted one below. Using API SHFileOperation would provide a Window with progress and allows you to pass a list of multiple file names.

I had someone tell me START() a new Thread for each COPY will run much faster. I never tried it, but that is what RoboCopy does that makes it faster. I would not Start more than 8 or 12 so now you need to manage threads.

Yeah Robocopy is very useful and has lots of options including only copying files that have been changed, or are missing from the destination folder etc. I use it to update my “current” folder by copying any files I have changed to the “current” backup folder.