File copy errors with Rclone in Windows

I'm using Windows 10 and rclone, rclone mount with Google drive. The remote and mount seem to work fine, I can browse and open files. But when I copy files to the mounted drive, the speed is good at first, then slows down to barely anything, then I get file copy errors e.g access denied.

I'm using my own client id. I see no 'ERROR' lines in the log file.

Here's my config

[gdrive]
type = drive
client_id = xxx
client_secret = xxx
scope = drive
token = {"access_token":xxx,"token_type":"Bearer","refresh_token":xxx,"expiry":"2020-01-05T11:51:03.4490186-08:00"}
root_folder_id = xxx

[gdrive_enc]
type = crypt
remote = gdrive:/enc
filename_encryption = standard
directory_name_encryption = true
password = xxx
password2 = xxx

and this is the mount command

mount gdrive_enc: G: --allow-other --dir-cache-time 72h --vfs-read-chunk-size 32M --vfs-read-chunk-size-limit 2G --drive-chunk-size 64M --vfs-cache-mode writes --tpslimit 10 --tpslimit-burst 10 --buffer-size 64M --log-level INFO --log-file "c:\rclone\logs\rclone.log" --cache-dir "d:\cache" --config "c:\rclone\config\rclone.conf"

What do you see in the log? Do you see ERROR messages?

No errors in log. I see things like this -

2020/01/04 16:48:01 INFO : Cleaned the cache: objects 6 (was 6), total size 4.367G (was 4.367G)
2020/01/04 16:48:08 INFO : Media/Movies/Journey To Le Mans (2014)/Journey To Le Mans (2014).nfo: Copied (new)
2020/01/04 16:48:11 INFO : Media/Movies/Journey To Le Mans (2014)/Journey To Le Mans (2014).nfo: Copied (replaced existing)
2020/01/04 16:49:01 INFO : Cleaned the cache: objects 9 (was 9), total size 13.064G (was 4.367G)

which I assume is normal? This is on initial copy.

I suspect these are the culprits... Try removing them and see what happens.

Didn't help. Still getting errors. I'd put those lines in an effort to rate limit the calls, should I use a different value.
I got a reply on reddit which says rclone mount is unstable and shouldn't be used.

Since my current solution clearly isn't working, what do you recommend for copying data to the remote?

hello,
can you share the EXACT error, perhaps a screenshot.

are you copying a lot of small files or one large one?

if you want to copy files, perhaps use rclone copy, not rclone mount

The error I get is in Windows Explorer and its usually 'error 5 access denied', its a dialog box that has to be acted on. I chose retry and it seems to work.

I have a lot of data to copy and I'm doing it in batches. rclone won't even create directories so its a major pain to use rclone copy, esp on Windows since you can't script it. I like the idea of mount - and it has a built in cache, retry mechanisms etc. But if mount is only recommended for reads then shouldn't this be documented?

What is the recommended way to use rclone copy to copy a large amount of data and making it work with retires/reboots etc.
e.g. I did 'rclone copy --dry-run xxxx' for a folder that exists and it didn't tell me that nothing would be copied, which I'd assumed it would.

i do not use rclone mount for copying files, i always use rclone copy or rclone sync.

but when i do use mount and copy files, i use a very simple command such as
rclone.exe mount wasabieast2:en07 b:\mount\test\
and i get reliable, fast uploads

you can see the upload speed increases over time, does not decrease.

image

your problem might be the way you run the cmd.
i suggest that you read this.
https://rclone.org/commands/rclone_mount/#installing-on-windows
there are several ways to work around that.

  1. you can use nssm but that is very complex.
  2. run the rclone mount as system user using task scheduler, quick and easy solution
  3. this is what is use
    https://support.microsoft.com/en-us/help/3035277/mapped-drives-are-not-available-from-an-elevated-prompt-when-uac-is-co

if you use rclone copy and the machine reboots or something happens, just re-run the command.
each time rclone copy is run, it compares the local file system to the cloud and only uploads files that are not already uploaded.

I've started using rclone copy from command prompt. Still don't like it as much as rclone mount which gives a drive letter that you can then use with many utilities, but I never got the mount to work reliably with writes. I don't know if this is a Windows specific issue.

Also I had to ignore mod times in the rclone copy flags as otherwise files always get copied.

sure you can script rclone, very easy to do.
what makes you think you cannot script rclone on windows?

if you want to copy files from local to cloud using a mount, have you tried my suggestion, to remove ALL flags. as i demonstrated, no flags are needed for simple file copying

as another test,
using
rclone.exe mount wasabieast2:zztest02 b:\mount\test\

i used secondcopy to copy 1.3GB to a folder named vdib on wasabi.
i set secondcopy to do a byte-for-byte verification of all files.

did the same with doublecommander, to a folder named vdib2, with verification.

then i used dummy file creater to create a 4GB file.

then i create .md5 checksum files on the local and compared that to the mount.

then i ran wiztree

in all cases, no problems at all

keep in mind that if you if you using rclone to backup files to cloud and use a mount.
if your computer is infected with ransomware, then that ransomware will see the mount and encrypt all those files.

This is probably because you are running the mount as administrator and using at as a different user.

Windows is quite sensitive about that. How are your starting your mount?

yes, i agree about the access denied.
in the post above i offered two options.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.