The handle is invaild & source is being updated error

Suspected Bug
1

hi guys,
every night i run a bat script that copies a bunch of folders to my onedrive folder (so it can upload to the cloud) then once a week i run a rclone that backs the onedrive folder (from my machine) to google drive.

This has been running without issues for months until now.

can someone please tell me why i’m seeing handle is invaild and source is being updated errors?

2018/04/21 07:00:27 ERROR : Backup Log: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\Backup Log: The handle is invalid.
2018/04/21 07:00:27 ERROR : Calibre Library: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:27 ERROR : HTPC Manager: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:27 ERROR : Movewatched: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:27 ERROR : Mylar: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:27 ERROR : NZBHydra: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\NZBHydra: The handle is invalid.
2018/04/21 07:00:27 ERROR : Nzbget: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\Nzbget: The handle is invalid.
2018/04/21 07:00:27 ERROR : Plex media list: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:28 ERROR : Plex request: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:28 ERROR : Plex-Cleaner: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:28 ERROR : Plexemail: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:28 ERROR : Plexpy: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:28 ERROR : Radarr: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\Radarr: The handle is invalid.
2018/04/21 07:00:28 ERROR : Router: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\Router: The handle is invalid.
2018/04/21 07:00:28 ERROR : Sonarr: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\Sonarr: The handle is invalid.
2018/04/21 07:00:28 ERROR : Scripts: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:29 ERROR : Startup: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:29 ERROR : Task Scheduler: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:29 ERROR : autohidemouse: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:29 ERROR : WindowsPowerShell (put in documents folder): Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\WindowsPowerShell (put in documents folder): The handle is invalid.
2018/04/21 07:00:29 ERROR : eventghost: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:29 ERROR : kindlegen: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:29 ERROR : letsencrypt: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:30 ERROR : nginx-1.8.0: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:30 ERROR : mymedia: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:30 ERROR : xampp: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:30 ERROR : Attempt 1/3 failed with 26 errors and: can't copy - source file is being updated
2018/04/21 07:00:30 ERROR : Movewatched: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:31 ERROR : HTPC Manager: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:31 ERROR : Calibre Library: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:31 ERROR : Backup Log: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\Backup Log: The handle is invalid.
2018/04/21 07:00:32 ERROR : Mylar: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:34 ERROR : NZBHydra: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\NZBHydra: The handle is invalid.
2018/04/21 07:00:34 ERROR : Nzbget: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\Nzbget: The handle is invalid.
2018/04/21 07:00:34 ERROR : Plex request: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:34 ERROR : Plex media list: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:35 ERROR : Plex-Cleaner: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:35 ERROR : Radarr: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\Radarr: The handle is invalid.
2018/04/21 07:00:35 ERROR : Plexpy: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:35 ERROR : Plexemail: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:35 ERROR : Router: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\Router: The handle is invalid.
2018/04/21 07:00:35 ERROR : Scripts: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:35 ERROR : Sonarr: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\Sonarr: The handle is invalid.
2018/04/21 07:00:35 ERROR : Startup: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:36 ERROR : Task Scheduler: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:36 ERROR : WindowsPowerShell (put in documents folder): Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\WindowsPowerShell (put in documents folder): The handle is invalid.
2018/04/21 07:00:36 ERROR : autohidemouse: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:36 ERROR : eventghost: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:36 ERROR : kindlegen: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:38 ERROR : nginx-1.8.0: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:38 ERROR : letsencrypt: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:38 ERROR : xampp: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:38 ERROR : mymedia: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:38 ERROR : Attempt 2/3 failed with 26 errors and: can't copy - source file is being updated
2018/04/21 07:00:39 ERROR : HTPC Manager: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:39 ERROR : Backup Log: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\Backup Log: The handle is invalid.
2018/04/21 07:00:39 ERROR : Calibre Library: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:39 ERROR : Movewatched: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:39 ERROR : Mylar: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:39 ERROR : NZBHydra: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\NZBHydra: The handle is invalid.
2018/04/21 07:00:39 ERROR : Nzbget: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\Nzbget: The handle is invalid.
2018/04/21 07:00:39 ERROR : Plex media list: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:41 ERROR : Plex request: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:44 ERROR : Plex-Cleaner: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:44 ERROR : Plexemail: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:45 ERROR : Radarr: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\Radarr: The handle is invalid.
2018/04/21 07:00:45 ERROR : Plexpy: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:45 ERROR : Scripts: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:45 ERROR : Router: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\Router: The handle is invalid.
2018/04/21 07:00:45 ERROR : Startup: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:47 ERROR : Sonarr: Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\Sonarr: The handle is invalid.
2018/04/21 07:00:47 ERROR : Task Scheduler: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:47 ERROR : autohidemouse: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:47 ERROR : WindowsPowerShell (put in documents folder): Failed to copy: read \\?\C:\Users\User\OneDrive\HTPC Backup\WindowsPowerShell (put in documents folder): The handle is invalid.
2018/04/21 07:00:47 ERROR : eventghost: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:47 ERROR : kindlegen: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:47 ERROR : letsencrypt: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:47 ERROR : nginx-1.8.0: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:48 ERROR : xampp: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:48 ERROR : mymedia: Failed to copy: can't copy - source file is being updated
2018/04/21 07:00:48 ERROR : Attempt 3/3 failed with 26 errors and: can't copy - source file is being updated
2018/04/21 07:00:48 Failed to copy: can't copy - source file is being updated
2

Hmmm, can you try the latest beta we were working on this recently…

3

Hi I installed the beta and I’m still seeing the same errors

4

Can you upload a complete log made with the beta and the flags -vv --log-file rclone.log and stick the rclone.log somewhere I can see it (share it on drive, or pastebin or something) please?

5

hi thanks for looking into this

i have another rclone script which backs up my photos and that runs fine, so did this one until recently

https://pastebin.com/dA6U05tr

6

Thanks for the log - much clearer what is going on now!

What seems to be happening is for files we get this

2018/04/21 23:18:10 DEBUG : Reminders.rtf: Clearing symlink bit to allow a file with reparse points to be copied
...
2018/04/21 23:18:12 DEBUG : Reminders.rtf: Clearing symlink bit to allow a file with reparse points to be copied
2018/04/21 23:18:12 DEBUG : Reminders.rtf: Size and modification time the same (differ by 0s, within tolerance 1ms)
2018/04/21 23:18:12 DEBUG : Reminders.rtf: Unchanged skipping

But for directories we get

2018/04/21 23:18:10 DEBUG : Calibre Library: Clearing symlink bit to allow a file with reparse points to be copied
...
2018/04/21 23:18:10 ERROR : Calibre Library: Failed to copy: can't copy - source file is being updated (size changed from 0 to 4096)

So why is is treating the directories as files?

Is the path "C:\Users\User\OneDrive\HTPC Backup" special in any way? Is it managed by something else?

Is there something you did to that directory to cause it to be full of files with “reparse points”? Maybe point it somewhere else or something?

Can you try copying something from somewhere else on your disk to see if that works?

Can you download this file https://pub.rclone.org/stat-windows-386.zip unpack the binary then run it on the Calibre Library directory and also on the Reminders.rtf file and post the output. It will print info about the file and will help me work out what is going on - thanks!

It prints something like this

$ ./stat stat.exe
Name = "stat.exe"
Size = 1930752
Mode = 0x1FD (-rwxrwxr-x)
ModTime = 2018-04-22 10:48:01.751356644 +0100 BST
IsDir = false
Sys = &{64769 6188109 1 33277 502 502 0 0 1930752 4096 3776 {1524390489 427380217} {1524390481 751356644} {1524390481 751356644} [0 0 0]}

Thanks

Nick

1 Like
7

hi it’s a onedrive folder which syns to my onedrive cloud account, the only this ive noticed is the onedrive program must have been updated, as there is a new status icon.

here is the calibre data
Name = “Calibre Library”
Size = 4096
Mode = 0x80001B6 (Lrw-rw-rw-)
ModTime = 2018-03-11 18:40:55.5912127 +0000 GMT
IsDir = false
Sys = &{1040 {3092611619 30601613} {2003112438 30652829} {2104298431 30652776} 0 4096}

here is the reminders.rft
Name = “Reminders.rtf”
Size = 212
Mode = 0x80001B6 (Lrw-rw-rw-)
ModTime = 2016-09-20 13:02:08 +0100 BST
IsDir = false
Sys = &{1056 {3123933155 30544694} {3952908340 30661207} {3483672576 30544694} 0 212}

i also run another rclone script which syns my plex photos to the same google drive, thwts works fine.

i have a feeling its to do with the onedrive folder being monitored in some new way.

oh last thing there are no reparse points files

8

That is super weird…

The go runtime thinks that Calbre Library is a file (IsDir = false) and is also is a reparse point… Windows file systems are complicated!

I wonder if this is related to: https://github.com/ncw/rclone/issues/2260

There is a potential solution you can try in that issue (disabling files on demand).

1 Like
9

So sorry for the late reply

Thank you for all you assistance the One drive update did change how it handles my files by keeping them online.

Unticking this defaulted option resolved my issue

1 Like