After renaming a sub-folder on remote backup (Dropbox) I’m getting this error for some files:
ERROR : [file name]: Couldn't move: move failed: from_lookup/not_found/..
And the referenced files are not being copied.
What exactly does this error mean?
According to the Dropbox docs that means There is nothing at the given path.
I don’t know exactly what has happened.
I assume you are using an rclone mount. What happens if you stop it and restart it?
Can you try to reproduce the problem and post the steps you followed to get it?
No, I’m not using mount, just syncing from local to Dropbox.
See a screen with the log for one of these files:
I use the --backup-dir option, so it looks like Rclone identified the file as a new version (although it was not changed), tried to move it, but some error occurred.
These files are still on my local disk, and will be sent again.
My guess is that since this is a very large folder with many files, Dropbox was still doing something internally (moving, or renaming, I don’t know how it does), and when I ran the Rclone job something was still not finished.
I’m running the sync job to send recent added files to this - renamed - storage, and so far no errors.
I keep getting the error messages, even running the job a day later (that’s for sure, it’s not an “inner run” issue of Dropbox).
Recapitulating:
I renamed a folder from “folder_old” to “new_folder”, without changing the location (via Dropbox web interface);
This folder was already being backed up for several months, without problems;
It is a large folder with a lot of files, almost never changed, but a few files are added weekly;
Now when I’m running the job I get several error messages for just some files:
src and dst identical but can't set mod time without deleting and re-uploading
Couldn't move: move failed: from_lookup/not_found/
And what’s worse: these files are being deleted from storage. All these files show history of events similar to the screen posted above.
The only thing I thought might interfere is that the new folder name is a bit bigger, but the full path has been around 170 characters, I think that’s not a problem.
This error is caused by rclone detecting that the modification time of the file is wrong. Unfortunately with dropbox you can’t just set the modification time, you have to upload the whole file.
So if you use either --checksum or --size-only this will work around that. Or let rclone upload the file to fix the modification time.
That is the error I really don’t understand. It is a bug in rclone or dropbox I’d say.
Can you send me a log with -vv --log-file rclone.log of this problem happening? If you want to keep it private you can email to nick@craig-wood.com - put a link to this page in the email so I know what it is referring to.
Thanks
Yes, this is what I was thinking would happen, but the files that shows the errors bellow are being deleted.
I sent you two log files:
2017/12/16 10:09:58 DEBUG : Diversos/avioes/aeromodelo SU27 - construcao/www.su27.de/504.jpg: Unchanged skipping
2018/03/12 19:22:39 INFO : Diversos/avioes/aeromodelo SU27 - construcao/www.su27.de/504.jpg: Copied (new)
and 2,063 entries with the “from_lookup/not_found” error, that were deleted from Dropbox and not uploaded again.
Thanks for the log.
I see what is happening now!
What is happening is this…
Rclone notices that the file has the incorrect modification time
2018/03/12 19:04:37 DEBUG : file.jpg: Modification times differ by -91373h52m55.7006214s: 2018-01-25 00:40:19.7006214 -0200 -02, 2007-08-23 20:47:24 +0000 UTC
Rclone figures out it will have to re-upload the file
2018/03/12 19:04:37 DEBUG : file.jpg: src and dst identical but can't set mod time without deleting and re-uploading
So it deletes it in preparation for the re-upload.
However since you have --backup-dir set, rclone thinks there is an existing file.jpg which it needs to move into the backup dir.
However trying that move gives this error (since rclone already deleted the file in preparation for the re-upload).
2018/03/12 19:04:42 ERROR : file.jpg: Couldn't move: move failed: from_lookup/not_found/
Rclone then abandons the upload of the file.
How to fix…
I think in an ideal world we’d like rclone to put the incorrectly dated file.jpg into the --backup-dir, but at minimum we want rclone to transfer the file straight away and not wait for the retry.
Here is my attempt to replicate the problem. Note I see from this that rclone does transfer the file correctly on the retry so you should be seeing that too.
$ echo "hello" > file.txt
$ rclone -v copy file.txt dropbox:test/test1
2018/03/13 09:37:30 INFO : Dropbox root 'test/test1': Modify window is 1s
2018/03/13 09:37:30 INFO : Dropbox root 'test/test1': Waiting for checks to finish
2018/03/13 09:37:30 INFO : Dropbox root 'test/test1': Waiting for transfers to finish
2018/03/13 09:37:31 INFO : file.txt: Copied (new)
2018/03/13 09:37:31 INFO :
Transferred: 6 Bytes (3 Bytes/s)
Errors: 0
Checks: 0
Transferred: 1
Elapsed time: 1.9s
$ touch file.txt
$ rclone -vv copy file.txt --backup-dir dropbox:test/test2 dropbox:test/test1
2018/03/13 09:38:05 DEBUG : Using config file from "/home/ncw/.rclone.conf"
2018/03/13 09:38:05 DEBUG : rclone: Version "v1.39-233-g383feca6" starting with parameters ["rclone" "-vv" "copy" "file.txt" "--backup-dir" "dropbox:test/test2" "dropbox:test/test1"]
2018/03/13 09:38:06 INFO : Dropbox root 'test/test1': Modify window is 1s
2018/03/13 09:38:07 INFO : Dropbox root 'test/test1': Waiting for checks to finish
2018/03/13 09:38:07 DEBUG : file.txt: Modification times differ by -45.652019585s: 2018-03-13 09:37:38.652019585 +0000 GMT, 2018-03-13 09:36:53 +0000 UTC
2018/03/13 09:38:07 DEBUG : file.txt: src and dst identical but can't set mod time without deleting and re-uploading
2018/03/13 09:38:08 ERROR : file.txt: Couldn't move: move failed: from_lookup/not_found/...
2018/03/13 09:38:08 INFO : Dropbox root 'test/test1': Waiting for transfers to finish
2018/03/13 09:38:08 ERROR : Attempt 1/3 failed with 1 errors and: move failed: from_lookup/not_found/...
2018/03/13 09:38:10 INFO : Dropbox root 'test/test1': Waiting for checks to finish
2018/03/13 09:38:10 INFO : Dropbox root 'test/test1': Waiting for transfers to finish
2018/03/13 09:38:10 INFO : file.txt: Copied (new)
2018/03/13 09:38:10 ERROR : Attempt 2/3 succeeded
2018/03/13 09:38:10 INFO :
Transferred: 6 Bytes (1 Bytes/s)
Errors: 0
Checks: 1
Transferred: 1
Elapsed time: 5s
2018/03/13 09:38:10 DEBUG : 7 go routines active
2018/03/13 09:38:10 DEBUG : rclone: Version "v1.39-233-g383feca6" finishing with parameters ["rclone" "-vv" "copy" "file.txt" "--backup-dir" "dropbox:test/test2" "dropbox:test/test1"]
$ rclone lsl dropbox:test/test1 # file is correct here
6 2018-03-13 09:37:39.000000000 file.txt
$ rclone lsl dropbox:test/test2 # nothing in the backup
2018/03/13 09:38:58 ERROR : : error listing: directory not found
2018/03/13 09:38:58 Failed to lsl: directory not found
$ ls -l --full-time file.txt
-rw-rw-r-- 1 ncw ncw 6 2018-03-13 09:37:38.652019585 +0000 file.txt
I created a new issue #2134 about this - can you subscribe to that issue and I’ll post fixes there.
I agree, in fact, I thought Rclone would do that.
Yes, maybe Dropbox has complained about a lot of attempts on multiple files in my case.
Thanks for the support! I’ll follow the issue on GitHub.
On the other hand, I always try to identify the “root cause” of a problem, and in this case it looks like it was Dropbox because I understand that something happened with “modification time” of the file when I renamed the folder.
Yes, the root cause of the issue looks like that something happened with the modification times when the folder was renamed.
How did you do the rename? Via the website or using rclone? I’m pretty sure that if you do it with rclone it will preserve the modification times (I have integration tests for that).
I did via Dropbox web interface.
Next time I will change a folder so large, I’ll use Rclone 
Thinking about what I’m evaluating on this topic, I think I’m going to do a rclone copy of the current folders to the new structure instead of using the Dropbox interface, and then delete the initial files.
In this case, will the copy command run on the server side (without donwload / upload all files)?
It should do yes.
However wouldn’t doing an rclone move be more efficient? I guess you might run into this problem!
I find the easiest way of re-arranging files is to run an rclone mount then use standard tools to shuffle things about.
You’re right, it’ll be new “modification times” …
I’ll try with mount.
I just tried this (rclone mount, then using mv file dest) but I still get the error:
2018/05/17 14:22:23 ERROR : file: Dir.Rename error: Fs "Azure container t" can't rename files (no Move)
Azure blob doesn’t yet support move on mount since you can’t actually move files on Azure blob. However I’m planning to implement it with copy then delete in this issue: https://github.com/ncw/rclone/issues/1965 which hopefully I’ll get done for v1.42. If you want updates then subscribe to the issue.