good, that command is working as expected.
do not know, try both options, see what happens.
would not worry about it.
i would do the following to the rclone command. note: change the path to match your system
-v to -vv
--log-path=/home/username/rclone.log
Cheers! Just to confirm, this is the code I should use?
rclone copy edu:source personal:dest --drive-server-side-across-configs --drive-stop-on-upload-limit --log-path=/home/username/rclone.log -P -vv
Do I need to change "username" with my Macbook name? Is there a way to easily find the right path to write?
username, not hostname, not machine name, not macbook name.
or use something like ~/rclone.log
does not matter, just pick a location inside your homedir.
run a test command to make sure the log file is created
rclone version -vv --log-path=~/rclone.log
Sorry, been out all day!
To make sure I understand, does this look ok to you?
rclone copy edu:source personal:dest --drive-server-side-across-configs --drive-stop-on-upload-limit --log-path=~/rclone.log -P -vv
Or do I need to add anything else?
looks good to me.
Awesome! I'm afraid to mess up with the limit of Google so I'll wait until 22:00 to try again. Thanks againn jojo!
Hey there!
I think rclone is all done with the transfer.
However I can see in my personal account I have 1.53 TB, while in the edu account I have 1.8 TB.
I have run the command once more so that rclone could run another check and I can see it's ignoring everything because it's only finding duplicates.
While it was running again, it came up with this error ("replacing/directory/name" is in place of some sensitive info)
2022-11-14 10:29:15 ERROR : 02a_Graphic Design/replacing/directory/name/_ Flyer Template/J000292 Design/Social Graphical Elements/pngs: error reading destination directory: couldn't list directory: googleapi: Error 401: Invalid Credentials, authError
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Checks: 59483 / 59483, 100%
I wonder if it's maybe the space between "_ " and "Flyer Template that's creating problems?
Once it was done (for the second time), it gave this:
2022-11-14 13:11:28 ERROR : Attempt 3/3 succeeded
Transferred: 36.722 MiB / 36.722 MiB, 100%, 136.194 KiB/s, ETA 0s
Checks: 276860 / 276860, 100%
Transferred: 17 / 17, 100%
Elapsed time: 1h28m46.0s
2022/11/14 13:11:28 INFO :
Transferred: 36.722 MiB / 36.722 MiB, 100%, 136.194 KiB/s, ETA 0s
Checks: 276860 / 276860, 100%
Transferred: 17 / 17, 100%
Elapsed time: 1h28m46.0s
The "personal" drive is now at 1.54TB (instead of the 1.53 as it was before I ran the code for the second time)
What could the difference in size mean? Is there any way to cross-check and make sure I'm not losing anything? Unfortunately I didn't do the log because it was giving me an erorr and didn't want to disturb you more than necessary.
no log, no idea. always use a debug log.
try run rclone check
I imagined that might be the case. Would running the command again with the log added be of any help? I'm happy to wait for another 1.5hour if that would give us some answers.
Last time, I think I had to specify a path but didn't know how to find out what to type in replace of "path" ...
The rclone check command is asking me for a flag - what should it be? Maybe -C --differ string or -h --missing-on-dst string?
Happy to run a check or re-run the whole command with the log, but in both events, I will need assistance on how to write the code properly - sorry!
rclone check d:\source d:\dest -vv
2022/11/14 08:58:56 DEBUG : rclone: Version "v1.60.0" starting with parameters ["rclone" "check" "d:\\source" "d:\\dest" "-vv"]
2022/11/14 08:58:56 DEBUG : Creating backend with remote "d:\\source"
2022/11/14 08:58:56 DEBUG : Using config file from "C:\\data\\rclone\\rclone.conf"
2022/11/14 08:58:56 DEBUG : fs cache: renaming cache item "d:\\source" to be canonical "//?/d:/source"
2022/11/14 08:58:56 DEBUG : Creating backend with remote "d:\\dest"
2022/11/14 08:58:56 DEBUG : fs cache: renaming cache item "d:\\dest" to be canonical "//?/d:/dest"
2022/11/14 08:58:56 INFO : Using md5 for hash comparisons
2022/11/14 08:58:56 DEBUG : Local file system at //?/d:/dest: Waiting for checks to finish
2022/11/14 08:58:56 ERROR : 01.file: file not in Local file system at //?/d:/dest
2022/11/14 08:58:56 ERROR : 02.file: file not in Local file system at //?/d:/source
2022/11/14 08:58:56 NOTICE: Local file system at //?/d:/dest: 1 files missing
2022/11/14 08:58:56 NOTICE: Local file system at //?/d:/source: 1 files missing
2022/11/14 08:58:56 NOTICE: Local file system at //?/d:/dest: 2 differences found
2022/11/14 08:58:56 NOTICE: Local file system at //?/d:/dest: 2 errors while checking
2022/11/14 08:58:56 INFO :
Sorry - are you suggesting to run this?
rclone check d:\source d:\dest -vv
Or did you mean to add something? I need a bit more direction - as you have had the chace to see I'm very much clueless about this. Thanks!
try this
rclone check edu:source personal:dest --fast-list -vv
Thanks jojo, as usual!
This is what it comes up with:
022/11/14 14:43:13 NOTICE: Google drive root 'dest': 36 files missing
2022/11/14 14:43:13 NOTICE: Google drive root 'dest': 36 differences found
2022/11/14 14:43:13 NOTICE: Google drive root 'dest': 36 errors while checking
2022/11/14 14:43:13 NOTICE: Google drive root 'dest': 2 hashes could not be checked
2022/11/14 14:43:13 NOTICE: Google drive root 'dest': 92353 matching files
2022/11/14 14:43:13 INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 36 (retrying may help)
Checks: 92353 / 92353, 100%
Elapsed time: 1m44.5s
2022/11/14 14:43:13 DEBUG : 37 go routines active
2022/11/14 14:43:13 Failed to check with 36 errors: last error was: 36 differences found
So it seems it failed to move 36 files? Any idea on how to check which one specifically? Happy to move them manually if needed.
post the full debug output.
Before I post - do you need all the lines after the command? There must be a thousand code lines there.
well, as per my example, the command should output the differences.
2022/11/14 08:58:56 ERROR : 01.file: file not in Local file system at //?/d:/dest
2022/11/14 08:58:56 ERROR : 02.file: file not in Local file system at //?/d:/source
so i am confused that you wrote
I have a ton of lines with "DEBUG" at the beginning right before what I posted. Before the DEBUG I have a ton with "INFO".
If I do a search I can find a few lines with "ERROR" - And actually, I can see at the end of the line there is written "File not found" - Is that what we are looking for? I could quickly skim through the files' names and see if I can leave them behind.
Update:
I did look for it and they seem more than 36.
2022-11-11 16:41:34 ERROR : 01_Documents/Apps/Adobe Suite CC 18/Adobe Creative Cloud CC 2018/Photoshop CC 2018 v19.0/Photoshop CC 2018/Build/Photoshop CC 2018_Uninstall.pkg/Contents/Resources/preinstall: Failed to copy: googleapi: Error 404: File not found: 11rYH7Jes8NpZzJ2JyoAk6cp5CMvPRvtr., notFound
The above is a sample of the error. Is there a way of deleting all other info and just keep the errors? These ones are buried in between "INFO" and "DEBUG" lines.
remove -vv and run the command again.
Here's what I get:
2022/11/14 15:25:21 Failed to create file system for "d:source": didn't find section in config file
MacBook-Pro-di-Francesco-3:~
Does this help?
that is the wrong command.