Need help with syntax to sync windows D Drive

rsclmumbai · August 10, 2021, 2:21am

On my Windows 10 laptop, I'm trying to sync full D drive to onedrive but my commands are not copying recursive files & folders.

Here are a few commands I have tried.

D:\>D:\rclone-v1.56.0\rclone.exe sync "d:" onedrive:/SSD-64G --config "C:\Users\Admin\AppData\Roaming\rclone\rclone.conf" --progress

Copies only root folder of D:\ and not subfolders & files

D:\>D:\rclone-v1.56.0\rclone.exe sync "d:\*" onedrive:/SSD-64G --config "C:\Users\Admin\AppData\Roaming\rclone\rclone.conf" --progress 2021-08-08 22:25:19 ERROR : Local file system at //?/d:/*: error reading source root directory: directory not found

&

D:\>D:\rclone-v1.56.0\rclone.exe sync "d:**" onedrive:/SSD-64G --config "C:\Users\Admin\AppData\Roaming\rclone\rclone.conf" --progress 2021-08-09 06:49:22 ERROR : Local file system at //?/D:/**: error reading source root directory: directory not found

Error response as above

D:\>D:\rclone-v1.56.0\rclone.exe sync d:\ onedrive:/SSD-64G --config "C:\Users\Admin\AppData\Roaming\rclone\rclone.conf" --progress

Copies only root folder of D:\ and not subfolders & files

Pls advise the correct syntax.

Thanks

rsclmumbai · August 10, 2021, 6:56am

Adding further updates to this issue, I just executed rclone twice & the results are as follows

D:\rclone-v1.56.0>D:\rclone-v1.56.0\rclone.exe sync D:\ onedrive:/SSD-64G/ --config "C:\Users\Admin\AppData\Roaming\rclone\rclone.conf" --progress
Transferred:      668.037Ki / 668.037 KiByte, 100%, 2.273 KiByte/s, ETA 0s
Checks:            116509 / 116509, 100%
Deleted:                2 (files), 0 (dirs)
Transferred:            3 / 3, 100%
Elapsed time:     39m48.1s

D:\rclone-v1.56.0>D:\rclone-v1.56.0\rclone.exe sync D:\ onedrive:/SSD-64G/ --config "C:\Users\Admin\AppData\Roaming\rclone\rclone.conf" --progress --log-file=mylogfile.txt --log-level DEBUG
Transferred:              0 / 0 Byte, -, 0 Byte/s, ETA -
Errors:                 1 (retrying may help)
Checks:            349530 / 349530, 100%
Elapsed time:    3h9m39.1s

D:\rclone-v1.56.0>

No files or folders were added or deleted from D drive between the above 2 rclone execution. But still 1st attempt shows 116509 checks & 2nd attempt shows 349530 checks.

My D drive shows 46GB used & OneDrive > SSD64G folder shows 27G size. Either one of them is reporting incorrect space utilization or I've not setup rclone correctly.

I'm really struggling to get this sorted, please help!!!!

Animosity022 · August 10, 2021, 12:44pm

Best way to get help is to use the template and not delete it.

You need to share a full debug log so run your command with -vv and share the full output.

asdffdsa · August 10, 2021, 1:04pm

on windows command line.
d: would point to the current working on drive d:, which in your case is D:\rclone-v1.56.0
d:\ would point to root of the d: drive.

to see this in action, on the command line, try
dir d: or rclone ls d:
dir d:\ or rclone ls d:\

Errors: 1 (retrying may help)
that rclone command had an error, normally, rclone will try the sync command three times.
i could be wrong, but perhaps rclone is re-checking the files three times as
116509 * 3 = 349527

on d:\, how many files are there?

can you post mylogfile.txt

rsclmumbai · August 10, 2021, 1:29pm

@asdffdsa Thanks for your explanation, its a good starting point for me.

rclone ls d: listed the files in the current working directory.
rclone ls d:\ listed files & folders recursively from the root of D drive

The interesting thing is "rclone ls d:" listed files & folders recursively from only 2 folders in the d drive.
I have 79 folders & 56 files in D drive.

What can be the reasons for the "rclone ls d:" command to only show the content of 2 directories?

Thanks

rsclmumbai · August 10, 2021, 1:30pm

Log file is 150 MB. anything specific I should look for in the log file and post here for review?

asdffdsa · August 10, 2021, 1:31pm

run rclone ls d: -vv, post the command and output.

i get it, the log is huge, so look for ERROR

rsclmumbai · August 10, 2021, 2:05pm

D:\rclone-v1.56.0>rclone ls d: -vv
2021/08/10 19:32:46 DEBUG : rclone: Version "v1.56.0" starting with parameters ["rclone" "ls" "d:" "-vv"]
2021/08/10 19:32:46 DEBUG : Creating backend with remote "d:"
2021/08/10 19:32:46 DEBUG : Using config file from "C:\\Users\\Adminl\\AppData\\Roaming\\rclone\\rclone.conf"
2021/08/10 19:32:46 DEBUG : fs cache: renaming cache item "d:" to be canonical "//?/D:/rclone-v1.56.0"
  1599540 README.html
  1312667 README.txt
    73492 git-log.txt
        0 mylogfile.txt
  1504743 rclone.1
 44692992 rclone.exe
2021/08/10 19:32:46 DEBUG : 2 go routines active

D:\rclone-v1.56.0>

Output of the command you asked for.

rsclmumbai · August 10, 2021, 2:08pm

D:\rclone-v1.56.0>D:\rclone-v1.56.0\rclone.exe sync D:\ onedrive:/SSD-64G/ --config "C:\Users\Admin\AppData\Roaming\rclone\rclone.conf" --progress --log-file=mylogfile.txt --log-level ERROR
Transferred:      362.345Ki / 362.345 KiByte, 100%, 22.678 KiByte/s, ETA 0s
Checks:              8304 / 8304, 100%
Transferred:            1 / 1, 100%
Elapsed time:      1m10.5s

I ran the sync command again with log level as ERROR instead of debug, the log file is 0 bytes.

asdffdsa · August 10, 2021, 2:16pm

from the debug log of rclone ls d: -vv.
see that d: points to the current working directly and so that is what rclone will use
renaming cache item "d:" to be canonical "//?/D:/rclone-v1.56.0"

-log-level ERROR should be -log-level DEBUG
try this, i added --dry-run, to simulate what rclone will do without actually copying/deleting files.
D:\rclone-v1.56.0\rclone.exe sync D:\ onedrive:/SSD-64G/ --config "C:\Users\Admin\AppData\Roaming\rclone\rclone.conf" --progress --log-file=mylogfile.txt --log-level DEBUG --dry-run

make sure to delete the mylogfile.txt before running the rclone sync command.

rsclmumbai · August 10, 2021, 3:47pm

On it, I'm expecting it to take an hour orso, I will post back as soon as its done.

rsclmumbai · August 10, 2021, 5:19pm

D:\rclone-v1.56.0>D:\rclone-v1.56.0\rclone.exe sync D:\ onedrive:/SSD-64G/ --config "C:\Users\Admin\AppData\Roaming\rclone\rclone.conf" --progress --log-file=mylogfile.txt --log-level DEBUG --dry-run
Transferred:        2.837Mi / 2.837 MiByte, 100%, 10.496 KiByte/s, ETA 0s
Checks:            116511 / 116511, 100%
Transferred:            1 / 1, 100%
Elapsed time:     58m56.7s

D:\rclone-v1.56.0>

Process completed in an hour.
Log file is 52 MB

The first few lines & the last few lines are as follows:

2021/08/10 21:15:10 DEBUG : rclone: Version "v1.56.0" starting with parameters ["D:\\rclone-v1.56.0\\rclone.exe" "sync" "D:\\" "onedrive:/SSD-64G/" "--config" "C:\\Users\\Sandeep Periwal\\AppData\\Roaming\\rclone\\rclone.conf" "--progress" "--log-file=mylogfile.txt" "--log-level" "DEBUG" "--dry-run"]
2021/08/10 21:15:10 DEBUG : Creating backend with remote "D:\\"
2021/08/10 21:15:10 DEBUG : Using config file from "C:\\Users\\Admin\\AppData\\Roaming\\rclone\\rclone.conf"
2021/08/10 21:15:10 DEBUG : fs cache: renaming cache item "D:\\" to be canonical "//?/D:/"
2021/08/10 21:15:10 DEBUG : Creating backend with remote "onedrive:/SSD-64G/"
2021/08/10 21:15:10 DEBUG : One drive root 'SSD-64G': Token expired but no uploads in progress - doing nothing
2021/08/10 21:15:10 DEBUG : onedrive: Loaded invalid token from config file - ignoring
2021/08/10 21:15:11 DEBUG : Saving config "token" in section "onedrive" of the config file
2021/08/10 21:15:11 DEBUG : Keeping previous permissions for config file: -rw-rw-rw-
2021/08/10 21:15:11 DEBUG : onedrive: Saved new token in config file
2021/08/10 21:15:13 DEBUG : fs cache: renaming cache item "onedrive:/SSD-64G/" to be canonical "onedrive:SSD-64G"
2021/08/10 21:15:14 DEBUG : Company_Presentation.pdf: Size and modification time the same (differ by 0s, within tolerance 1s)
2021/08/10 21:15:14 DEBUG : 146-All.tar.gz: Size and modification time the same (differ by 0s, within tolerance 1s)
2021/08/10 21:15:14 DEBUG : Company_Presentation.pdf: Unchanged skipping
2021/08/10 21:15:14 DEBUG : AudioAlert.mp3: Size and modification time the same (differ by 0s, within tolerance 1s)
2021/08/10 21:15:14 DEBUG : AudioAlert.mp3: Unchanged skipping
...
...
...

2021/08/10 22:14:07 DEBUG : XMPP/Backup/rajesh/ChatApp/.git/objects/pack/pack-ca0ba3d906271e797f9bf79e7c38a43b2524d9ab.idx: Unchanged skipping
2021/08/10 22:14:07 DEBUG : XMPP/Backup/rajesh/ChatApp/.git/objects/pack/pack-ca0ba3d906271e797f9bf79e7c38a43b2524d9ab.pack: Size and modification time the same (differ by 0s, within tolerance 1s)
2021/08/10 22:14:07 DEBUG : XMPP/Backup/rajesh/ChatApp/.git/objects/pack/pack-ca0ba3d906271e797f9bf79e7c38a43b2524d9ab.pack: Unchanged skipping
2021/08/10 22:14:07 DEBUG : One drive root 'SSD-64G': Waiting for checks to finish
2021/08/10 22:14:07 DEBUG : One drive root 'SSD-64G': Waiting for transfers to finish
2021/08/10 22:14:07 DEBUG : Waiting for deletions to finish
2021/08/10 22:14:07 NOTICE: 
Transferred:   	    2.837Mi / 2.837 MiByte, 100%, 10.496 KiByte/s, ETA 0s
Checks:            116511 / 116511, 100%
Transferred:            1 / 1, 100%
Elapsed time:     58m56.8s

2021/08/10 22:14:07 DEBUG : 3 go routines active

system · October 10, 2021, 1:19pm

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.