The full command you’re attempting to use.
rclone.exe sync E:\DataShares %customernumber%:%customernumber%-migrationbucket/%importcustomernumber%/175250Files --config C:\Users<removed>\Desktop\Rclone\rclone.conf --log-file %logpath%%logtimestamp%-%customernumber%-Database.log --log-level DEBUG
A logfile of rclone’s output with personal information removed. If it is large, you can use services like
2021/10/21 08:32:10 DEBUG : rclone: Version "v1.56.2" starting with parameters ["rclone.exe" "sync" "E:\DataShares" "000003://175250Files" "--config" "C:\Users\\Desktop\Rclone\rclone.conf" "--log-file" "C:\Users\\Desktop\Rclone\logs\10-21-2021_083209-000003-Database.log" "--log-level" "DEBUG" "--size-only"]
2021/10/21 08:32:10 DEBUG : Creating backend with remote "E:\DataShares"
2021/10/21 08:32:10 DEBUG : Using config file from "C:\Users\\Desktop\Rclone\rclone.conf"
2021/10/21 08:32:10 DEBUG : fs cache: renaming cache item "E:\DataShares" to be canonical "//?/E:/DataShares"
2021/10/21 08:32:10 DEBUG : Creating backend with remote "000003://175250Files"
2021/10/21 08:32:10 NOTICE: S3 bucket path /175250Files: Odd name received "175250Files/UsersSharedFolders/"
2021/10/21 08:32:10 NOTICE: S3 bucket path /175250Files: Odd name received "175250Files/data/"
2021/10/21 08:32:11 DEBUG : data/OS (C) - Shortcut.lnk: md5 = 967fe1383f09a17b6cc46e2b9365c36a OK
2021/10/21 08:32:11 INFO : data/OS (C) - Shortcut.lnk: Copied (new)
2021/10/21 08:32:11 DEBUG : data/shared (S) - Shortcut (2).lnk: md5 = 8fd0bcbdfaff1972b1669c07870ee6b8 OK
2021/10/21 08:32:11 INFO : data/shared (S) - Shortcut (2).lnk: Copied (new)
2021/10/21 08:32:11 DEBUG : data/shared (S) - Shortcut (3).lnk: md5 = ffda709700fed5abb6c0f6e8e099f57d OK
2021/10/21 08:32:11 INFO : data/shared (S) - Shortcut (3).lnk: Copied (new)
2021/10/21 08:32:11 DEBUG : data/shared (S) - Shortcut (4).lnk: md5 = cca344706d608d6592f145964b6b8281 OK
2021/10/21 08:32:11 INFO : data/shared (S) - Shortcut (4).lnk: Copied (new)
2021/10/21 08:32:11 DEBUG : data/shared (S) - Shortcut.lnk: md5 = e9d9ca10f194e512ec3a134594e8499b OK
2021/10/21 08:32:11 INFO : data/shared (S) - Shortcut.lnk: Copied (new)
2021/10/21 08:32:11 DEBUG : UsersSharedFolders/TBinneboese/2021 Morgan nomination letter for LEAD.doc: md5 = 7d126eca00e32613ca7038d3bf0480ac OK
2021/10/21 08:32:11 INFO : UsersSharedFolders/TBinneboese/2021 Morgan nomination letter for LEAD.doc: Copied (new)
2021/10/21 08:32:12 DEBUG : UsersSharedFolders/TBinneboese/2019 Funded Program List - 82+.docx: md5 = 8d820e3833e9aa1dd61a55dbbc635164 OK
2021/10/21 08:32:12 INFO : UsersSharedFolders/TBinneboese/2019 Funded Program List - 82+.docx: Copied (new)
2021/10/21 08:32:12 DEBUG : UsersSharedFolders/TBinneboese/2021 letter for Brad Galles.pdf: md5 = b9497dacf58994485868271b4b3cb8cf OK
2021/10/21 08:32:12 INFO : UsersSharedFolders/TBinneboese/2021 letter for Brad Galles.pdf: Copied (new)
2021/10/21 08:32:12 DEBUG : UsersSharedFolders/TBinneboese/2016 year end eval.pdf: md5 = 39615277fa2d34f6a4d4c179709b39fd OK
2021/10/21 08:32:12 INFO : UsersSharedFolders/TBinneboese/2016 year end eval.pdf: Copied (new)
2021/10/21 08:32:12 DEBUG : UsersSharedFolders/TBinneboese/2012_Human_Capital_Study_Benefits_Report.docx: md5 = db6a1578d31c86d32e4484ce138aa864 OK
The rclone config you’re using. If you don’t know where to find it, check here. Before posting ensure you’ve removed any confidential information like credentials.
[000003]
type = s3
env_auth = false
access_key_id =
secret_access_key =
region = us-east-1
endpoint =
location_constraint =
acl = private
server_side_encryption = AES256
storage_class = STANDARD
What version of rclone you’re using. It’s also helpful to try rclone with the latest beta if you’re using a stable release to understand if your issue was recently fixed.
v1.56.2
v1.56.1
v1.55.0
I'm having an issue with running Rclone Sync from a Windows 2019 Server to S3 to where the files are always being copied again. I've tried --checksum and --size-only to see if anything would change but still every time I run the sync all files are re-copied. I've also tried different versions to see if anything would change and nothing. Looking at the S3 object (most files below 200M). I do not see any Tags and only see a Content-Type and x-amz-meta-mtime Metadata on the files.
I did check on the "Odd name received" by changing the bucket path to have non-numeric naming and same issue as well.
one thing i noticed is that the dest on the command line %customernumber%:%customernumber%-migrationbucket/%importcustomernumber%/175250Files
but the dest that rclone gets is this, notice the double slashes, perhaps some of the %% variables are empty 000003://175250Files
would be helpful to enclose command with single backtick so it looks rclone.exe sync E:\DataShares %customernumber%:%customernumber%-migrationbucket/%importcustomernumber%/175250Files --config C:\Users<removed>\Desktop\Rclone\rclone.conf --log-file %logpath%%logtimestamp%-%customernumber%-Database.log --log-level DEBUG
and debug log with three backticks
2021/10/21 08:33:37 DEBUG : rclone: Version "v1.56.2" starting with parameters ["rclone.exe" "sync" "E:\DataShares" "000003://175250Files" "--config" "C:\Users\\Desktop\Rclone\rclone.conf" "--log-file" "C:\Users\\Desktop\Rclone\logs\10-21-2021_083337-000003-Database.log" "--log-level" "DEBUG"]
2021/10/21 08:33:37 DEBUG : Creating backend with remote "E:\DataShares"
2021/10/21 08:33:37 DEBUG : Using config file from "C:\Users\\Desktop\Rclone\rclone.conf"
2021/10/21 08:33:37 DEBUG : fs cache: renaming cache item "E:\DataShares" to be canonical "//?/E:/DataShares"
2021/10/21 08:33:37 DEBUG : Creating backend wit
I missed the ETag on the S3 object. Though it would be with other tags. I have confirmed that I am able to use rclone lsd to browse the s3 bucket and confirmed that the MD5 checksum values match on the example files.