Copy fails when directory to create has partial match with existing directory

chscott · June 2, 2022, 3:31pm

What is the problem you are having with rclone?

Create two directories in Source: Forms Development and Forms.
Copy Source to Target.

You should see Forms Development copy successfully and the following for Forms:

2022/06/02 10:21:55 ERROR : Forms/Sample1.txt: Failed to copy: failed to make directory: invalidRequest: One of the provided arguments is not acceptable.
2022/06/02 10:21:57 ERROR : Attempt 1/1 failed with 1 errors and: failed to make directory: invalidRequest: One of the provided arguments is not acceptable.
2022/06/02 10:21:57 Failed to copy: failed to make directory: invalidRequest: One of the provided arguments is not acceptable.

Run the command 'rclone version' and share the full output of the command.

rclone v1.58.1
- os/version: Microsoft Windows 10 Pro 21H2 (64 bit)
- os/kernel: 10.0.19044.1706 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.17.9
- go/linking: dynamic
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Google Drive -> Microsoft OneDrive

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

2022/06/02 10:21:51 DEBUG : rclone: Version "v1.58.1" starting with parameters ["C:\\Temp\\rclone.exe" "copy" "--check-first" "--config" "C:\\Temp\\tmpC2B9.tmp" "--drive-skip-dangling-shortcuts" "--log-file" "C:\\Temp\\user.log" "--order-by" "size,mixed" "--retries" "1" "--stats" "5s" "--stats-one-line-date" "--update" "--log-level" "DEBUG" "--create-empty-src-dirs" "Source:Migration" "Target:"]

The rclone config contents with secrets removed.

[Source]
type = drive
scope = drive.readonly
service_account_file = C:\Temp\google.json
impersonate = user@example.org
[Target]
type = onedrive
client_id = ...
client_secret = ...
token = {"access_token":"...","token_type":"Bearer","refresh_token":"none","expiry":"2022-06-02T11:21:50.5256023-05:00"}
token_url = http://localhost:1976/m365/refreshTokenProxy
drive_id = ...
drive_type = business

A log from the command with the `-vv` flag

2022/06/02 10:21:51 DEBUG : Creating backend with remote "Source:Migration"
2022/06/02 10:21:51 DEBUG : Using config file from "C:\\Temp\\tmpC2B9.tmp"
2022/06/02 10:21:51 DEBUG : Source: detected overridden config - adding "{yHZXc}" suffix to name
2022/06/02 10:21:52 DEBUG : Google drive root 'Migration': 'root_folder_id = 0AJMfyeX0r9BWUk9PVA' - save this in the config to speed up startup
2022/06/02 10:21:52 DEBUG : fs cache: renaming cache item "Source:Migration" to be canonical "Source{yHZXc}:Migration"
2022/06/02 10:21:52 DEBUG : Creating backend with remote "Target:"
2022/06/02 10:21:52 INFO  : One drive root '': Running all checks before starting transfers
2022/06/02 10:21:53 DEBUG : One drive root '': Waiting for checks to finish
2022/06/02 10:21:53 INFO  : One drive root '': Checks finished, now starting transfers
2022/06/02 10:21:53 DEBUG : One drive root '': Waiting for transfers to finish
2022/06/02 10:21:54 DEBUG : Forms Development/Sample2.txt: Starting multipart upload
2022/06/02 10:21:55 DEBUG : Forms Development/Sample3.txt: Starting multipart upload
2022/06/02 10:21:55 ERROR : Forms/Sample1.txt: Failed to copy: failed to make directory: invalidRequest: One of the provided arguments is not acceptable.
2022/06/02 10:21:56 DEBUG : Forms Development/Sample3.txt: Uploading segment 0/22 size 22
2022/06/02 10:21:56 DEBUG : Forms Development/Sample2.txt: Uploading segment 0/22 size 22
2022/06/02 10:21:57 INFO  : Forms Development/Sample2.txt: Copied (new)
2022/06/02 10:21:57 INFO  : Forms Development/Sample3.txt: Copied (new)
2022/06/02 10:21:57 ERROR : Attempt 1/1 failed with 1 errors and: failed to make directory: invalidRequest: One of the provided arguments is not acceptable.
2022/06/02 10:21:57 INFO  : 2022/06/02 10:21:57 -          44 B / 44 B, 100%, 14 B/s, ETA 0s
2022/06/02 10:21:57 DEBUG : 15 go routines active
2022/06/02 10:21:57 Failed to copy: failed to make directory: invalidRequest: One of the provided arguments is not acceptable.

asdffdsa · June 2, 2022, 3:51pm

hi,
can you reproduce the issue using rclone mkdir?

and the top lines of the debug is missing?

chscott · June 2, 2022, 4:21pm

Yes, it is reproducible with rclone mkdir.

rclone lsf Target:
Forms Development/
rclone mkdir Target:/Forms
2022/06/02 11:20:25 ERROR : Attempt 1/3 failed with 1 errors and: failed to make directory: invalidRequest: One of the provided arguments is not acceptable.
2022/06/02 11:20:26 ERROR : Attempt 2/3 failed with 1 errors and: failed to make directory: invalidRequest: One of the provided arguments is not acceptable.
2022/06/02 11:20:26 ERROR : Attempt 3/3 failed with 1 errors and: failed to make directory: invalidRequest: One of the provided arguments is not acceptable.
2022/06/02 11:20:26 Failed to mkdir: failed to make directory: invalidRequest: One of the provided arguments is not acceptable.

asdffdsa · June 2, 2022, 4:38pm

ok, i can reproduce.

can you post the output of
rclone mkdir Target:/Forms --dump=headers --retries=1 --low-level-retries=1

chscott · June 2, 2022, 5:51pm

Sure.

2022/06/02 11:57:19 NOTICE: Automatically setting -vv as --dump is enabled
2022/06/02 11:57:19 DEBUG : rclone: Version "v1.57.0" starting with parameters ["C:\\Temp\\rclone.exe" "mkdir" "Target:/Forms" "--dump=headers" "--retries=1" "--low-level-retries=1"]
2022/06/02 11:57:19 DEBUG : Creating backend with remote "Target:/Forms"
2022/06/02 11:57:19 DEBUG : Using config file from "C:\\Users\\Chad\\.config\\rclone\\rclone.conf"
2022/06/02 11:57:19 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2022/06/02 11:57:19 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/06/02 11:57:19 DEBUG : HTTP REQUEST (req 0xc0006e6c00)
2022/06/02 11:57:19 DEBUG : GET /v1.0/drives/b!pcf_.../root HTTP/1.1
Host: graph.microsoft.com
User-Agent: rclone/v1.57.0
Authorization: XXXX
Accept-Encoding: gzip

2022/06/02 11:57:19 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/06/02 11:57:19 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/06/02 11:57:19 DEBUG : HTTP RESPONSE (req 0xc0006e6c00)
2022/06/02 11:57:19 DEBUG : HTTP/1.1 200 OK
Transfer-Encoding: chunked
Cache-Control: no-store, no-cache
Client-Request-Id: 7b083d45-1895-4a9c-b4da-69e394e5052e
Content-Type: application/json;odata.metadata=minimal;odata.streaming=true;IEEE754Compatible=false;charset=utf-8
Date: Thu, 02 Jun 2022 16:57:19 GMT
Odata-Version: 4.0
Request-Id: 7b083d45-1895-4a9c-b4da-69e394e5052e
Strict-Transport-Security: max-age=31536000
Vary: Accept-Encoding
X-Ms-Ags-Diagnostic: {"ServerInfo":{"DataCenter":"South Central US","Slice":"E","Ring":"5","ScaleUnit":"000","RoleInstance":"SN1PEPF00016C65"}}

2022/06/02 11:57:19 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/06/02 11:57:19 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/06/02 11:57:19 DEBUG : HTTP REQUEST (req 0xc0006e6500)
2022/06/02 11:57:19 DEBUG : GET /v1.0/drives/b!pcf_.../items/01P...:/Forms: HTTP/1.1
Host: graph.microsoft.com
User-Agent: rclone/v1.57.0
Authorization: XXXX
Accept-Encoding: gzip

2022/06/02 11:57:19 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/06/02 11:57:20 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/06/02 11:57:20 DEBUG : HTTP RESPONSE (req 0xc0006e6500)
2022/06/02 11:57:20 DEBUG : HTTP/1.1 404 Not Found
Transfer-Encoding: chunked
Cache-Control: no-store, no-cache
Client-Request-Id: 75249e0a-efc2-47d3-857f-40c9affcc448
Content-Type: application/json
Date: Thu, 02 Jun 2022 16:57:19 GMT
Request-Id: 75249e0a-efc2-47d3-857f-40c9affcc448
Strict-Transport-Security: max-age=31536000
Vary: Accept-Encoding
X-Ms-Ags-Diagnostic: {"ServerInfo":{"DataCenter":"South Central US","Slice":"E","Ring":"5","ScaleUnit":"000","RoleInstance":"SN1PEPF00016C6B"}}

2022/06/02 11:57:20 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/06/02 11:57:20 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/06/02 11:57:20 DEBUG : HTTP REQUEST (req 0xc0006e6e00)
2022/06/02 11:57:20 DEBUG : GET /v1.0/drives/b!pcf_.../root:/Forms HTTP/1.1
Host: graph.microsoft.com
User-Agent: rclone/v1.57.0
Authorization: XXXX
Accept-Encoding: gzip

2022/06/02 11:57:20 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/06/02 11:57:20 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/06/02 11:57:20 DEBUG : HTTP RESPONSE (req 0xc0006e6e00)
2022/06/02 11:57:20 DEBUG : HTTP/1.1 404 Not Found
Transfer-Encoding: chunked
Cache-Control: no-store, no-cache
Client-Request-Id: dfa2a89a-a642-4990-9f96-d38088959529
Content-Type: application/json
Date: Thu, 02 Jun 2022 16:57:20 GMT
Request-Id: dfa2a89a-a642-4990-9f96-d38088959529
Strict-Transport-Security: max-age=31536000
Vary: Accept-Encoding
X-Ms-Ags-Diagnostic: {"ServerInfo":{"DataCenter":"South Central US","Slice":"E","Ring":"5","ScaleUnit":"000","RoleInstance":"SN1PEPF000051D0"}}

2022/06/02 11:57:20 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/06/02 11:57:20 DEBUG : fs cache: renaming cache item "Target:/Forms" to be canonical "Target:Forms"
2022/06/02 11:57:20 DEBUG : One drive root 'Forms': Making directory
2022/06/02 11:57:20 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/06/02 11:57:20 DEBUG : HTTP REQUEST (req 0xc0002fae00)
2022/06/02 11:57:20 DEBUG : GET /v1.0/drives/b!pcf_.../items/01P...:/Forms: HTTP/1.1
Host: graph.microsoft.com
User-Agent: rclone/v1.57.0
Authorization: XXXX
Accept-Encoding: gzip

2022/06/02 11:57:20 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/06/02 11:57:20 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/06/02 11:57:20 DEBUG : HTTP RESPONSE (req 0xc0002fae00)
2022/06/02 11:57:20 DEBUG : HTTP/1.1 404 Not Found
Transfer-Encoding: chunked
Cache-Control: no-store, no-cache
Client-Request-Id: 1d400a90-5925-41f0-be89-7a491f9b2a5c
Content-Type: application/json
Date: Thu, 02 Jun 2022 16:57:20 GMT
Request-Id: 1d400a90-5925-41f0-be89-7a491f9b2a5c
Strict-Transport-Security: max-age=31536000
Vary: Accept-Encoding
X-Ms-Ags-Diagnostic: {"ServerInfo":{"DataCenter":"South Central US","Slice":"E","Ring":"5","ScaleUnit":"000","RoleInstance":"SN1PEPF0000808A"}}

2022/06/02 11:57:20 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/06/02 11:57:20 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/06/02 11:57:20 DEBUG : HTTP REQUEST (req 0xc00037e100)
2022/06/02 11:57:20 DEBUG : POST /v1.0/drives/b!pcf_.../items/01P.../children HTTP/1.1
Host: graph.microsoft.com
User-Agent: rclone/v1.57.0
Content-Length: 74
Authorization: XXXX
Content-Type: application/json
Accept-Encoding: gzip

2022/06/02 11:57:20 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2022/06/02 11:57:21 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/06/02 11:57:21 DEBUG : HTTP RESPONSE (req 0xc00037e100)
2022/06/02 11:57:21 DEBUG : HTTP/1.1 400 Bad Request
Transfer-Encoding: chunked
Cache-Control: no-store, no-cache
Client-Request-Id: 91804579-3bf9-4e5c-8073-efafd01fb054
Content-Type: application/json
Date: Thu, 02 Jun 2022 16:57:21 GMT
Request-Id: 91804579-3bf9-4e5c-8073-efafd01fb054
Strict-Transport-Security: max-age=31536000
Vary: Accept-Encoding
X-Ms-Ags-Diagnostic: {"ServerInfo":{"DataCenter":"South Central US","Slice":"E","Ring":"5","ScaleUnit":"000","RoleInstance":"SN1PEPF00008F1A"}}

2022/06/02 11:57:21 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2022/06/02 11:57:21 ERROR : Attempt 1/1 failed with 1 errors and: failed to make directory: invalidRequest: One of the provided arguments is not acceptable.
2022/06/02 11:57:21 DEBUG : 3 go routines active
2022/06/02 11:57:21 Failed to mkdir: failed to make directory: invalidRequest: One of the provided arguments is not acceptable.

ncw · June 3, 2022, 2:12pm

I replicated on onedrive business - it works fine on personal.

$ rclone mkdir onedriveb:"Forms Development"
$ rclone mkdir onedriveb:"Forms"
2022/06/03 15:07:59 ERROR : Attempt 1/3 failed with 1 errors and: failed to make directory: invalidRequest: One of the provided arguments is not acceptable.
2022/06/03 15:08:01 ERROR : Attempt 2/3 failed with 1 errors and: failed to make directory: invalidRequest: One of the provided arguments is not acceptable.
2022/06/03 15:08:03 ERROR : Attempt 3/3 failed with 1 errors and: failed to make directory: invalidRequest: One of the provided arguments is not acceptable.
2022/06/03 15:08:03 Failed to mkdir: failed to make directory: invalidRequest: One of the provided arguments is not acceptable.

It looks like it is just the Forms directory that is a problem

$ rclone mkdir onedriveb:"Worms Development"
$ rclone mkdir onedriveb:"Worms"

Can you have an experiment and see if you can find any more problematic cases?

I'm 99% sure this is a onedrive bug and I'll report it when I'm back in the office.

chscott · June 3, 2022, 3:13pm

Indeed, it is a limitation on the OneDrive side specific to "Forms."

ncw · June 3, 2022, 3:18pm

Great find. Who would have thought it!

system · June 6, 2022, 3:19pm

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.