Rclone union: OneDrive Personal: quotaLimitReached: Why?

juanbretti · January 4, 2023, 11:37am

What is the problem you are having with rclone?

Using rclone union with multiple OneDrive Personal, I get this error:
Failed to sync with xx errors: last error was: quotaLimitReached: Insufficient Space Available.
I do have available space in some of the accounts. I am uploading multiple 50MiB files.
Should I set up some union-action-policy or union-create-policy different than default?

My available storage:

$ rclone about jb_union:
Total:   2.039 TiB
Used:    1.759 TiB
Free:    287.027 GiB
Trashed: 47.626 GiB

Run the command 'rclone version' and share the full output of the command.

rclone is running inside a docker container.

rclone v1.61.1
- os/version: alpine 3.17.0 (64 bit)
- os/kernel: 3.10.108 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.19.4
- go/linking: static
- go/tags: none

Are you on the latest version of rclone? You can validate by checking the version listed here: Rclone downloads
--> Yes.

Which cloud storage system are you using? (eg Google Drive)

Multiple OneDrive Personal. Some have 1TB storage (2 accounts), and others have only the free 5 GB (6 accounts). To be improved with all 1TB accounts.

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone sync /usbshare/document.hbk/ jb_union:usbshare/document.hbk/ --transfers=200 --checkers=300 --delete-before --ignore-checksum --log-file=/temp/rclone_document.log -vv

The rclone config contents with secrets removed.

Some accounts were removed, but all have the same configuration.

[jb_01]
type = onedrive
client_id = f...
client_secret = N...
token = {"access_token":"E..."}
drive_id = 3...
drive_type = personal

[jb_02]
type = onedrive
client_id = 8...
client_secret = x...
token = {"access_token":"E..."}
drive_id = b...
drive_type = personal

[...]

[jb_union]
type = union
upstreams = jb_01:union jb_02:union jb_03:union jb_04:union jb_05:union jb_06:union jb_07:union jb_08:union

A log from the command with the `-vv` flag

2023/01/04 12:13:23 DEBUG : rclone: Version "v1.61.1" starting with parameters ["rclone" "sync" "/usbshare/document.hbk/" "jb_union:usbshare/document.hbk/" "--transfers=200" "--checkers=300" "--delete-before" "--ignore-checksum" "--log-file=/temp/rclone_document.log" "-vv"]
2023/01/04 12:13:23 DEBUG : Creating backend with remote "/usbshare/document.hbk/"
2023/01/04 12:13:23 DEBUG : Using RCLONE_CONFIG_PASS password.
2023/01/04 12:13:23 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/01/04 12:13:23 DEBUG : fs cache: renaming cache item "/usbshare/document.hbk/" to be canonical "/usbshare/document.hbk"
2023/01/04 12:13:23 DEBUG : Creating backend with remote "jb_union:usbshare/document.hbk/"
2023/01/04 12:13:23 DEBUG : Creating backend with remote "jb_08:union"
2023/01/04 12:13:23 DEBUG : jb_08: Loaded invalid token from config file - ignoring
2023/01/04 12:13:23 DEBUG : Creating backend with remote "jb_01:union"
2023/01/04 12:13:23 DEBUG : jb_01: Loaded invalid token from config file - ignoring
2023/01/04 12:13:23 DEBUG : Creating backend with remote "jb_02:union"
2023/01/04 12:13:23 DEBUG : jb_02: Loaded invalid token from config file - ignoring
2023/01/04 12:13:23 DEBUG : Creating backend with remote "jb_03:union"
2023/01/04 12:13:23 DEBUG : jb_03: Loaded invalid token from config file - ignoring
2023/01/04 12:13:23 DEBUG : Creating backend with remote "jb_04:union"
2023/01/04 12:13:23 DEBUG : Creating backend with remote "jb_05:union"
2023/01/04 12:13:23 DEBUG : Creating backend with remote "jb_06:union"
2023/01/04 12:13:23 DEBUG : Creating backend with remote "jb_07:union"
2023/01/04 12:13:23 DEBUG : jb_05: Loaded invalid token from config file - ignoring
2023/01/04 12:13:23 DEBUG : jb_04: Loaded invalid token from config file - ignoring
2023/01/04 12:13:23 DEBUG : jb_06: Loaded invalid token from config file - ignoring
2023/01/04 12:13:23 DEBUG : jb_07: Loaded invalid token from config file - ignoring
2023/01/04 12:13:23 DEBUG : Saving config "token" in section "jb_06" of the config file
2023/01/04 12:13:23 DEBUG : Keeping previous permissions for config file: -rwxr-xr-x
2023/01/04 12:13:23 DEBUG : jb_06: Saved new token in config file
2023/01/04 12:13:23 DEBUG : Saving config "token" in section "jb_02" of the config file
2023/01/04 12:13:23 DEBUG : Keeping previous permissions for config file: -rwxr-xr-x
2023/01/04 12:13:23 DEBUG : jb_02: Saved new token in config file
2023/01/04 12:13:23 DEBUG : Saving config "token" in section "jb_08" of the config file
2023/01/04 12:13:23 DEBUG : Saving config "token" in section "jb_04" of the config file
2023/01/04 12:13:23 DEBUG : Saving config "token" in section "jb_07" of the config file
2023/01/04 12:13:23 DEBUG : Saving config "token" in section "jb_01" of the config file
2023/01/04 12:13:23 DEBUG : Saving config "token" in section "jb_05" of the config file
2023/01/04 12:13:23 DEBUG : Saving config "token" in section "jb_03" of the config file
2023/01/04 12:13:23 DEBUG : Keeping previous permissions for config file: -rwxr-xr-x
2023/01/04 12:13:23 DEBUG : jb_08: Saved new token in config file
2023/01/04 12:13:23 DEBUG : Keeping previous permissions for config file: -rwxr-xr-x
2023/01/04 12:13:23 DEBUG : jb_04: Saved new token in config file
2023/01/04 12:13:23 DEBUG : Keeping previous permissions for config file: -rwxr-xr-x
2023/01/04 12:13:23 DEBUG : jb_03: Saved new token in config file
2023/01/04 12:13:23 DEBUG : Keeping previous permissions for config file: -rwxr-xr-x
2023/01/04 12:13:23 DEBUG : jb_01: Saved new token in config file
2023/01/04 12:13:23 DEBUG : Keeping previous permissions for config file: -rwxr-xr-x
2023/01/04 12:13:23 DEBUG : jb_07: Saved new token in config file
2023/01/04 12:13:23 DEBUG : Keeping previous permissions for config file: -rwxr-xr-x
2023/01/04 12:13:23 DEBUG : jb_05: Saved new token in config file
2023/01/04 12:13:24 DEBUG : Creating backend with remote "jb_06:union/usbshare/document.hbk"
2023/01/04 12:13:24 DEBUG : Creating backend with remote "jb_02:union/usbshare/document.hbk"
2023/01/04 12:13:24 DEBUG : Creating backend with remote "jb_01:union/usbshare/document.hbk"
2023/01/04 12:13:24 DEBUG : Creating backend with remote "jb_03:union/usbshare/document.hbk"
2023/01/04 12:13:24 DEBUG : Creating backend with remote "jb_07:union/usbshare/document.hbk"
2023/01/04 12:13:24 DEBUG : Creating backend with remote "jb_04:union/usbshare/document.hbk"
2023/01/04 12:13:24 DEBUG : Creating backend with remote "jb_05:union/usbshare/document.hbk"
2023/01/04 12:13:25 DEBUG : Creating backend with remote "jb_08:union/usbshare/document.hbk"
2023/01/04 12:13:26 DEBUG : union root 'usbshare/document.hbk': actionPolicy = *policy.EpAll, createPolicy = *policy.EpMfs, searchPolicy = *policy.FF
2023/01/04 12:13:26 DEBUG : fs cache: renaming cache item "jb_union:usbshare/document.hbk/" to be canonical "jb_union:usbshare/document.hbk"
2023/01/04 12:13:26 DEBUG : Waiting for deletions to finish
2023/01/04 12:14:26 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Elapsed time:       1m3.8s

2023/01/04 12:14:56 DEBUG : union root 'usbshare/document.hbk': Waiting for checks to finish
2023/01/04 12:14:56 DEBUG : union root 'usbshare/document.hbk': Waiting for transfers to finish
2023/01/04 12:14:57 DEBUG : _Syno_TaskConfig: Size and modification time the same (differ by -70ms, within tolerance 1s)
2023/01/04 12:14:57 DEBUG : _Syno_TaskConfig: Unchanged skipping
2023/01/04 12:14:57 DEBUG : last_status.conf: Size and modification time the same (differ by -600ms, within tolerance 1s)
2023/01/04 12:14:57 DEBUG : last_status.conf: Unchanged skipping
2023/01/04 12:14:57 DEBUG : storage_statistics.db: Size and modification time the same (differ by -610ms, within tolerance 1s)
2023/01/04 12:14:57 DEBUG : storage_statistics.db: Unchanged skipping
[...]
2023/01/04 12:14:57 DEBUG : Pool/vkey.db: Sizes differ (src 1420288 vs dst 1418240)
2023/01/04 12:14:57 DEBUG : Pool/vkey.db: Starting multipart upload
2023/01/04 12:14:57 DEBUG : Pool/vkey.db_dup: Sizes differ (src 1420288 vs dst 1418240)
2023/01/04 12:14:57 DEBUG : Pool/vkey.db_dup: Starting multipart upload
[...]
2023/01/04 12:14:58 DEBUG : Config/candidate_chunk.db_dup: Sizes differ (src 282336256 vs dst 282324992)
2023/01/04 12:14:58 DEBUG : Config/candidate_chunk.db_dup: Starting multipart upload
2023/01/04 12:14:58 DEBUG : Pool/vkey.db: Cancelling multipart upload: quotaLimitReached: Insufficient Space Available
2023/01/04 12:14:58 DEBUG : Pool/vkey.db_dup: Cancelling multipart upload: quotaLimitReached: Insufficient Space Available
2023/01/04 12:14:58 ERROR : Pool/vkey.db: Failed to copy: quotaLimitReached: Insufficient Space Available
2023/01/04 12:14:58 ERROR : Cancelling sync due to fatal error: quotaLimitReached: Insufficient Space Available
2023/01/04 12:14:58 NOTICE: Pool/vkey.db_dup: Failed to cancel multipart upload: Delete "https://api.onedrive.com/rup/57xxx/xxx/xxx-xxx/xxx/xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx-xxx": context canceled (upload failed due to: quotaLimitReached: Insufficient Space Available)
2023/01/04 12:14:58 ERROR : Pool/vkey.db_dup: Failed to copy: quotaLimitReached: Insufficient Space Available
2023/01/04 12:14:58 ERROR : Guard/local_file.db_dup: Failed to copy: Post "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!171:/local_file.db_dup:/createUploadSession": context canceled
2023/01/04 12:14:58 ERROR : Config/candidate_chunk.db: Failed to copy: Post "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!145:/candidate_chunk.db:/createUploadSession": context canceled
2023/01/04 12:14:58 ERROR : Pool/chunk_index: error reading destination directory: 3 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Pool:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Pool:": context canceled; jb_08: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/57xxx/items/57xxx!574/children?$top=1000": context canceled
2023/01/04 12:14:59 ERROR : Config/virtual_file.index: error reading destination directory: 8 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Config:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131571:/virtual_file.index:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Config:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Config:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Config:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Config:": context canceled; jb_07: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!156/children?$top=1000": context canceled; jb_08: couldn't list files: context canceled
2023/01/04 12:14:59 ERROR : Config/@Share: error reading destination directory: 8 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Config:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131571:/@Share:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Config:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Config:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Config:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Config:": context canceled; jb_07: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!557/children?$top=1000": context canceled; jb_08: Get "https://graph.microsoft.com/v1.0/drives/57xxx/items/57xxx!173:/@Share:": context canceled
2023/01/04 12:14:59 ERROR : Config/file_chunk1.index: error reading destination directory: 8 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Config:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131571:/file_chunk1.index:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Config:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Config:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Config:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Config:": context canceled; jb_07: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!189/children?$top=1000": context canceled; jb_08: Get "https://graph.microsoft.com/v1.0/drives/57xxx/items/57xxx!173:/file_chunk1.index:": context canceled
2023/01/04 12:14:59 ERROR : Config/file_chunk2.index: error reading destination directory: 8 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Config:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131571:/file_chunk2.index:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Config:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Config:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Config:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Config:": context canceled; jb_07: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!543/children?$top=1000": context canceled; jb_08: Get "https://graph.microsoft.com/v1.0/drives/57xxx/items/57xxx!173:/file_chunk2.index:": context canceled
2023/01/04 12:14:59 ERROR : Config/file_chunk3.index: error reading destination directory: 8 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Config:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131571:/file_chunk3.index:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Config:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Config:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Config:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Config:": context canceled; jb_07: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!190/children?$top=1000": context canceled; jb_08: Get "https://graph.microsoft.com/v1.0/drives/57xxx/items/57xxx!173:/file_chunk3.index:": context canceled
2023/01/04 12:14:59 ERROR : Config/candidate_chunk.db_dup: Failed to copy: Post "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!145:/candidate_chunk.db_dup:/createUploadSession": context canceled
2023/01/04 12:14:59 ERROR : Config/file_chunk4.index: error reading destination directory: 8 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Config:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131571:/file_chunk4.index:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Config:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Config:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Config:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Config:": context canceled; jb_07: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!555/children?$top=1000": context canceled; jb_08: Get "https://graph.microsoft.com/v1.0/drives/57xxx/items/57xxx!173:/file_chunk4.index:": context canceled
2023/01/04 12:14:59 ERROR : Config/read_locker: error reading destination directory: 8 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Config:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131571:/read_locker:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Config:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Config:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Config:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Config:": context canceled; jb_07: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!121230/children?$top=1000": context canceled; jb_08: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/57xxx/items/57xxx!2533/children?$top=1000": context canceled
2023/01/04 12:15:00 ERROR : Pool/0/9: error reading destination directory: 8 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Pool:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131567:/Pool:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Pool:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Pool:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Pool:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Pool:": context canceled; jb_07: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!114:/0:": context canceled; jb_08: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/57xxx/items/57xxx!4087/children?$top=1000": context canceled
2023/01/04 12:15:01 ERROR : Pool/0/2: error reading destination directory: 8 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Pool:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131567:/Pool:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Pool:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Pool:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Pool:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Pool:": context canceled; jb_07: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!114:/0:": context canceled; jb_08: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/57xxx/items/57xxx!11889/children?$top=1000": context canceled
2023/01/04 12:15:01 ERROR : Pool/0/4: error reading destination directory: 8 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Pool:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131567:/Pool:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Pool:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Pool:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Pool:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Pool:": context canceled; jb_07: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!114:/0:": context canceled; jb_08: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/57xxx/items/57xxx!11884/children?$top=1000": context canceled
2023/01/04 12:15:01 ERROR : Pool/0/1: error reading destination directory: 8 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Pool:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131567:/Pool:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Pool:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Pool:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Pool:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Pool:": context canceled; jb_07: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!114:/0:": context canceled; jb_08: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/57xxx/items/57xxx!11886/children?$top=1000": context canceled
2023/01/04 12:15:01 ERROR : Pool/0/5: error reading destination directory: 8 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Pool:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131567:/Pool:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Pool:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Pool:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Pool:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Pool:": context canceled; jb_07: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!114:/0:": context canceled; jb_08: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/57xxx/items/57xxx!8052/children?$top=1000": context canceled
2023/01/04 12:15:01 ERROR : Pool/0/6: error reading destination directory: 8 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Pool:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131567:/Pool:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Pool:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Pool:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Pool:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Pool:": context canceled; jb_07: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!114:/0:": context canceled; jb_08: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/57xxx/items/57xxx!11888/children?$top=1000": context canceled
2023/01/04 12:15:01 ERROR : Pool/0/8: error reading destination directory: 8 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Pool:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131567:/Pool:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Pool:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Pool:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Pool:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Pool:": context canceled; jb_07: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!114:/0:": context canceled; jb_08: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/57xxx/items/57xxx!11892/children?$top=1000": context canceled
2023/01/04 12:15:01 ERROR : Pool/0/0: error reading destination directory: 8 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Pool:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131567:/Pool:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Pool:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Pool:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Pool:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Pool:": context canceled; jb_07: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!114:/0:": context canceled; jb_08: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/57xxx/items/57xxx!11878/children?$top=1000": context canceled
2023/01/04 12:15:01 ERROR : Pool/0/7: error reading destination directory: 8 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Pool:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131567:/Pool:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Pool:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Pool:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Pool:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Pool:": context canceled; jb_07: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!114:/0:": context canceled; jb_08: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/57xxx/items/57xxx!11895/children?$top=1000": context canceled
2023/01/04 12:15:01 ERROR : Config/@cand_file: error reading destination directory: 7 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Config:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131571:/@cand_file:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Config:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Config:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Config:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Config:": context canceled; jb_07: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!1416/children?$top=1000": context canceled
2023/01/04 12:15:01 ERROR : Pool/0/3: error reading destination directory: 8 errors: jb_01: Get "https://graph.microsoft.com/v1.0/drives/32xxx/items/32xxx!49078:/Pool:": context canceled; jb_02: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!131567:/Pool:": context canceled; jb_03: Get "https://graph.microsoft.com/v1.0/drives/46xxx/items/46xxx!1089:/Pool:": context canceled; jb_04: Get "https://graph.microsoft.com/v1.0/drives/2bxxx/items/2bxxx!2119:/Pool:": context canceled; jb_05: Get "https://graph.microsoft.com/v1.0/drives/b8xxx/items/b8xxx!2613:/Pool:": context canceled; jb_06: Get "https://graph.microsoft.com/v1.0/drives/5bxxx/items/5bxxx!111:/Pool:": context canceled; jb_07: Get "https://graph.microsoft.com/v1.0/drives/d8xxx/items/d8xxx!114:/0:": context canceled; jb_08: couldn't list files: Get "https://graph.microsoft.com/v1.0/drives/57xxx/items/57xxx!11894/children?$top=1000": context canceled
2023/01/04 12:15:01 DEBUG : union root 'usbshare/document.hbk': Waiting for checks to finish
2023/01/04 12:15:01 DEBUG : union root 'usbshare/document.hbk': Waiting for transfers to finish
2023/01/04 12:15:01 ERROR : Fatal error received - not attempting retries
2023/01/04 12:15:01 INFO  : 
Transferred:   	    2.709 MiB / 2.709 MiB, 100%, 923.568 KiB/s, ETA 0s
Errors:                24 (fatal error encountered)
Checks:                32 / 32, 100%
Elapsed time:      1m38.6s

2023/01/04 12:15:01 DEBUG : 22 go routines active
2023/01/04 12:15:01 Failed to sync with 24 errors: last error was: quotaLimitReached: Insufficient Space Available

Thank you for your ideas and support!

juanbretti · January 4, 2023, 2:21pm

I tried adding --union-action-policy=epmfs.
So, the full command is:

rclone sync /usbshare/document.hbk/ jb_union:usbshare/document.hbk/ --transfers=200 --checkers=300 --delete-before --ignore-checksum --log-file=/temp/rclone_document.log --union-action-policy=epmfs

But I get this new error:

panic: interface conversion: upstream. Entry is nil, not *upstream.Object

goroutine 5711 [running]:
github.com/rclone/rclone/backend/union.(*Object).Update(0xc00096ba10, {0x2440e88?, 0xc000b0df00}, {0x242b740, 0xc0003cc900}, {0x7f2bca699218?, 0xc00111c4e0}, {0xc002a6e450, 0x1, 0x1})
	github.com/rclone/rclone/backend/union/entry.go:76 +0x598
github.com/rclone/rclone/fs/operations.Copy({0x2440e88, 0xc000b0df00}, {0x2453da0, 0xc0001c81e0}, {0x2453e10?, 0xc00096ba10?}, {0xc000be5c80, 0xc}, {0x2452bb0, 0xc00111c4e0})
	github.com/rclone/rclone/fs/operations/operations.go:437 +0x1c05
github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairCopyOrMove(0xc00056e900, {0x2440e88, 0xc000b0df00}, 0xc000774000?, {0x2453da0, 0xc0001c81e0}, 0x114000000e3?, 0x123000000d3?)
	github.com/rclone/rclone/fs/sync/sync.go:423 +0x1bb
created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startTransfers
	github.com/rclone/rclone/fs/sync/sync.go:450 +0x45

Any ideas? Thanks!

ncw · January 4, 2023, 2:59pm

That is definitely a bug! Can you open a new issue on Github with it in, and put full details so I can reproduce the problem - thanks.

juanbretti · January 4, 2023, 3:30pm

@ncw , thank you for your reply.
Doing more research, I noticed on one of the OneDrive Personal accounts (with 1TiB of storage), I get this error:

Looks like rclone had uploaded more than the maximum storage (1024+5GiB).
I checked with the command:

$ rclone size jb_08:
Enter configuration password:
password:
Total objects: 77815
Total size: 1.005 TBytes (1104896322594 Bytes)

$ rclone about jb_08:
Enter configuration password:
password:
Total:   1.005T
Used:    1.005T
Free:    off
Trashed: 0

Doing some conversions:


Used
1,104,896,322,594	Used Bytes


Purchased
1,099,511,627,776	1TiB in Bytes
5,368,709,120	5GiB in Bytes
1,104,880,336,896	Sum in Bytes


Difference
-15,985,698	Overused in Bytes
-15.25	Overused in MiB

Is there any way to use something like --union-min-free-space?
Something to protect not overusing a remote storage?

Also, any ideas how to fix it on my OneDrive Personal account?

juanbretti · January 13, 2023, 7:36pm

Any ideas on how to achieve this?
I mean, not overusing a remote storage.

Thanks!

asdffdsa · January 26, 2023, 3:11pm

hi,
as mentioned, this is definitely a bug!

the next step is to create the new github issue?
can you post the link here?

juanbretti · January 26, 2023, 3:37pm

Here is the issue: union: quotaLimitReached: OneDrive Personal · Issue #6731 · rclone/rclone · GitHub

rcloneuser29754 · February 1, 2023, 9:31pm

have you tried --union-action-policy=mfs or --union-action-policy=lus?

juanbretti · February 1, 2023, 10:08pm

Thank you for your reply.
@ncw thinks the issue is because I had set too many transfers (I had set 200). Maybe those were competing for the same space.
More here: union: quotaLimitReached: OneDrive Personal · Issue #6731 · rclone/rclone · GitHub

system · February 6, 2023, 9:16am

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.