Onedrive using crypt lots of errors

tonyard · January 14, 2024, 10:49am

What is the problem you are having with rclone?

Onedrive using crypt lots of errors, ok if not using crypt

024/01/14 11:33:45 ERROR : PGP/movies/MyData/local/cache/http/7b5ecaadf502ef811c1e48d237b23616.0: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent

Run the command 'rclone version' and share the full output of the command.

rclone v1.66.0-beta.7604.1045f5412

os/version: debian bookworm/sid (64 bit)
os/kernel: 6.5.0-14-generic (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.21.5
go/linking: static
go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Onedrive

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

Paste command here

/usr/bin/rclone sync /media/PGP/ pgp:PGP --skip-links --backup-dir pgp:Ver/VersionsPgp/date -I --transfers=20 --max-backlog 999999 --fast-list --log-level INFO --log-file=/home/ard/Desktop/rcpgp.txt --onedrive-delta --exclude=/media/PGP/PGP/mail/**

The rclone config contents with secrets removed.

Paste config here

[onedrive]
type = onedrive
token = {"access_token":xyz"}
drive_id = 6cd8238b248e6fc6
drive_type = personal

[pgp]
type = crypt
remote = onedrive:pgp
directory_name_encryption = false
password = xyz
password2 = xyz
filename_encryption = off

A log from the command with the `-vv` flag

2024/01/14 11:45:26 INFO :
Transferred: 164.492 MiB / 164.492 MiB, 100%, 0 B/s, ETA -
Checks: 21959 / 21959, 100%
Renamed: 227
Transferred: 182 / 182, 100%
Server Side Moves: 227 @ 164.406 MiB
Elapsed time: 15m3.7s

2024/01/14 11:45:33 NOTICE: PGP/Vote/Basingstoke - IER Declaration Expiry Renewal - Overseas (1).pdf: Duplicate object found in source - ignoring
2024/01/14 11:45:34 ERROR : PGP/movies/MyData/.directory: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:34 INFO : PGP/mail/.thunderbird/session.json.backup: Moved (server-side)
2024/01/14 11:45:34 INFO : PGP/mail/.thunderbird/AlternateServices.txt: Moved (server-side)
2024/01/14 11:45:35 INFO : PGP/mail/.thunderbird/session.json: Moved (server-side)
2024/01/14 11:45:35 INFO : PGP/mail/.thunderbird/datareporting/aborted-session-ping: Deleted
2024/01/14 11:45:35 ERROR : PGP/movies/MyData/local/.directory: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:35 INFO : PGP/mail/.thunderbird/datareporting/aborted-session-ping: Moved (server-side)
2024/01/14 11:45:35 ERROR : PGP/movies/MyData/local/cache/imdb_ratings.db: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:36 ERROR : PGP/movies/MyData/seen/.directory: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:37 ERROR : PGP/movies/MyData/local/cache/http/7b5ecaadf502ef811c1e48d237b23616.0: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:37 ERROR : PGP/movies/MyData/all/data/.directory: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:38 ERROR : PGP/movies/MyData/all/data/movies.db: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:38 ERROR : PGP/movies/MyData/all/data/movies.json: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:39 ERROR : PGP/movies/MyData/all/data/tmm.json: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:40 ERROR : PGP/movies/MyData/local/cache/http/7b5ecaadf502ef811c1e48d237b23616.1: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:41 ERROR : PGP/movies/MyData/local/cache/http/8101d9459655e922ad8ae53a8a38fa7f.0: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:42 ERROR : PGP/movies/MyData/local/data/scraper_imdb_movie.conf: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:42 ERROR : PGP/movies/MyData/all/data/tmm.prop: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:43 ERROR : PGP/movies/MyData/local/cache/http/bf8db4e4a8a8e8b5fa12f6504ebd496b.0: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:43 ERROR : PGP/movies/MyData/all/data/tmm.lic: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:44 ERROR : PGP/movies/MyData/all/data/tvshows.db: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:44 ERROR : PGP/movies/MyData/all/data/tvShows.json: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:45 ERROR : PGP/movies/MyData/local/cache/http/bf8db4e4a8a8e8b5fa12f6504ebd496b.1: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:45 ERROR : PGP/movies/MyData/local/cache/http/8101d9459655e922ad8ae53a8a38fa7f.1: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:46 ERROR : PGP/movies/MyData/local/cache/http/d4ac308a49925865ada2fc4b5867c9e9.0: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:47 ERROR : PGP/movies/MyData/local/cache/http/d4ac308a49925865ada2fc4b5867c9e9.1: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:47 ERROR : PGP/movies/MyData/local/cache/http/journal: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:48 ERROR : PGP/movies/MyData/seen/data/movies.db: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:48 ERROR : PGP/movies/MyData/seen/data/movies.json: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:49 ERROR : PGP/movies/MyData/seen/data/tmm.json: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:49 ERROR : PGP/movies/MyData/seen/data/tmm.lic: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:49 ERROR : PGP/movies/MyData/seen/data/tmm.prop: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:50 ERROR : PGP/movies/MyData/seen/data/tvshows.db: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:50 ERROR : PGP/movies/MyData/seen/data/tvShows.json: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:51 ERROR : PGP/movies/MyData/local/data/movies.db: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:51 ERROR : PGP/movies/MyData/local/data/movies.json: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:52 ERROR : PGP/movies/MyData/local/data/tmm.json: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:52 ERROR : PGP/movies/MyData/local/data/tmm.lic: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:52 ERROR : PGP/movies/MyData/local/data/tmm.prop: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:53 ERROR : PGP/movies/MyData/local/data/tvshows.db: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:53 ERROR : PGP/movies/MyData/local/data/tvShows.json: Failed to copy: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent

2024/01/14 11:45:56 ERROR : Encrypted drive 'pgp:PGP': not deleting files as there were IO errors
2024/01/14 11:45:56 ERROR : Encrypted drive 'pgp:PGP': not deleting directories as there were IO errors
2024/01/14 11:45:56 ERROR : Attempt 3/3 failed with 36 errors and: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent
2024/01/14 11:45:56 INFO :
Transferred: 165.633 MiB / 165.633 MiB, 100%, 0 B/s, ETA -
Errors: 36 (retrying may help)
Checks: 32785 / 32785, 100%
Deleted: 1 (files), 0 (dirs)
Renamed: 231
Transferred: 234 / 234, 100%
Server Side Moves: 231 @ 164.477 MiB
Elapsed time: 15m33.8s

2024/01/14 11:45:56 Failed to sync with 36 errors: last error was: failed to make directory: nameAlreadyExists: An item with the same name already exists under the parent

Paste  log here

ncw · January 14, 2024, 11:18am

Onedrive is case insensitive, so hello.txt and HELLO.txt will refer to the same file. So if you have two files which different only in case on the source then you'll get this error.

Do you have file names which differ only by case on the source?

I think that somehow crypt is masking the Case Insensitive nature of onedrive, but the case insensitive nature is showing through because you have filename_encryption = off.

Looking at the code this should work though...

What does this print?

rclone backend features pgp:PGP | grep Case

tonyard · January 14, 2024, 11:57am

Hi Thanks for your response
rclone backend features pgp:PGP | grep Case
"CaseInsensitive": true,

ncw · January 14, 2024, 5:44pm

That is as it should be.

Do you have file names which differ only by case on the source?

tonyard · January 15, 2024, 10:49am

Having both filename and folder name encryption seems to solve the problem, now seeing an error regarding folder paths too long. Don't think I'll continue with onedrive, I used the same command on googledrive with out an issue, going to try icedrive next with webdav. I checked my filenames and there aren't any that only differ in upper/lower case

kapitainsky · January 15, 2024, 10:59am

Google drive allows names up to 32768 characters. It is real oddball among cloud storages in this aspect. Usually you will encounter 255 characters limit. So migrating from gdrive somewhere else often requires renaming things. Worst still gdrive allows duplicates... which are pretty much not possible anywhere else - most likely you have to deduplicate things before transferring.

For onedrive use base32768 filenames encoding - it allows much longer names than default base32

system · February 14, 2024, 10:59am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.