Rclone check discrepancies

Help and Support
1

STOP and READ USE THIS TEMPLATE NO EXCEPTIONS - By not using this, you waste your time, our time and really hate puppies. Please remove these two lines and that will confirm you have read them.

What is the problem you are having with rclone?

Hi Guys, the situation that im having is that im trying to do a check of the files and the files that i have are exactly the same as the ones i have stored on idrive however it appears the following error:

Local file system at /home/daniel/Downloads/transfer: 555 files missing
2023/12/28 17:10:39 NOTICE: S3 bucket gamesystem path Nintendo DS System/Roms (USA): 555 differences found
2023/12/28 17:10:39 NOTICE: S3 bucket gamesystem path Nintendo DS System/Roms (USA): 555 errors while checking
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 555 (retrying may help)
Elapsed time: 6.3s
2023/12/28 17:10:39 NOTICE:
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 555 (retrying may help)
Elapsed time: 6.3s

2023/12/28 17:10:39 Failed to check with 555 errors: last error was: 555 differences found

Run the command 'rclone version' and share the full output of the command.

daniel@daniel-p6-2135la:~/Downloads/transfer$ rclone version

rclone v1.65.0

  • os/version: ubuntu 20.04 (64 bit)

  • os/kernel: 5.4.0-166-generic (x86_64)

  • os/type: linux

  • os/arch: amd64

  • go/version: go1.21.4

  • go/linking: static

  • go/tags: none

yes

Which cloud storage system are you using? (eg Google Drive)

idrive e2

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone check /home/daniel/Downloads/transfer idrive:/gamesystem/"Nintendo DS System"/"Roms (USA)" --dry-run -P

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

Paste config here

A log from the command that you were trying to run with the -vv flag

2023/12/28 17:13:09 NOTICE: Club Penguin - Elite Penguin Force (USA)/Club Penguin - Elite Penguin Force (USA).zip: Skipped check as --dry-run is set (size 33.327Mi)
2023/12/28 17:13:09 NOTICE: Club Penguin - Elite Penguin Force (USA) (En,Fr,Es) (Rev 2)/Club Penguin - Elite Penguin Force (USA) (En,Fr,Es) (Rev 2).zip: Skipped check as --dry-run is set (size 34.130Mi)
2023/12/28 17:13:09 NOTICE: Club Penguin - Elite Penguin Force (USA) (Rev 1)/Club Penguin - Elite Penguin Force (USA) (Rev 1).zip: Skipped check as --dry-run is set (size 32.922Mi)
2023/12/28 17:13:09 NOTICE: Clubhouse Games (USA) (En,Fr,De,Es,It)/Clubhouse Games (USA) (En,Fr,De,Es,It).zip: Skipped check as --dry-run is set (size 8.282Mi)
2023/12/28 17:13:09 NOTICE: Cocoto - Kart Racer (USA) (En,Fr,Es)/Cocoto - Kart Racer (USA) (En,Fr,Es).zip: Skipped check as --dry-run is set (size 25.846Mi)
2023/12/28 17:13:09 NOTICE: Clubhouse Games (USA) (En,Fr,De,Es,It) (Rev 1)/Clubhouse Games (USA) (En,Fr,De,Es,It) (Rev 1).zip: Skipped check as --dry-run is set (size 8.388Mi)
2023/12/28 17:13:09 NOTICE: Code Lyoko (USA) (En,Fr,Es)/Code Lyoko (USA) (En,Fr,Es).zip: Skipped check as --dry-run is set (size 54.794Mi)
2023/12/28 17:13:09 NOTICE: Code Lyoko - Fall of X A N A  (USA) (En,Fr)/Code Lyoko - Fall of X.A.N.A. (USA) (En,Fr).zip: Skipped check as --dry-run is set (size 26.821Mi)
2023/12/28 17:13:09 NOTICE: Code Lyoko - Fall of X A N A  (USA) (En,Fr) (Beta)/Code Lyoko - Fall of X.A.N.A. (USA) (En,Fr) (Beta).zip: Skipped check as --dry-run is set (size 26.812Mi)
2023/12/28 17:13:09 NOTICE: Color Cross (USA) (Beta 1)/Color Cross (USA) (Beta 1).zip: Skipped check as --dry-run is set (size 3.398Mi)
2023/12/28 17:13:09 NOTICE: Color Cross (USA) (Beta 2)/Color Cross (USA) (Beta 2).zip: Skipped check as --dry-run is set (size 1.806Mi)
2023/12/28 17:13:09 NOTICE: Command and Destroy (USA)/Command and Destroy (USA).zip: Skipped check as --dry-run is set (size 4.197Mi)
2023/12/28 17:13:09 NOTICE: Color Cross (USA) (En,Fr,Es)/Color Cross (USA) (En,Fr,Es).zip: Skipped check as --dry-run is set (size 3.399Mi)
2023/12/28 17:13:09 NOTICE: Color Cross (USA) (Beta 3)/Color Cross (USA) (Beta 3).zip: Skipped check as --dry-run is set (size 3.398Mi)
2023/12/28 17:13:09 NOTICE: Commando - Steel Disaster (USA) (En,Fr,Es)/Commando - Steel Disaster (USA) (En,Fr,Es).zip: Skipped check as --dry-run is set (size 14.322Mi)
2023/12/28 17:13:09 NOTICE: Contact (USA)/Contact (USA).zip: Skipped check as --dry-run is set (size 37.199Mi)
2023/12/28 17:13:09 NOTICE: Contra 4 (USA)/Contra 4 (USA).zip: Skipped check as --dry-run is set (size 15.614Mi)
2023/12/28 17:13:09 NOTICE: Cookie & Cream (USA)/Cookie & Cream (USA).zip: Skipped check as --dry-run is set (size 11.779Mi)
2023/12/28 17:13:09 NOTICE: Cookie Shop - Create Your Dream Shop (USA) (En,Fr,Es)/Cookie Shop - Create Your Dream Shop (USA) (En,Fr,Es).zip: Skipped check as --dry-run is set (size 13.301Mi)
2023/12/28 17:13:09 NOTICE: Cooking Mama (USA)/Cooking Mama (USA).zip: Skipped check as --dry-run is set (size 7.154Mi)
2023/12/28 17:13:09 NOTICE: Cooking Mama 2 - Dinner with Friends (USA)/Cooking Mama 2 - Dinner with Friends (USA).zip: Skipped check as --dry-run is set (size 16.712Mi)
2023/12/28 17:13:09 NOTICE: Cooking Mama 3 - Shop & Chop (USA)/Cooking Mama 3 - Shop & Chop (USA).zip: Skipped check as --dry-run is set (size 28.264Mi)
2023/12/28 17:13:09 NOTICE: Coraline (USA) (En,Fr,Es)/Coraline (USA) (En,Fr,Es).zip: Skipped check as --dry-run is set (size 14.566Mi)
2023/12/28 17:13:09 NOTICE: Cory in the House (USA)/Cory in the House (USA).zip: Skipped check as --dry-run is set (size 10.778Mi)
2023/12/28 17:13:09 NOTICE: Corvette Evolution GT (USA)/Corvette Evolution GT (USA).zip: Skipped check as --dry-run is set (size 13.313Mi)
2023/12/28 17:13:09 NOTICE: Cosmos Chaos (USA)/Cosmos Chaos (USA).zip: Skipped check as --dry-run is set (size 54.487Mi)
2023/12/28 17:13:09 NOTICE: Cradle of Rome (USA) (En,Fr,De,Es,It,Nl)/Cradle of Rome (USA) (En,Fr,De,Es,It,Nl).zip: Skipped check as --dry-run is set (size 6.147Mi)
2023/12/28 17:13:09 NOTICE: Cosmos Chaos (USA) (Beta)/Cosmos Chaos (USA) (Beta).zip: Skipped check as --dry-run is set (size 54.489Mi)
2023/12/28 17:13:09 NOTICE: Crafting Mama (USA)/Crafting Mama (USA).zip: Skipped check as --dry-run is set (size 20.451Mi)
2023/12/28 17:13:09 NOTICE: Crash - Mind over Mutant (USA) (En,Fr,De,Es,It,Nl)/Crash - Mind over Mutant (USA) (En,Fr,De,Es,It,Nl).zip: Skipped 
2023/12/28 17:13:12 NOTICE: Paws & Claws - Pet Resort (USA)/Paws & Claws - Pet Resort (USA).zip: Skipped check as --dry-run is set (size 18.808Mi)
2023/12/28 17:13:12 NOTICE: Paws & Claws - Pet Vet - Australian Adventures (USA)/Paws & Claws - Pet Vet - Australian Adventures (USA).zip: Skipped check as --dry-run is set (size 36.324Mi)
2023/12/28 17:13:12 NOTICE: Paws & Claws - Pet Vet (USA)/Paws & Claws - Pet 
2023/12/28 17:13:12 NOTICE: Prey the Stars (USA)/Prey the Stars (USA).zip: Skipped check as --dry-run is set (size 14.194Mi)
2023/12/28 17:13:12 NOTICE: Prince of Persia - The Fallen King (USA) (En,Fr,De,Es,It)/Prince of Persia - The Fallen King (USA) (En,Fr,De,Es,It).zip: Skipped check as --dry-run is set (size 17.819Mi)
2023/12/28 17:13:12 ERROR : Price Is Right, The - 2010 Edition (USA) (En,Fr)/Price Is Right, The - 2010 Edition (USA) (En,Fr).zip: file not in Local file system at /home/daniel/Downloads/transfer
2023/12/28 17:13:12 NOTICE: Prince of Persia - The Forgotten Sands (USA) (Beta 1)/Prince of Persia - The Forgotten Sands (USA) (Beta 1).zip: Skipped check as --dry-run is set (size 18.761Mi)
2023/12/28 17:13:12 NOTICE: Local file system at /home/daniel/Downloads/transfer: 550 files missing
2023/12/28 17:13:12 NOTICE: S3 bucket gamesystem path Nintendo DS System/Roms (USA): 550 differences found
2023/12/28 17:13:12 NOTICE: S3 bucket gamesystem path Nintendo DS System/Roms (USA): 550 errors while checking
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:               550 (retrying may help)
Elapsed time:         6.3s
2023/12/28 17:13:12 NOTICE: 
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:               550 (retrying may help)
Elapsed time:         6.3s
2

the only change that i did its that i place the files on the idrive e2 into a different bucket , i dont know if that has something to do with checking the files inside the bucket itself. since the files im doing the check agains are the same ones from the source i got them.

3

to see what is going on, remove --dry-run, add -vv and then run the command again.

4

My man asdffdsa

im sorry man i think i got it now it was the freakin --dry-run but as you know yesterday i almost accidentally delete a file which in that ocassion you help me with my questions, since then man " dry run " its just my best friend for all .

I think i abuse of it to the extent that rclone jsut didnt wanna work for me anymore, i was laughing man like 10 minutes now ,, " THE FORCE" just left me somehow :smiley:

1 Like
5

good move, dude

try --magic :wink:

1 Like
6

im missing out something because the source says the following:
daniel@daniel-p6-2135la:~/Downloads/transfer$ rclone size /home/daniel/Downloads/"NES ROMS"/NES/NES

Total objects: 1.256k (1256)

Total size: 122.585 MiB (128539978 Byte)

and in the idrive e2 storage states this:

daniel@daniel-p6-2135la:~/Downloads/transfer$ rclone size idrive:/gamesystem/"Nintendo Entertaiment System (NES)"/"Roms (USA)"

Total objects: 1.256k (1256)

Total size: 122.585 MiB (128539978 Byte)

7

if i ran the following:

rclone check /home/daniel/Downloads/"NES ROMS"/NES/NES idrive:/gamesystem/"Nintendo Entertaiment System (NES)"/"Roms (USA)" -vv

it gives the follwoing result:

ownloads/NES ROMS/NES/NES
2023/12/28 17:33:29 ERROR : Zelda II - The Adventure of Link (USA) (GameCube Edition)/Zelda II - The Adventure of Link (USA) (GameCube Edition).zip: file not in Local file system at /home/daniel/Downloads/NES ROMS/NES/NES
2023/12/28 17:33:29 ERROR : Zelda II - The Adventure of Link (USA) (Virtual Console)/Zelda II - The Adventure of Link (USA) (Virtual Console).zip: file not in Local file system at /home/daniel/Downloads/NES ROMS/NES/NES
2023/12/28 17:33:29 ERROR : Zen - Intergalactic Ninja (USA)/Zen - Intergalactic Ninja (USA).zip: file not in Local file system at /home/daniel/Downloads/NES ROMS/NES/NES
2023/12/28 17:33:29 ERROR : Zoda's Revenge - StarTropics II (USA)/Zoda's Revenge - StarTropics II (USA).zip: file not in Local file system at /home/daniel/Downloads/NES ROMS/NES/NES
2023/12/28 17:33:29 ERROR : Zoda's Revenge - StarTropics II (USA, Europe) (Virtual Console)/Zoda's Revenge - StarTropics II (USA, Europe) (Virtual Console).zip: file not in Local file system at /home/daniel/Downloads/NES ROMS/NES/NES
2023/12/28 17:33:29 ERROR : Zombie Nation (USA)/Zombie Nation (USA).zip: file not in Local file system at /home/daniel/Downloads/NES ROMS/NES/NES
2023/12/28 17:33:29 ERROR : [BIOS] Demo Vision (USA) (Rev 2) (Program)/[BIOS] Demo Vision (USA) (Rev 2) (Program).zip: file not in Local file system at /home/daniel/Downloads/NES ROMS/NES/NES
2023/12/28 17:33:29 NOTICE: S3 bucket gamesystem path Nintendo Entertaiment System (NES)/Roms (USA): 1256 files missing
2023/12/28 17:33:29 NOTICE: Local file system at /home/daniel/Downloads/NES ROMS/NES/NES: 1256 files missing
2023/12/28 17:33:29 NOTICE: S3 bucket gamesystem path Nintendo Entertaiment System (NES)/Roms (USA): 2512 differences found
2023/12/28 17:33:29 NOTICE: S3 bucket gamesystem path Nintendo Entertaiment System (NES)/Roms (USA): 2512 errors while checking
2023/12/28 17:33:29 INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 2512 (retrying may help)
Elapsed time: 4.1s

2023/12/28 17:33:29 DEBUG : 21 go routines active
2023/12/28 17:33:29 Failed to check with 2512 errors: last error was: 2512 differences found

8

I think i know what happened Mr. asdffdsa,

if i restore the files that im trying to compare vs the ones i downloaded from the source it gives me that the files are not the same even though as i show before the files seems to be consistent with the size and the amount of it .

and if i download the same files that i have on my bucket and i do a check then the files are all consistent, so what happened i think its that the source change something about those files that are not the same as the ones that i gather from them before , even though the quantity and the size are the same.

I think that's what happened asdffdsa. i just checekd the source discord server and they stated that the files have different dates and are not the same version anymore. so the ones i have are more updated since they roll back to another date of the files.

that has to impact something on the checks at rclone level i guess.

9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.