Again...remote folders wiped out, help!

Ok, I can follow you.
A concrete scenario is:

  • Sync of folder H:\Anime to the remote
  • I/O Error and no other retries
  • Sync of folder H:\Cartoons to the remote. The folder contains data from the same offline disk of Anime folder
  • The folder Cartoons on the remote is wiped partially or completelly based on where the data is spanned

At the moment the situation of concrete data (not empty folder) in my server is balanced by the software like this:

  1. Anime -> Q
  2. Cartoons -> P
  3. Movies -> S / T / U
  4. Tv shows -> M / V / W / X

So 2 different syncs are not reading data from the same source usually but this is a good scenario.

What I'm thinking now are different solutions

  • Use the --retries 1 and make a batch delay between sync1 and sync2
  • Use the --retries 5 and add --retries-sleep 60s to give drivepool time to recover. Additionally make a batch delay between sync1 and sync2

Both solution obviously cover only a temporary disk down. If the disk dies the solutions doesn't works..

Another idea can be to intercept a sort of return code from the sync and stop all the next syncs in this case..is this possible? this is a extreme solution. IMHO if a drive dies and the sync run, all the files are placed in the backup directory and are easily recoverable with a rclone move command.

So to recap, I'm starting to thing that the best solution is:

  • Use the --retries 5 and add --retries-sleep 60s to give drivepool time to recover. Additionally make a batch delay between sync1 and sync2

Sorry, but the above log shows that your idea of using --retries and --retries-sleep will not protect you from a deletion of (almost) all the files in a sync.

I can only see the following possibilities:

  • Sync as now, but limit damage with --max-delete
  • Sync all the individual parts one-by-one
  • Sync an rclone union remote on top of the individual parts
  • Eliminate/Reduce the risk of disks going offline
  • Replace the DrivePool with something else.
    An rclone mount of an rclone union can perhaps do the trick - I have no experience.

Here's the incriminated full log of when the folder was wiped:

  • The batch started at 2021/08/18 02:00:03
  • At 2021/08/18 02:02:34 we have the error
2021/08/18 02:02:34 ERROR : Shaman King (2021)/Stagione 01 [1080p x264] [ITA+JPN]/Shaman King (2021) - S01E02 - 002 - Un altro sciamano [WEBDL 1080p x264] [ITA+JPN].mkv: Failed to copy: Post "https://www.googleapis.com/upload/drive/v3/files?alt=json&fields=id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks&supportsAllDrives=true&uploadType=resumable&upload_id=ADPycduAlhRtJg2VX0jgJo8uDfNF5_yq7sn9Qg9A9RDnxH_ByCsskks8r88FDKZuZbJcaVEzocBSBgfhTBzQG83qJ_Qi7atoYg": read \\?\H:\Anime\Shaman King (2021)\Stagione 01 [1080p x264] [ITA+JPN]\Shaman King (2021) - S01E02 - 002 - Un altro sciamano [WEBDL 1080p x264] [ITA+JPN].mkv: A device which does not exist was specified.
2021/08/18 02:02:34 ERROR : Shaman King (2021)/Stagione 01 [1080p x264] [ITA+JPN]/Shaman King (2021) - S01E05 - 005 - Oltreanima! [WEBDL 1080p x264] [ITA+JPN].mkv: Failed to copy: failed to open source object: The system cannot find the path specified.
2021/08/18 02:02:34 ERROR : Shaman King (2021)/Stagione 01 [1080p x264] [ITA+JPN]/Shaman King (2021) - S01E06 - 006 - Yoh vs Horohoro! [WEBDL 1080p x264] [ITA+JPN].mkv: Failed to copy: failed to open source object: The system cannot find the path specified.
2021/08/18 02:02:34 ERROR : Shaman King (2021)/Stagione 01 [1080p x264] [ITA+JPN]/Shaman King (2021) - S01E07 - 007 - Una forma di coraggio [WEBDL 1080p x264] [ITA+JPN].mkv: Failed to copy: failed to open source object: The system cannot find the path specified.
2021/08/18 02:02:34 ERROR : Shaman King (2021)/Stagione 01 [1080p x264] [ITA+JPN]/Shaman King (2021) - S01E08 - 008 - Evoluzione [WEBDL 1080p x264] [ITA+JPN].mkv: Failed to copy: failed to open source object: The system cannot find the path specified.
2021/08/18 02:02:34 ERROR : Shaman King (2021)/Stagione 01 [1080p x264] [ITA+JPN]/Shaman King (2021) - S01E09 - 009 - Di nuovo Yoh vs Ren! [WEBDL 1080p x264] [ITA+JPN].mkv: Failed to copy: failed to open source object: The system cannot find the path specified.
2021/08/18 02:02:34 ERROR : Shaman King (2021)/Stagione 01 [1080p x264] [ITA+JPN]/Shaman King (2021) - S01E10 - 010 - Notte in fiamme [WEBDL 1080p x264] [ITA+JPN].mkv: Failed to copy: failed to open source object: The system cannot find the path specified.
2021/08/18 02:02:34 ERROR : Shaman King (2021)/Stagione 01 [1080p x264] [ITA+JPN]/Shaman King (2021) - S01E11 - 011 - Storia di due uomini [WEBDL 1080p x264] [ITA+JPN].mkv: Failed to copy: failed to open source object: The system cannot find the path specified.
2021/08/18 02:02:34 ERROR : Shaman King (2021)/Stagione 01 [1080p x264] [ITA+JPN]/Shaman King (2021) - S01E12 - 012 - Ren vs En - La fine dei Tao [WEBDL 1080p x264] [ITA+JPN].mkv: Failed to copy: failed to open source object: The system cannot find the path specified.
2021/08/18 02:02:34 ERROR : Shaman King (2021)/Stagione 01 [1080p x264] [ITA+JPN]/Shaman King (2021) - S01E13 - 013 - E poi Hao! [WEBDL 1080p x264] [ITA+JPN].mkv: Failed to copy: failed to open source object: The system cannot find the path specified.
2021/08/18 02:02:34 ERROR : Shaman King (2021)/Stagione 01 [720p x264] [JAPANESE]/Shaman King (2021) - S01E14 - 014 - Lyserg the Avenger [WEBDL 720p x264] [JAPANESE].mkv: Failed to copy: failed to open source object: The system cannot find the path specified.
2021/08/18 02:02:34 ERROR : Shaman King (2021)/Stagione 01 [720p x264] [JAPANESE]/Shaman King (2021) - S01E15 - 015 - When the Pieces Come Together [WEBDL 720p x264] [JAPANESE].mkv: Failed to copy: failed to open source object: The system cannot find the path specified.
2021/08/18 02:02:34 ERROR : Shaman King (2021)/Stagione 01 [720p x264] [JAPANESE]/Shaman King (2021) - S01E16 - 016 - Enter the Ultra-Pompadour [WEBDL 720p x264] [JAPANESE].mkv: Failed to copy: failed to open source object: The system cannot find the path specified.
2021/08/18 02:02:34 ERROR : Shaman King (2021)/Stagione 01 [720p x264] [JAPANESE]/Shaman King (2021) - S01E17 - 017 - Guns of the Angels [WEBDL 720p x264] [JAPANESE].mkv: Failed to copy: failed to open source object: The system cannot find the path specified.
2021/08/18 02:02:34 ERROR : Bleach (2004)/Stagione 00 - Specials [480p x264] [JAPANESE]/Bleach (2004) - S00E03 - Bleach Memories of Nobody [WEBDL 480p x264] [JAPANESE].mkv: Failed to copy: failed to open source object: The system cannot find the path specified.
2021/08/18 02:02:34 ERROR : Bleach (2004)/Stagione 00 - Specials [480p x264] [JAPANESE]/Bleach (2004) - S00E04 - Bleach The DiamondDust Rebellion [WEBDL 480p x264] [JAPANESE].mkv: Failed to copy: failed to open source object: The system cannot find the path specified.
  • at 2021/08/18 02:02:34 the message of no file deletion.
    PS how this will work with --delete during option?
2021/08/18 02:02:34 ERROR : Encrypted drive 'Media:Anime': not deleting files as there were IO errors
2021/08/18 02:02:34 ERROR : Encrypted drive 'Media:Anime': not deleting directories as there were IO errors
2021/08/18 02:02:34 ERROR : Attempt 1/5 failed with 46 errors and: Post "https://www.googleapis.com/upload/drive/v3/files?alt=json&fields=id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks&supportsAllDrives=true&uploadType=resumable&upload_id=ADPycdu8aAIxGYzVWcrw-wFtjkE9IH532qGMLhdr0pQyKSA0raWg67JSubBJcflcQ3jRo3I_pI_zDkyYUzvJYpky7OEAfetoBw": read \\?\H:\Anime\Shaman King (2021)\Stagione 01 [1080p x264] [ITA+JPN]\Shaman King (2021) - S01E01 - 001 - Il ragazzo che balla con i fantasmi [WEBDL 1080p x264] [ITA+JPN].mkv: A device which does not exist was specified.
  • at 2021/08/18 02:02:35 rclone is transfering new files. I have reason to think that at this time the drive is again online. is this the first or second attempt?
2021/08/18 02:02:35 INFO  : 
Transferred:   	  731.163M / 731.163 MBytes, 100%, 5.723 MBytes/s, ETA 0s
Checks:              8897 / 8897, 100%
Transferred:            1 / 1, 100%
Elapsed time:      2m31.6s

2021/08/18 02:02:45 INFO  : 
Transferred:   	  731.163M / 731.163 MBytes, 100%, 5.723 MBytes/s, ETA 0s
Checks:              8897 / 8897, 100%
Transferred:            1 / 1, 100%
Elapsed time:      2m41.6s

2021/08/18 02:02:50 INFO  : Encrypted drive 'Media:Anime': Making map for --track-renames
2021/08/18 02:02:50 INFO  : Encrypted drive 'Media:Anime': Finished making map for --track-renames
2021/08/18 02:02:55 INFO  : 
Transferred:   	  750.637M / 3.390 GBytes, 22%, 5.661 MBytes/s, ETA 8m
Checks:             16967 / 16967, 100%
Transferred:            1 / 6, 17%
Elapsed time:      2m51.6s
Transferring:
 *        Bleach (2004)/Stagione 00 - Specials [480p x264] [JAPANESE]/Bleach (2004) - S00E03 - Bleach Memories of Nobody [WEBDL 480p x264] [JAPANESE].mkv:  1% /515.979M, 2.230M/s, 3m47s
 * Bleach (2004)/Stagione 00 - Specials [480p x264] [JAPANESE]/Bleach (2004) - S00E04 - Bleach The DiamondDust Rebellion [WEBDL 480p x264] [JAPANESE].mkv:  0% /514.706M, 893.340k/s, 9m45s
 *             Bleach (2004)/Stagione 00 - Specials [480p x264] [JAPANESE]/Bleach (2004) - S00E05 - Bleach Fade to Black [WEBDL 480p x264] [JAPANESE].mkv:  0% /515.553M, 544.709k/s, 16m3s
 *                Bleach (2004)/Stagione 00 - Specials [720p x264] [JAPANESE]/Bleach (2004) - S00E06 - Bleach Hell Verse [WEBDL 720p x264] [JAPANESE].mkv:  0% /930.747M, 702.004k/s, 22m33s

2021/08/18 02:03:05 INFO  : 
Transferred:   	  809.182M / 3.390 GBytes, 23%, 5.675 MBytes/s, ETA 7m49s
Checks:             16967 / 16967, 100%
Transferred:            1 / 6, 17%
Elapsed time:       3m1.6s
Transferring:
 *        Bleach (2004)/Stagione 00 - Specials [480p x264] [JAPANESE]/Bleach (2004) - S00E03 - Bleach Memories of Nobody [WEBDL 480p x264] [JAPANESE].mkv:  4% /515.979M, 1.655M/s, 4m57s
 * Bleach (2004)/Stagione 00 - Specials [480p x264] [JAPANESE]/Bleach (2004) - S00E04 - Bleach The DiamondDust Rebellion [WEBDL 480p x264] [JAPANESE].mkv:  3% /514.706M, 1.299M/s, 6m21s
 *             Bleach (2004)/Stagione 00 - Specials [480p x264] [JAPANESE]/Bleach (2004) - S00E05 - Bleach Fade to Black [WEBDL 480p x264] [JAPANESE].mkv:  3% /515.553M, 1.197M/s, 6m55s
 *                Bleach (2004)/Stagione 00 - Specials [720p x264] [JAPANESE]/Bleach (2004) - S00E06 - Bleach Hell Verse [WEBDL 720p x264] [JAPANESE].mkv:  1% /930.747M, 1.258M/s, 12m5s
  • at 2021/08/18 02:10:45 rclone started to delete files. Here we are in the second attempt I think. If not we have some problems
2021/08/18 02:10:45 INFO  : 
Transferred:   	    3.391G / 3.391 GBytes, 100%, 5.763 MBytes/s, ETA 0s
Checks:             16967 / 16971, 100%
Deleted:                4 (files), 0 (dirs)
Transferred:            6 / 6, 100%
Elapsed time:     10m41.6s
Checking:
 * Aggretsuko (2018)/Stagione 01 [1080p HEVC] [ITA+JPN]/Aggretsuko (2018) - S0…001 - Un giorno nella vita di Retsuko [NetflixHD 1080p HEVC] [ITA+JPN].mkv: checking
 * Black Clover (2017)/Stagione 00 - Specials [720p x264] [JAPANESE]/Black Clo…17) - S00E03 - Squishy! Black Clover - 01 [WEBDL 720p x264] [JAPANESE].mkv: checking
 * Mobile Suit Gundam 0083 Stardust Memory (1991)/Stagione 01 [480p x264] [ITA… - S01E07 - 007 - Con le splendenti fiamme blu [WEBDL 480p x264] [ITA].mkv: checking
 * Saint Seiya I Cavalieri dello zodiaco (2019)/Stagione 02 [1080p x264] [ITA+… S02E01 - 007 - I Cavalieri d'Argento [NetflixHD 1080p x264] [ITA+JPN].mkv: checking

2021/08/18 02:10:46 INFO  : Saint Seiya I Cavalieri dello zodiaco (2019)/Stagione 02 [1080p x264] [ITA+JPN]/Saint Seiya I Cavalieri dello zodiaco (2019) - S02E01 - 007 - I Cavalieri d'Argento [NetflixHD 1080p x264] [ITA+JPN].mkv: Moved (server-side)
2021/08/18 02:10:46 INFO  : Saint Seiya I Cavalieri dello zodiaco (2019)/Stagione 02 [1080p x264] [ITA+JPN]/Saint Seiya I Cavalieri dello zodiaco (2019) - S02E01 - 007 - I Cavalieri d'Argento [NetflixHD 1080p x264] [ITA+JPN].mkv: Moved into backup dir
2021/08/18 02:10:48 INFO  : Mobile Suit Gundam 0083 Stardust Memory (1991)/Stagione 01 [480p x264] [ITA]/Mobile Suit Gundam 0083 Stardust Memory (1991) - S01E07 - 007 - Con le splendenti fiamme blu [WEBDL 480p x264] [ITA].mkv: Moved (server-side)
2021/08/18 02:10:48 INFO  : Mobile Suit Gundam 0083 Stardust Memory (1991)/Stagione 01 [480p x264] [ITA]/Mobile Suit Gundam 0083 Stardust Memory (1991) - S01E07 - 007 - Con le splendenti fiamme blu [WEBDL 480p x264] [ITA].mkv: Moved into backup dir
2021/08/18 02:10:50 INFO  : Aggretsuko (2018)/Stagione 01 [1080p HEVC] [ITA+JPN]/Aggretsuko (2018) - S01E01 - 001 - Un giorno nella vita di Retsuko [NetflixHD 1080p HEVC] [ITA+JPN].mkv: Moved (server-side)
2021/08/18 02:10:50 INFO  : Aggretsuko (2018)/Stagione 01 [1080p HEVC] [ITA+JPN]/Aggretsuko (2018) - S01E01 - 001 - Un giorno nella vita di Retsuko [NetflixHD 1080p HEVC] [ITA+JPN].mkv: Moved into backup dir
2021/08/18 02:10:52 INFO  : Black Clover (2017)/Stagione 00 - Specials [720p x264] [JAPANESE]/Black Clover (2017) - S00E03 - Squishy! Black Clover - 01 [WEBDL 720p x264] [JAPANESE].mkv: Moved (server-side)
2021/08/18 02:10:52 INFO  : Black Clover (2017)/Stagione 00 - Specials [720p x264] [JAPANESE]/Black Clover (2017) - S00E03 - Squishy! Black Clover - 01 [WEBDL 720p x264] [JAPANESE].mkv: Moved into backup dir
2021/08/18 02:10:54 INFO  : Conan il ragazzo del futuro (1978)/Stagione 02 - JAP NO SUB [480p x264] [JAPANESE]/Conan il ragazzo del futuro (1978) - S02E05 - 041 - Hiirogane [WEBDL 480p x264] [JAPANESE].mkv: Moved (server-side)
2021/08/18 02:10:54 INFO  : Conan il ragazzo del futuro (1978)/Stagione 02 - JAP NO SUB [480p x264] [JAPANESE]/Conan il ragazzo del futuro (1978) - S02E05 - 041 - Hiirogane [WEBDL 480p x264] [JAPANESE].mkv: Moved into backup dir
2021/08/18 02:10:55 INFO  : 
Transferred:   	    3.391G / 3.391 GBytes, 100%, 5.669 MBytes/s, ETA 0s
Checks:             16977 / 16981, 100%
Deleted:                9 (files), 0 (dirs)
Renamed:                5
Transferred:            6 / 6, 100%
Elapsed time:     10m51.6s

talking about the last point. Rclone started to delete files because during the checks he didn't find the files in the directories. But this check when did started?
at 2021/08/18 02:16:17 we have the message

2021/08/18 02:16:17 ERROR : Attempt 2/5 succeeded

can you help me intepret this log?

I see, what possibility do you consider best:

or would you like to continue on your own?

What I want is understand the sync flow checking the log.
if in attempt 1 the message not deleting files as there were IO errors appear, so the attempt 2 has deleted the files.
Checking the log when the attempt 2 started? it's not logged.
Attempt 2 started at 2021/08/18 02:02:35?
or at 2021/08/18 02:02:50 with the message Encrypted drive 'Media:Anime': Making map for --track-renames?
at 2021/08/18 02:02:55 rclone started to trasfer files, so the drive was online. But after that the delete started at 2021/08/18 02:10:45.
why?

this is a log with 30s between attempts:

no deletion as when the attempt 2 starts the drive is back online

this is a log without sleep between attempts:

sudden after the attempt1 fail:

  • the attempt 2 started
  • the checks continued from when it was left but the drive is not online
  • rclone started to delete.

sorry for stressing you about my problems but I think you miss something from the log I posted..

We seem to have different views on the situation, let’s get in sync (no pun intended).

Do you agree with this:

Nope @Ole .
Check the logs:

2021/08/23 15:28:18 ERROR : Encrypted drive 'Media:Anime test': not deleting files as there were IO errors
2021/08/23 15:28:18 ERROR : Encrypted drive 'Media:Anime test': not deleting directories as there were IO errors
2021/08/23 15:28:18 ERROR : Attempt 1/5 failed with 8231 errors and: failed to open source object: The system cannot find the file specified.

or this log (oldest one)

2021/08/18 02:02:34 ERROR : Mobile Suit Gundam (1979)/Stagione 01 [720p HEVC] [ITA+JPN]/Mobile Suit Gundam (1979) - S01E15 - L'isola di Kukurs Doan [BDRip 720p HEVC] [ITA+JPN].mkv: Failed to copy: failed to open source object: The system cannot find the file specified.
2021/08/18 02:02:34 ERROR : Shaman King (2021)/Stagione 01 [1080p x264] [ITA+JPN]/Shaman King (2021) - S01E03 - 003 - Anna e Jun Tao [WEBDL 1080p x264] [ITA+JPN].mkv: Failed to copy: Post "https://www.googleapis.com/upload/drive/v3/files?alt=json&fields=id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks&supportsAllDrives=true&uploadType=resumable&upload_id=ADPycdt2qyuuRCPGWRIOPnNogybUiMeaRvJC80MGwabqxZVdbMu1SK0pz0V-PTpYa_vMJcJZ5NKmZJSAACTDR2D-WSt_pIiypw": read \\?\H:\Anime\Shaman King (2021)\Stagione 01 [1080p x264] [ITA+JPN]\Shaman King (2021) - S01E03 - 003 - Anna e Jun Tao [WEBDL 1080p x264] [ITA+JPN].mkv: A device which does not exist was specified.
2021/08/18 02:02:34 ERROR : Shaman King (2021)/Stagione 01 [1080p x264] [ITA+JPN]/Shaman King (2021) - S01E04 - 004 - Il posto preferito [WEBDL 1080p x264] [ITA+JPN].mkv: Failed to copy: Post "https://www.googleapis.com/upload/drive/v3/files?alt=json&fields=id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks&supportsAllDrives=true&uploadType=resumable&upload_id=ADPycdv01H3ipm1Suq18qBld2Bi5VEBRtrk5GdGGyQsdbift8MZNBa5143gG4XIhW5Lkgugma2plJy78oJH--yIas7D8YFqhEg": read \\?\H:\Anime\Shaman King (2021)\Stagione 01 [1080p x264] [ITA+JPN]\Shaman King (2021) - S01E04 - 004 - Il posto preferito [WEBDL 1080p x264] [ITA+JPN].mkv: A device which does not exist was specified.
2021/08/18 02:02:34 ERROR : Shaman King (2021)/Stagione 01 [1080p x264] [ITA+JPN]/Shaman King (2021) - S01E01 - 001 - Il ragazzo che balla con i fantasmi [WEBDL 1080p x264] [ITA+JPN].mkv: Failed to copy: Post "https://www.googleapis.com/upload/drive/v3/files?alt=json&fields=id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks&supportsAllDrives=true&uploadType=resumable&upload_id=ADPycdu8aAIxGYzVWcrw-wFtjkE9IH532qGMLhdr0pQyKSA0raWg67JSubBJcflcQ3jRo3I_pI_zDkyYUzvJYpky7OEAfetoBw": read \\?\H:\Anime\Shaman King (2021)\Stagione 01 [1080p x264] [ITA+JPN]\Shaman King (2021) - S01E01 - 001 - Il ragazzo che balla con i fantasmi [WEBDL 1080p x264] [ITA+JPN].mkv: A device which does not exist was specified.
2021/08/18 02:02:34 ERROR : Encrypted drive 'Media:Anime': not deleting files as there were IO errors
2021/08/18 02:02:34 ERROR : Encrypted drive 'Media:Anime': not deleting directories as there were IO errors
2021/08/18 02:02:34 ERROR : Attempt 1/5 failed with 46 errors and: Post "https://www.googleapis.com/upload/drive/v3/files?alt=json&fields=id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks&supportsAllDrives=true&uploadType=resumable&upload_id=ADPycdu8aAIxGYzVWcrw-wFtjkE9IH532qGMLhdr0pQyKSA0raWg67JSubBJcflcQ3jRo3I_pI_zDkyYUzvJYpky7OEAfetoBw": read \\?\H:\Anime\Shaman King (2021)\Stagione 01 [1080p x264] [ITA+JPN]\Shaman King (2021) - S01E01 - 001 - Il ragazzo che balla con i fantasmi [WEBDL 1080p x264] [ITA+JPN].mkv: A device which does not exist was specified.

Rclone intercept that there's an I/O error. The drivepool software too se an I/O error. So the first attempt doesn't delete things (aside the actual deleted ones because the --delete-during option).
After the I/O error Drivepool gives no error reading the folder structure because he work even without some disks.
Lets take a pratical case:

  • Disk X: with Anime folder in the pool
  • Disk Y: with Anime folder in the pool

Drivepool software let you see the content of x:\poolpartxxxxx\Anime and y:\poolpartxxxxx\Anime merged into a unique H:\Anime.
If the drive X: goes offline or something happen on the x:\poolpartxxxxx:

  • An I/O error occurs (check rclone logs)
  • Drivepool is configured to send me a mail
  • Drivepool stabilize itself showing the H:\Anime with the content he have online. In our case only the y:\poolpartxxxxx\Anime

it's more clear like this?

at the end of the day, you shouldn't be using sync automatically if the file system can be in an inconsistent state. You should be using copy automatically and then run sync periodically by hand (when you can also verify by hand that everything is in a good state) to clean up the remote.

1 Like

the file system isn't in an inconsistent state..
but can happen sometimes that the usb box disconnect and reconnect.
sometimes it happen twice a day..sometimes once a week..

You have broken hardware.

Rather than trying to continue to work around it, it's better to fix the root cause.

1 Like

a usb drive disconnecting is to me a file system that gets into an inconsistent state.

And if that would happen to me, I'd just use copy, and then use sync to clean things up when I can verify by hand that things are good as I said.

OK, then let’s take another step back to get in sync.

Do you agree that the provoked disconnect happening in Anime test_20210821_ 04822.log is representative for the daily/weekly disconnects?

All I need right now is a simple yup or nope to find the point in the thread where we got out of sync.

@spotter and @Animosity022 I fully agree, thanks for the input.

yep. In this test I manually disconnect the drive and as you see the attempt 1 didn't delete anything

I can agree about the maybe not 100% 7/7 24/24 working hardware..but guys, I'm a developer too (cobol cics db2 some java and c#).. why we handle the I/O errors or some absurd exeptions? because these can happens and rclone handle the I/O errors well.
It's the drivepool software that after an I/O error try to give you a consinstent file system with the actual online drives..

and this is exactly the catch!

By doing so the DrivePool hides the error/inconsistency of your filesystem to the OS. rclone therefore has no way to know that it is receiving incomplete results from a DIR command - everything looks fine, but it isn't.

rclone sync will therefore occasionally (every 2-3 month) get a wrong picture of your local filesystem and consequently start deleting (almost) all of the files in your target.

If you agree to this then we are ready to move on, otherwise let’s get in back in sync from the right place in the thread...

Do you agree that this is a correct extract of the log you just confirmed above:

but always on the attempt number 2.
Are a fact that after the I/O error, rclone end the attempt without deleting files? we agree with this assumption?

yes, I agree that's the correct extract.
From this piece of log is very clear that the delete occurs always on the attempt number 2
do you agree?

the timeline of the logs is:

  1. rclone sync starts the attempt number 1
  2. a drive goes offline
  3. rclone catch the I/O error
  4. rclone end the attempt number 1 with error without delete nothing
  5. rclone start the attempt number 2 with the drivepool giving the Anime folder in the status based on the actual online drives. If the drive is not online, the anime folder is empty and the sync start delete files

what I see in this list is that before point 4, no problem happens. The problems starts with point 5. Are we in sync?

I fully agree, do you agree that the two sync attempts can be seen like two independent syncs (with --retries=1) started in a row like this:

yep.
This is the reason why I suggested to use the --retries 1 for the Anime folder sync once a day and add a delay batch for the next folder:
example:

Today the sync gives error. No other try.
Tomorrow the sync is ok because all the drives are always on and the drivepool mail me when the drives goes down and after when return up. here's the incriminated day:

as you see the downtime was less than 1 second but for the software the took some times to restore the Anime folder to be fully consistent.

this is the reason why I suggested this:

are we in sync?

Good!

No, because I see a glitch in your reasoning. I will explain it when we are in better sync.

Do you agree to the following based on the log you just confirmed above:

the sync B you're assuming are for the same folder as sync A?
or another folder?

Sync B is this command:

where --retries=1. Just started 39 seconds later while a part of the DrivePool is offline.

I wouldn't call it an assumption, I see it as an interpretation of the log where a sync with --retries=5 is viewed as 5 syncs with --retries=1.

Do you agree that the log can be read like that?