incomplete and corrupt files when aborting a copy operation to the local server

Suspected Bug
1

Hi.
First of all, I apologize if I did not understand the publication rules, I do not speak English. This message is translated with google.
I have a problem with any command or flag related to rclone copy / rclone sync when importing data from Google Drive to the local server where rclone is running from. What happens is that if for some reason a long process with large files must be aborted, the files that were being transferred at that moment appear incomplete and corrupt. When restarting the task, rclone automatically continues through the next files and does not correct incomplete files. This is tested with the TrueNas "Cloud Sync Tasks" function with which I want to execute the task to bring files to the server from Google Drive. I have also tried it from the Debian terminal using different commands like rclone copy / sync without flags or with the -u flags among others described on the web page. Nothing works. Everything works fine, files are copied the way I want until the task is interrupted. Is there a way for the system to replace or recopy the files that were truncated from the previous session? Thank you all!

2

hello and welcome to the forum,

should be able to run the rclone sync/copy again.
if a source file is missing in the dest, rclone will copy it.

fwiw, for testing,

  • use the terminal, not truenas tool.
  • do not use -u

please post the output of

  • rclone version
  • rclone listremotes
  • from the terminal, run the command with -vv and post the full output.
3

Hi. thanks for the reply!
rclone version -vv:

Welcome to TrueNAS
nas# rclone version -vv
2023/02/09 15:45:13 DEBUG : rclone: Version "v1.57.0-DEV" starting with parameters ["rclone" "version" "-vv"]
rclone v1.57.0-DEV

  • os/version: freebsd 13.1-release-p2 (64 bit)
  • os/kernel: 13.1-release-p2 (amd64)
  • os/type: freebsd

rclone listremotes -vv:

nas# rclone listremotes -vv
2023/02/09 15:54:41 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2023/02/09 15:54:42 DEBUG : rclone: Version "v1.57.0-DEV" finishing with parameters ["rclone" "listremotes" "-vv"]
nas#

nas# rclone config
2023/02/09 15:59:51 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config

It seems that in the case of TrueNas, rclone is modified. Anyway a few days ago I was testing a classic installation of rclone on a clean Debian where I could run and configure in the traditional way from the terminal, but had exactly the same problem as in the case of TrueNas. Everything is copied or transferred ok, except those files that were truncated or aborted. Important: This only happens when transferring files from the remote to the local system. Conversely, in the case of Google Drive, the truncated files are not displayed and are resubmitted on the next execution of the same task. thank you!

4

I did not clarify that in most cases or those that it detects are .mp4 .mov .wav files.
truncated files cannot be played, it is a media server for video editing with google drive backup. Problem are only when bringing files from google, not the other way around. Google removes the incomplete files and they are no longer there, so rclone re-uploads them without problems when restarting the task from the terminal in Debian by running "rclone copy" or from the rclone-based TrueNas application (same thing happens either way). ). The problem is that rclone doesn't delete the truncated files when aborting the task, leaving them incomplete, and restarting the command skips them because the files appear to already be at the destination.

The truncated files show the same modification date as the source, I guess that's why rclone doesn't transfer them back. thank you!

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.