Can't copy/move from smb share to remote

Andre_Luis · September 6, 2020, 6:39pm

What is the problem you are having with rclone?

Can't copy from smb share to google drive, but can from local directory.
I'm trying to move an entire directory tree:

output
- 2020
  - 09
    - 06

In the last folder, it contains a 4mb mp4 file. However, it seems it's trying to upload a lot of things.

What is your rclone version (output from `rclone version`)

rclone v1.53.0

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Arch Linux and MacOS. Both 64 bits

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

 rclone copy -vv --progress --stats-one-line /home/user/server/cameras/camera1/record/output/* camera_source:/cameras/camera1

camera1 is the smb share mounted

The rclone config contents with secrets removed.

[camera_source]
type = drive
scope = drive
toke = xxx
team_drive = xxx
root_folder_id =

A log from the command with the `-vv` flag

Even though it says rate limit exceed, I can still copy the files from local (not smb) to remote.

2020/09/06 15:30:04 DEBUG : rclone: Version "v1.53.0" starting with parameters ["rclone" "copy" "-vv" "--progress" "--stats-one-line" "/home/user/server/cameras/camera1/record/output/2020" "camera_source:/cameras/camera1"]
2020/09/06 15:30:04 DEBUG : Creating backend with remote "/home/user/server/cameras/camera1/record/output/2020"
2020/09/06 15:30:04 DEBUG : Using config file from "/home/user/server/cameras/.rclone.conf"
2020/09/06 15:30:04 DEBUG : Creating backend with remote "camera_source:/cameras/camera1"
2020/09/06 15:30:04 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2020/09/06 15:30:04 DEBUG : pacer: Rate limited, increasing sleep to 1.413563794s
2020/09/06 15:30:04 DEBUG : pacer: Reducing sleep to 0s
2020/09/06 15:30:06 DEBUG : fs cache: renaming cache item "camera_source:/cameras/camera1" to be canonical "camera_source:cameras/camera1"
2020-09-06 15:30:07 DEBUG : Google drive root 'cameras/camera1': Waiting for checks to finish
2020-09-06 15:30:07 DEBUG : Google drive root 'cameras/camera1': Waiting for transfers to finish
2020-09-06 15:30:08 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 1/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:08 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 2/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:08 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 3/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:08 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 4/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:08 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 5/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:08 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 6/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:08 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 7/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:08 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 8/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:08 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 9/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:08 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 10/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:08 DEBUG : 09/06/14-16-29.mp4: Reopen failed after 4190208 bytes read: failed to reopen: too many retries
2020-09-06 15:30:09 ERROR : 09/06/14-16-29.mp4: Failed to copy: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:09 INFO  : There was nothing to transfer
2020-09-06 15:30:09 ERROR : Attempt 1/3 failed with 1 errors and: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:10 DEBUG : Google drive root 'cameras/camera1': Waiting for checks to finish
2020-09-06 15:30:10 DEBUG : Google drive root 'cameras/camera1': Waiting for transfers to finish
2020-09-06 15:30:10 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 1/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:10 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 2/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:10 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 3/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:10 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 4/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:10 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 5/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:10 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 6/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:10 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 7/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:10 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 8/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:10 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 9/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:10 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 10/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:10 DEBUG : 09/06/14-16-29.mp4: Reopen failed after 4190208 bytes read: failed to reopen: too many retries
2020-09-06 15:30:11 ERROR : 09/06/14-16-29.mp4: Failed to copy: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:11 INFO  : There was nothing to transfer
2020-09-06 15:30:11 ERROR : Attempt 2/3 failed with 1 errors and: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:12 DEBUG : Google drive root 'cameras/camera1': Waiting for checks to finish
2020-09-06 15:30:12 DEBUG : Google drive root 'cameras/camera1': Waiting for transfers to finish
2020-09-06 15:30:13 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 1/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:13 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 2/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:13 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 3/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:13 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 4/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:13 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 5/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:13 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 6/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:13 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 7/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:13 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 8/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:13 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 9/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:13 DEBUG : 09/06/14-16-29.mp4: Reopening on read failure after 4190208 bytes: retry 10/10: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:13 DEBUG : 09/06/14-16-29.mp4: Reopen failed after 4190208 bytes read: failed to reopen: too many retries
2020-09-06 15:30:13 ERROR : 09/06/14-16-29.mp4: Failed to copy: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
2020-09-06 15:30:13 INFO  : There was nothing to transfer
2020-09-06 15:30:13 ERROR : Attempt 3/3 failed with 1 errors and: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error
11.988M / 11.988 MBytes, 100%, 2.957 MBytes/s, ETA 0s
2020/09/06 15:30:13 INFO  :    11.988M / 11.988 MBytes, 100%, 2.957 MBytes/s, ETA 0s

2020/09/06 15:30:13 DEBUG : 4 go routines active
2020/09/06 15:30:13 Failed to copy: read /home/user/server/cameras/camera1/record/output/2020/09/06/14-16-29.mp4: input/output error

ookla-ariel-ride · September 29, 2020, 6:58pm

This looks like a read permissions issue. How is your SMB share being mounted? What permissions do you have on the mounted share? Can you access the .mp4 from the user you are attempting the rclone copy with? What happens if you copy the .mp4 from the SMB share to a local fileystem using the same user account?

system · November 29, 2020, 2:58pm

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.