Include file extention and exclude directory

Help and Support
1

What is the problem you are having with rclone?

the local machine stops responding on sync some files
i want to include file extension and ignore or skip corrupted files which crashing the hosted files server

what i'm tried to include the files extensions and exclude the directory it show error not to have both same line or command and also has same result "crashing the hosted files server"

What is your rclone version (output from rclone version)

rclone v1.52.2

Which OS you are using and how many bits (eg Windows 7, 64 bit)

- os/arch: windows7 / amd64

Which cloud storage system are you using? (eg Google Drive)

digitalocean spaces

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync z:\fileserver\data\  DigitalOcean:/backup/weekday/Data  --include *.{xls,xlsx,jpg,pdf} -P

The rclone config contents with secrets removed.

[DigitalOcean]
type = s3
env_auth = false
access_key_id = mykey
secret_access_key = access_key
endpoint = xxx.digitaloceanspaces.com
acl = private

A log from the command with the -vv flag

Paste  log here
2

hello and welcome the forum,

filtering can be confusing.
when i test a filter, i use rclone ls, once that is working then rclone sync

can you post the command with a debug log?

3

@asdffdsa Thanks for your response

2021/01/30 22:30:34 DEBUG : username/2019/Copy of 2019.xls: Reopening on read failure after 7335936 bytes: retry 1/10: read \\?\Z:\Fin\username\2019\Copy of 2019.xls: The specified network name is no longer available.

which cause of crashing the hosted file server

now trying re-run it and feed you back with the results without any other backup scripts

4

not sure what is going on, all these corrupted files and crashing of servers.
at this point, need a debug log, not just a one line snippet.

specified network name is no longer available

z: drive is local....

5 
2021/01/31 02:38:09 DEBUG : user1/main/com 11 day 20-12-2020/helper.pdf: Reopening on read failure after 9433088 bytes: retry 1/10: can't read status of source file while transferring: GetFileInformationByHandle \\?\Z:\fin\user1\main\com 11 day 20-12-2020\helper.pdf: The specified network name is no longer available.
2021/01/31 02:38:09 DEBUG : TamerMohamed/New folder (2)/file_1).jpg: Reopening on read failure after 0 bytes: retry 1/10: can't read status of source file while transferring: GetFileInformationByHandle \\?\Z:\fin\TamerMohamed\New folder (2)\file_1).jpg: The specified network name is no longer available.
2021/01/31 02:39:48 ERROR : TamerMohamed/New folder (2)/file_1).jpg: Failed to copy: s3 upload: 400 Bad Request: <?xml version="1.0" encoding="UTF-8"?><Error><Code>RequestTimeout</Code><BucketName>MyBucketName</BucketName><RequestId>tx0000000000000290e6b2c-006015fbf1-2e1d3b4-xxxxxxxx</RequestId><HostId>2e1d3b4-xxxxxxxx-fra1-zg02</HostId></Error>
2021/01/31 02:39:48 ERROR : user1/main/com 11 day 20-12-2020/helper.pdf: Failed to copy: s3 upload: 400 Bad Request: <?xml version="1.0" encoding="UTF-8"?><Error><Code>RequestTimeout</Code><BucketName>MyBucketName</BucketName><RequestId>tx0000000000000290319a7-006015fbf1-2e1cef0-xxxxxxxx</RequestId><HostId>2e1cef0-xxxxxxxx-fra1-zg02</HostId></Error>
2021/01/31 02:44:43 DEBUG : FaredAhady/supl/Copy of 2019.xls: Reopening on read failure after 7335936 bytes: retry 1/10: read \\?\Z:\fin\FaredAhady\supl\Copy of 2019.xls: The specified network name is no longer available.
2021/01/31 02:44:44 DEBUG : TamerMohamed/New folder (2)/file  27 - 10 - 2020.pdf: Reopening on read failure after 4096 bytes: retry 1/10: can't read status of source file while transferring: GetFileInformationByHandle \\?\Z:\fin\TamerMohamed\New folder (2)\file  27 - 10 - 2020.pdf: The specified network name is no longer available.

z: is network mapped drive "as local HDD "
yes sir i know all corrupted files even i can't access them on servers they cause crash the server , over days a new file corrupted , thats why i need solution to backup all files without access those corrupted files

example :
file1 , file2 , file3"corrupted " , file4 and file5
while rclone sync Z:\Fin DigitalOcean:/user/weekday/UsersData --include *.{xls,xlsx,jpg,pdf} -P -vv --log-file=log_Sunday.txt
file1 transferred ok
file2 transferred ok
file3"corrupted " -> crash the server -> cause "specified network name is no longer available "
file4 "specified network name is no longer available "
file5"specified network name is no longer available "
hope explained fine and thanks in advanced

6

if you do not know in advance which file is corrupted, how do you plan to create a filter for it?

what does that mean, that the server reboots without warning?

7

sadly yes, sir it does
i would like to thank you for your help , support and let you know a temporary solution i was trying for 2 days i started by copying the files on target server "which crashed while trying access corrupted files " to the local machine '' which have rclone on " with robocopy using this command
robocopy Z:\Fin D:\PreCloudFolder /e /XF *.avi *.exe /R:0 /W:0
i think the helper here is no retry and no wait time , which helped to skip corrupted files
then starts my rclone command

8

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.