Virus checking - Possible to skip files marked as infected in Sharepoint?

Feature
1

I am using my own Graph API keys to synchronise files from Teams/Sharepoint.

In the case where Sharepoint has marked a file as infected, Teams client will not allow you to open the file, and OneDrive for Business (if synced with that Sharepoint) will not allow you to download the file locally to your PC for opening.

However Rclone still can see these files and will copy them to the target location (see screenshot). Is there any way to detect this file status through Graph API and disable copying of potentially infected files?

Thanks in advance.

image
image1142×676 41.9 KB

2

hello and welcome to the forum,

rclone does not have a way to detect if a file has been infected.
perhaps sharepoint sends a specific message to rclone.

when you posted, there was a template of questions, asking for information.
can you post that so we can help you?

3

Hi asdffdsa,

Thanks for the reply. I didn't think it was applicable as it's not a problem as such, the remote is set up and works fine, just wondering about whether rclone can detect and skip these "infected" files.

Here it is anyway:

  1. The full command you’re attempting to use.

rclone sync remote:General/AWS_Feed "xxx/AWS_Feed/" -vv --dry-run

  1. A logfile of rclone’s output with personal information removed.

2021/04/29 14:19:28 DEBUG : Using config file from "/home/user/.config/rclone/rclone.conf"
2021/04/29 14:19:28 DEBUG : rclone: Version "v1.55.0" starting with parameters ["rclone" "sync" "remote:General/AWS_Feed" "xxx/AWS_Feed/" "-vv" "--dry-run"]
2021/04/29 14:19:28 DEBUG : Creating backend with remote "remote:General/AWS_Feed"
2021/04/29 14:19:38 DEBUG : Creating backend with remote "xxx/AWS_Feed/"
2021/04/29 14:19:41 NOTICE: eicar.txt: Skipped copy as --dry-run is set (size 68)
2021/04/29 14:19:41 DEBUG : Local file system at xxx/AWS_Feed/: Waiting for checks to finish
2021/04/29 14:19:41 DEBUG : exampledata1.xlsx: Size and modification time the same (differ by 0s, within tolerance 1s)
2021/04/29 14:19:41 DEBUG : test.docx: Size and modification time the same (differ by 0s, within tolerance 1s)
2021/04/29 14:19:41 DEBUG : exampledata1.xlsx: Unchanged skipping
2021/04/29 14:19:41 DEBUG : test.docx: Unchanged skipping
2021/04/29 14:19:41 DEBUG : Local file system at xxx/AWS_Feed/: Waiting for transfers to finish
2021/04/29 14:19:41 DEBUG : Waiting for deletions to finish
2021/04/29 14:19:41 NOTICE:
Transferred: 68 / 68 Bytes, 100%, 267.982 kBytes/s, ETA 0s
Checks: 2 / 2, 100%
Transferred: 1 / 1, 100%
Elapsed time: 13.5s

  1. The rclone config you’re using.

[remote]
type = onedrive
client_id = xxx
client_secret = xxx
region = global
token = {"access_token":"xxx","expiry":"xxx"}
drive_id = xxx
drive_type = documentLibrary

  1. What version of rclone you’re using.

rclone v1.55.0

4

need to run the command without --dry-run and see what message sharepoint sends to rclone.

5

I was using dry-run to avoid annoying my admins if it actually did download. But yes I've given it a go now. Good news - it is recognised as malware (presume that's from Sharepoint) and is classed by Rclone as an error. Thank you!

2021/04/29 14:50:26 DEBUG : Using config file from "/home/user/.config/rclone/rclone.conf"
2021/04/29 14:50:26 DEBUG : rclone: Version "v1.55.0" starting with parameters ["rclone" "sync" "remote:General/AWS_Feed" "xxx/AWS_Feed/" "-vv" "--retries" "1"]
2021/04/29 14:50:26 DEBUG : Creating backend with remote "remote:General/AWS_Feed"
2021/04/29 14:50:35 DEBUG : Creating backend with remote "xxx/AWS_Feed/"
2021/04/29 14:50:39 DEBUG : test.docx: Size and modification time the same (differ by 0s, within tolerance 1s)
2021/04/29 14:50:39 DEBUG : test.docx: Unchanged skipping
2021/04/29 14:50:39 DEBUG : exampledata1.xlsx: Size and modification time the same (differ by 0s, within tolerance 1s)
2021/04/29 14:50:39 DEBUG : exampledata1.xlsx: Unchanged skipping
2021/04/29 14:50:39 DEBUG : Local file system at xxx/AWS_Feed/: Waiting for checks to finish
2021/04/29 14:50:39 DEBUG : Local file system at xxx/AWS_Feed/: Waiting for transfers to finish
2021/04/29 14:50:47 ERROR : eicar.txt: Failed to copy: failed to open source object: malwareDetected: Malware detected
2021/04/29 14:50:47 ERROR : Local file system at xxx/AWS_Feed/: not deleting files as there were IO errors
2021/04/29 14:50:47 ERROR : Local file system at xxx/AWS_Feed/: not deleting directories as there were IO errors
2021/04/29 14:50:47 ERROR : Attempt 1/1 failed with 1 errors and: failed to open source object: malwareDetected: Malware detected
2021/04/29 14:50:47 INFO :
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors: 1 (retrying may help)
Checks: 2 / 2, 100%
Elapsed time: 21.2s

2021/04/29 14:50:47 DEBUG : 5 go routines active
2021/04/29 14:50:47 Failed to sync: failed to open source object: malwareDetected: Malware detected

6

Looking at the info returned in the API

https://docs.microsoft.com/en-us/onedrive/developer/rest-api/resources/driveitem?view=odsp-graph-online

There is a malware field which I guess might be set for your infected file.

Assuming it is, it would be possible to make a flag for onedrive to make rclone skip infected files.

7

Hi Nick,

I've done a --dump responses. Not quite sure what they all but this is from the big one which lists all the files in the remote. This is what it looks like for the infected file. For other files the malware field is not listed at all.

..."name":"eicar.txt","webUrl":"https://xxx.sharepoint.com/sites/xxx/eicar.txt","cTag":""c:{xxx},3"","malware":{},"size":68,"createdBy"...

Would be great if this could be used to skip these files in a future version, as otherwise Rclone thinks there's an error and keeps retrying (up to the retry limit of course).

8

might be better to have a NOTICE in the log about the malware.
as i have a script that scans the logs, that would be useful info.

9

Yeah that's fine too, just some mechanism to stop it being considered as an "error" and retrying :slight_smile:

10

Great.

We could do something this this error which might be easier...

2021/04/29 14:50:47 ERROR : eicar.txt: Failed to copy: failed to open source object: malwareDetected: Malware detected

What would you want it to do? This would be my preference

Skip the file, make an error (so the exit status of rclone is non-zero) but don't do a retry.

Or something else?

11

Yeah that would be great thanks :slight_smile:

12

Can you open a new issue on Github about this? Maybe you'd like to help develop the feature? If so I'm happy to walk you through it.

13

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.