Syncing a filesystem to GDrive with active log

Help and Support
1

Hello,

We are trying to sync a filesystem to Google Drive using rclone 1.46. The problem I am having is that I have specified a log file using "--log-file=" to a location on the filesystem being synced. Even though I have used an "--exclude" to try to prevent that log file being synced, rclone still attempts to sync it, and as such exits with an error.

The command we are using is:

rclone --log-file="/pool/log/$(date +%Y-%m-%d_%H:%M)-remote-backup-cea-archive.log" -vP --fast-list --max-backlog 500000 --exclude 'log/$(date +%Y-%m-%d_%H:%M)-remote-backup-cea-archive.log' --delete-excluded sync /pool cea-archive:/Backup/server/pool

I have tried variations on the '--exclude' to try and match the log file, but rclone still tries to sync it.

Here is the error it reports in the log:

2019/05/20 02:04:28 ERROR : log/2019-05-20_01:22-remote-backup-cea-archive.log: Failed to copy: can't copy - source file is being updated (size changed from 58 to 8853)
2019/05/20 02:20:19 ERROR : Google drive root 'Backup/server/pool': not deleting files as there were IO errors
2019/05/20 02:20:19 ERROR : Google drive root 'Backup/server/pool': not deleting directories as there were IO errors
2019/05/20 02:20:19 ERROR : Attempt 1/3 failed with 1 errors and: can't copy - source file is being updated (size changed from 58 to 8853)

I'm sure it's something simple but I've tried quite a few variations and still can't get rclone to ignore this file in the sync.

Any help with this would be greatly appreciated!

Tristan.

2

I think there is some relative stuff going on.

Try:

--exclude $(date +%Y-%m-%d_%H:%M)-remote-backup-cea-archive.log'

here is what I did to test:

[felix@gemini test]$ rclone copy /home/felix/test/ --log-file=/home/felix/test/test.log --exclude=test.log -vv GD:
[felix@gemini test]$ less test.log
[felix@gemini test]$ cat test.log
2019/05/20 22:02:05 DEBUG : rclone: Version "v1.47.0" starting with parameters ["rclone" "copy" "/home/felix/test/" "--log-file=/home/felix/test/test.log" "--exclude=test.log" "-vv" "GD:"]
2019/05/20 22:02:05 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2019/05/20 22:02:05 DEBUG : test.log: Excluded
2019/05/20 22:02:05 INFO  : Google drive root '': Waiting for checks to finish
2019/05/20 22:02:05 DEBUG : hosts: Size and modification time the same (differ by -104.639µs, within tolerance 1ms)
2019/05/20 22:02:05 DEBUG : hosts: Unchanged skipping
2019/05/20 22:02:05 INFO  : Google drive root '': Waiting for transfers to finish
2019/05/20 22:02:05 INFO  :
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 1 / 1, 100%
Transferred:            0 / 0, -
Elapsed time:       400ms

2019/05/20 22:02:05 DEBUG : 4 go routines active
2019/05/20 22:02:05 DEBUG : rclone: Version "v1.47.0" finishing with parameters ["rclone" "copy" "/home/felix/test/" "--log-file=/home/felix/test/test.log" "--exclude=test.log" "-vv" "GD:"]
3

Hi,

There appears to be a stray inverted comma at the end of your example? Is there meant to be one at the beginning as well? I've already tried that but will try without inverted commas.

4

It seems to work if I omit the inverted commas. I made a modified version of my script to test. Testing now on the main script. Thanks for pointing me in the right direction.

5

What was the final command line? I wasn't sure how you were passing the date.

6

So passing this option worked:

--exclude $(date +%Y-%m-%d_%H:%M)-remote-backup-cea-archive.log

And this option did not work:

--exclude '$(date +%Y-%m-%d_%H:%M)-remote-backup-cea-archive.log'

The difference being the quotes.

Should --exclude parameters not have quotes?

7

You can test with --dump filters and see if something is different.

8

It is the shell (bash) that interprets the quotes, not rclone.

If you put things in 'single quotes' bash will not interpret anything in them, so the $(date ...) won't be interpreted. However if you use "double quotes" stuff will be interpreted. If the argument has no spaces then you can use no quotes like you have done.

9

Right, thanks that makes sense. I've removed all the single quotes now from my excludes. All seems to be working well. Thanks for all the help.

1 Like
10

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.