I'm working on writing a script that will interact with the files that have been successfully copied or skipped using Rclone. To do this, I'm reading the log-file after rclone has run. Does anyone know what the output line looks like in each of those cases? Or can you point me to the file in github that I can read the template for the logs?
What is your rclone version (output from rclone version)
rclone v1.56.0
Which cloud storage system are you using? (eg Google Drive)
AWS S3
The command you were trying to run (eg rclone copy /tmp remote:tmp)
at first, i read the log files, found the text entries i want.
then i created a set of regular expressions.
the script will read each line, try to match regular expression.
if match then do something...
with rclone, for a file copied, there are multiple text to match
for example, 2021/09/07 08:48:06 INFO : xxx.xxx: Copied (new) 2021/09/07 08:48:06 INFO : xxx.xxx: Copied (server-side copy)
note i am not an expert at reg ex and perhaps these two rules can be combined into one.
this is works well for me. ^.* INFO :.*: Copied .new.$ ^.* INFO :.*: Copied .server-side copy.$
it is possible to use one rule, but might cause a false positive ^.* INFO.*:.*: Copied
as for skipped files, 2021/09/07 08:48:06 DEBUG : xxx.xxx: Unchanged skipping
Hi,
Thank you for your response! The regex is very helpful.
I looked through my log file, and for skipped files I found:
2021/09/02 16:45:30 DEBUG : xxx.txt: Destination mod time is within 1ns of source and files identical, skipping
I'm just still wondering if there is a way to look at all the possible combinations that could be outputted to the log file. I want to make the script as robust as possible and I want to account for all the possible ways the log file could say that it skipped a file or that a file was successful.
actually, i had forgotten i was already doing that.
i have a python script that takes a rclone debug log, strips out stuff and emails to me a rclone.short.log ^.* DEBUG :.*Waiting for (checks|transfers|deletions) to finish$