We are using rclone to upload to both an SFTP server and an S3 bucket.
We are executing the rclone command from within PHP. The upload is super reliable and fast and handles all the deduplication, filtering, etc. Love it.
But I am trying to provide some feedback to my users via the web interface to let them know the progress and current status of the upload.
I know there's a "progress bar" when running via command line. But not an option here.
I don't need the progress to be fully real-time. A refresh of the window is fine.
I am already outputting the logs for each upload as a json file. But I think it would be super inefficient to scan that json log file upon refresh and count the objects uploaded and return that every time. So unless I'm missing something or a flag I can turn on to have a more efficient way to do this in closer real time.
When the copy is done we are able to extract the data we need from the json to error check, time, etc
our copy command looks like this:
rclone copy --files-from $directoryName/".$FavoritesHeaderID."-filter.txt ". $imagepath ." events:events-lowres/ --size-only -P --checkers 64 --transfers 12 -v --log-file=$directoryName/$FavoritesHeaderID.json --use-json-log