Hi Folks,
Hope everyone is doing well!
Thanks to everyone in Rclone for their amazing work and effort, huge fan, and thanks to anyone willing to help, I really appreciate it, means a lot to me!
What is the problem you are having with rclone?
I am trying to use rclone to first sync and then check several source and destination directories.
The destinations are different media types:
- HDD 7200RPM
- HDD 5400RPM
- SSD
Initially I was using rclone to sync around 2TB of data to the HDD 7200RPM.
This worked ok, albeit a little slow - took around 24 hours.
Then I tried to sync the same data, around 2TB to the HDD 5400RPM and this was ridiculously slow.
The sync command did not finish even after 6 days, after which I manually stopped it.
I tried to speed up the rclone sync command and ended up writing the following script, to try and use and tweak parameters for the different types of media.
Anyway, before I was able to try and speed up optimizations I realized I hadn't tried syncing to the SSD.
This is where the problem manifested.
When I tried to run the rclone sync command to the SSD destination the Ubuntu machine froze up after some time and became absolutely non responsive. The only way was to do a HARD shut down and start it again.
I tweaked the transfers from 8 to 4 (--transfers=4) and this seemed to work for the sync command.
Then the same problem manifested for the rclone check command.
However lowering the checkers even to 1 did not help (--checkers=1).
I played with all parameter/flags I could find which seemed relevant in the rclone documentation, but could not solve the problem.
It looks like when it's running against a faster media such as an SSD it is going over some kind of a limit and it's crashing the Ubuntu machine.
I thought it could be an out of memory issue, however all indicators of the Ubuntu Server machine seem fine.
The script I use to run the rclone sync and rclone check commands:
#!/bin/bash
# Source directories
SRC_DIRS=(
"localsrc1:/mnt/tns-google-drive-smb"
"localsrc2:/mnt/tns-maindatads-smb"
"localsrc3:/mnt/tns-app-configs-smb"
"localsrc4:/mnt/tns-proxbkps-ds-1"
)
# Destination directories for single storage units SSD and NVME
DEST_DIRS_SINGLE_STORE_SSD_NVME=(
"localdest4:/mnt/sd25-4tb-bkp4-mnt4"
)
# Destination directories for single storage units HDD 7200RPM
DEST_DIRS_SINGLE_STORE_7200RPM=(
"localdest1:/mnt/hd35-6tb-bkp1-mnt1"
)
# Destination directories for multiple storage units HDD 5400RPM
DEST_DIRS_MULTI_STORE_5400RPM=(
"localdest3:/mnt/hd25-500-bkp1-mnt1"
"localdest2:/mnt/hd25-2tb-bkp1-mnt1"
)
TEMP_UPLOAD_PATH="/home/usertemp/rclone_cache_temp/tmp_upload"
CHUNK_PATH="/home/usertemp/rclone_cache_temp/chunks"
# Transfers and checkers configuration
declare -A TRANSFERS_CONFIG
declare -A CHECKERS_CONFIG
# Populate TRANSFERS_CONFIG and CHECKERS_CONFIG for single storage units SSD and NVME
for dest in "${DEST_DIRS_SINGLE_STORE_SSD_NVME[@]}"; do
TRANSFERS_CONFIG["$dest"]=4
CHECKERS_CONFIG["$dest"]=4
done
# Populate TRANSFERS_CONFIG and CHECKERS_CONFIG for single storage units HDD 7200RPM
for dest in "${DEST_DIRS_SINGLE_STORE_7200RPM[@]}"; do
TRANSFERS_CONFIG["$dest"]=8
CHECKERS_CONFIG["$dest"]=8
done
# Populate TRANSFERS_CONFIG and CHECKERS_CONFIG for multi storage units HDD 5400RPM
for dest in "${DEST_DIRS_MULTI_STORE_5400RPM[@]}"; do
TRANSFERS_CONFIG["$dest"]=8
CHECKERS_CONFIG["$dest"]=8
done
# Debug: Print TRANSFERS_CONFIG and CHECKERS_CONFIG
# for dest in "${!TRANSFERS_CONFIG[@]}"; do
# echo "Destination: $dest, Transfers: ${TRANSFERS_CONFIG[$dest]}, Checkers: ${CHECKERS_CONFIG[$dest]}"
# done
# Function to get the current timestamp for logging
get_timestamp() {
date '+%Y-%m-%d %H:%M:%S'
}
# Function to get the current timestamp for filenames
get_filename_timestamp() {
date '+%Y%m%d_%H%M%S'
}
# Log file setup
LOG_DIR="./logs2"
mkdir -p "$LOG_DIR"
LOG_FILE="${LOG_DIR}/rclone_sync_$(get_filename_timestamp).log"
# LOG_FILE_CHECK="${LOG_DIR}/rclone_check_$(get_filename_timestamp).log"
# Function to log messages with a timestamp
log() {
local message="$1"
echo "$(get_timestamp) - $message" >> "$LOG_FILE"
}
# Function to check files between source and destination
check_files_md5() {
local src="$1"
local dest="$2"
# Generate file list for source in parallel
log "<<<INFO>>> Generating file list for source..."
find "$src" -type f -print0 | xargs -0 -P 4 -I {} md5sum {} > source_files.md5
# Generate file list for destination in parallel
log "<<<INFO>>> Generating file list for destination..."
find "$dest" -type f -print0 | xargs -0 -P 4 -I {} md5sum {} > dest_files.md5
# Compare the file lists
log "<<<INFO>>> Comparing file lists..."
diff source_files.md5 dest_files.md5 > rclone_file_diff.log
if [ -s file_diff.log ]; then
log "<<<ERROR>>> Differences found. Check file_diff.log for details."
else
log "<<<INFO>>> File lists match. No differences found."
fi
}
# Function to check files between source and destination
check_files_rclone_check() {
local src="$1"
local dest="$2"
local delay="${3:-0}" # Optional delay parameter, defaults to 0 if not provided
# Extract base path from destination, including everything before and including the colon
local dest_base=$(echo "$dest" | sed -E 's/^([^:]+:\/[^\/]+\/[^\/]+).*$/\1/')
local checkers="${CHECKERS_CONFIG[$dest_base]:-1}"
log "<<<DEBUG>>> rclone check --checkers: $checkers"
# local checkers=2;
# Some parameters which are not currently used:
# --stats=10s \
# --buffer-size=0 \
# --multi-thread-streams=0 \
# --rc \ # remote control
# --max-depth 2
# --fast-list
# strace below is for low level debugging, but generates huge log, more than 16GB so far...
# strace -f -o "${LOG_DIR}/rclone_check_strace_$(get_filename_timestamp).log" rclone check "$src" "$dest" \
# nice -n 10 rclone check # This command runs rclone check with lower CPU priority, which can help reduce its impact on system performance.
# ionice -c2 -n7 rclone check # Adjust the priority of the rclone check process to reduce its impact on system I/O and CPU usage.
# ionice -c2 -n7 nice -n 10 rclone check
log "<<<INFO>>> Running rclone check from $src to $dest"
{
log "<<<DEBUG>>> 1 Rclone check sleep delay = $delay seconds."
rclone check "$src" "$dest" \
--checkers="$checkers" \
--fast-list \
--multi-thread-streams=0 \
--buffer-size=0 \
--one-way \
--checksum \
--log-file="$LOG_FILE" \
--log-level=DEBUG \
--retries 3 \
--retries-sleep 3s \
--progress
log "<<<DEBUG>>> 2 Rclone check sleep delay = $delay seconds."
# Optional sleep delay
if [ -n "$delay" ] && [ "$delay" -gt 0 ]; then
log "<<<INFO>>> Sleeping rclone check for $delay seconds..."
sleep "$delay"
fi
if [ $? -eq 0 ]; then
log "<<<INFO>>> Rclone check completed successfully for $src and $dest"
else
log "<<<ERROR>>> Rclone check failed for $src and $dest."
# Uncomment the exit 1 below for fail fast debugging:
# exit 1
fi
} >> "$LOG_FILE" 2>&1
}
# Function to tar and copy files with colons
copy_colon_files_with_tar() {
local src="$1"
local dest="$2"
find "$src" -name '*:*' | while IFS= read -r path; do
if [ -e "$path" ]; then
local relative_path="${path#$src/}"
# Replace colons with underscores for the tarball name
local tarball_name="$(echo "$(basename "$path")" | sed 's/:/_/g').tar.gz"
local dest_tarball="$dest/$(dirname "$relative_path")/$tarball_name"
log "<<<INFO>>> Archiving $path to $dest_tarball"
mkdir -p "$(dirname "$dest_tarball")"
tar czf "$dest_tarball" -C "$(dirname "$path")" "$(basename "$path")"
if [ $? -eq 0 ]; then
log "<<<INFO>>> Successfully archived $path to $dest_tarball"
else
log "<<<ERROR>>> Error archiving $path to $dest_tarball"
fi
fi
done
}
# Function to check if directories exist
check_directories() {
local src="$1"
local dest="$2"
if [ ! -d "${src#*:}" ]; then
log "<<<ERROR>>> Source directory does not exist: ${src#*:}. Skipping."
return 1
fi
# It is normal for destinations to not exist, so this is not needed for now.
# if [ ! -d "${dest#*:}" ]; then
# log "Destination directory does not exist: ${dest#*:}. Skipping."
# return 1
# fi
return 0
}
# Function to perform rclone sync
# Some parameters:
# Local encoding all:
# --local-encoding "Asterisk,BackQuote,BackSlash,Colon,CrLf,Ctl,Del,Dollar,Dot,DoubleQuote,Hash,InvalidUtf8,LeftCrLfHtVt,LeftPeriod,LeftSpace,LeftTilde,LtGt,None,Percent,Pipe,Question,RightCrLfHtVt,RightPeriod,RightSpace,Semicolon,SingleQuote,Slash,SquareBracket"
# --local-encoding "None"
# --local-encoding None
# None removed in encoding below:
# --local-encoding Asterisk,BackQuote,BackSlash,Colon,CrLf,Ctl,Del,Dollar,Dot,DoubleQuote,Hash,InvalidUtf8,LeftCrLfHtVt,LeftPeriod,LeftSpace,LeftTilde,LtGt,Percent,Pipe,Question,RightCrLfHtVt,RightPeriod,RightSpace,Semicolon,SingleQuote,Slash,SquareBracket
# --local-nounc
# --local-unicode-normalization
# Some include and exclude parameters:
# --include="*:*"
# --exclude=".DS_Store" \
# --exclude="**/.@__thumb/**" \
# --exclude="**/.Icon/**" \
# --exclude="**/.ipynb_checkpoints/**" \
# --exclude="*:*" \
# --exclude="*^M*" \
# Some other parameters:
# --checksum
# $checksum_option \
# --stats=10s \
# --no-prealloc \
# --no-use-mmap \
# strace below is for low level debugging, but generates huge log, more than 16GB so far...
# strace -f -o "${LOG_DIR}/rclone_sync_strace_$(get_filename_timestamp).log" rclone sync "$src" "$dest" \
rclone_sync() {
local src="$1"
local dest="$2"
local use_checksum="${3:-false}" # Defaults to false if not provided
local buffer_size="$4" # Optional
local bwlimit="$5" # Optional
local cache_chunk_total_size="$6" # Optional
local cache_tmp_upload_path="$7" # Optional
local cache_chunk_path="$8" # Optional
local cache_info_age="$9" # Optional
local delay="${10:-0}" # Optional delay parameter, defaults to 0 if not provided
# Extract base path from destination, including everything before and including the colon
local dest_base=$(echo "$dest" | sed -E 's/^([^:]+:\/[^\/]+\/[^\/]+).*$/\1/')
# Debug output
log "<<<DEBUG>>> Full destination path: ${dest#*:}"
log "<<<DEBUG>>> Base destination directory: $dest_base"
local transfers="${TRANSFERS_CONFIG[$dest_base]:-4}" # Defaults to default value if not provided
local checkers="${CHECKERS_CONFIG[$dest_base]:-1}" # Defaults to default value if not provided
log "<<<DEBUG>>> rclone sync --transfers: $transfers"
log "<<<DEBUG>>> rclone sync --checkers: $checkers"
# Determine if checksum should be used based on the use_checksum parameter
local checksum_option=""
if [ "$use_checksum" = true ]; then
checksum_option="--checksum"
fi
# Optional parameters
local buffer_size_option=""
if [ -n "$buffer_size" ]; then
buffer_size_option="--buffer-size=$buffer_size"
fi
local bwlimit_option=""
if [ -n "$bwlimit" ]; then
bwlimit_option="--bwlimit=$bwlimit"
fi
local cache_chunk_total_size_option=""
if [ -n "$cache_chunk_total_size" ]; then
cache_chunk_total_size_option="--cache-chunk-total-size=$cache_chunk_total_size"
fi
local cache_tmp_upload_path_option=""
if [ -n "$cache_tmp_upload_path" ]; then
cache_tmp_upload_path_option="--cache-tmp-upload-path=$cache_tmp_upload_path"
fi
local cache_chunk_path_option=""
if [ -n "$cache_chunk_path" ]; then
cache_chunk_path_option="--cache-chunk-path=$cache_chunk_path"
fi
local cache_info_age_option=""
if [ -n "$cache_info_age" ]; then
cache_info_age_option="--cache-info-age=$cache_info_age"
fi
log "<<<INFO>>> Syncing from $src to $dest"
{
rclone sync "$src" "$dest" \
--transfers="$transfers" \
--checkers="$checkers" \
--log-level=DEBUG \
--log-file="$LOG_FILE" \
--retries 1 \
--retries-sleep 1s \
--copy-links \
--progress \
--fast-list \
--use-mmap \
$checksum_option \
$buffer_size_option \
$bwlimit_option \
$cache_chunk_total_size_option \
$cache_tmp_upload_path_option \
$cache_chunk_path_option \
$cache_info_age_option \
--delete-during
# Optional sleep delay
if [ -n "$delay" ] && [ "$delay" -gt 0 ]; then
log "<<<INFO>>> Sleeping rclone sync for $delay seconds..."
sleep "$delay"
fi
if [ $? -eq 0 ]; then
log "<<<INFO>>> Rclone sync completed successfully for $src to $dest"
else
log "<<<ERROR>>> Rclone sync failed for $src to $dest after 1 retry."
# Uncomment the exit 1 below for fail fast debugging:
# exit 1
fi
} >> "$LOG_FILE" 2>&1
}
# Function to sync source and destination directories to single storage units SSD and NVME
sync_sources_and_destinations_to_single_stores_ssd_nvme() {
USE_CHECKSUM=false
for DEST_DIR in "${DEST_DIRS_SINGLE_STORE_SSD_NVME[@]}"; do
for SRC in "${SRC_DIRS[@]}"; do
if check_directories "$SRC" "$DEST_DIR/$(basename "$SRC")"; then
# Working 1 for SSD with below parameters + transfers=4 and checkers=1:
# Working 1 for SSD with below parameters + transfers=2 and checkers=1:
# rclone_sync "$SRC" "$DEST_DIR/$(basename "$SRC")" "$USE_CHECKSUM" "16M" "10M" "1G"
rclone_sync "$SRC" "$DEST_DIR/$(basename "$SRC")" \
"$USE_CHECKSUM" \
"16M" \
"1.5M" \
"" \
"" \
"" \
""
fi
done
done
}
# Function to sync source and destination directories to single storage units HDD 7200RPM
sync_sources_and_destinations_to_single_stores_7200rpm() {
USE_CHECKSUM=false
for DEST_DIR in "${DEST_DIRS_SINGLE_STORE_7200RPM[@]}"; do
for SRC in "${SRC_DIRS[@]}"; do
if check_directories "$SRC" "$DEST_DIR/$(basename "$SRC")"; then
# Working 1 for SSD with below parameters + transfers=8 and checkers=8:
# rclone_sync "$SRC" "$DEST_DIR/$(basename "$SRC")" "$USE_CHECKSUM" "1G" "150M" "8G"
rclone_sync "$SRC" "$DEST_DIR/$(basename "$SRC")" \
"$USE_CHECKSUM" \
"32M" \
"10M" \
"8G" \
"/home/usertemp/rclone_cache_temp/tmp_upload" \
"/home/usertemp/rclone_cache_temp/chunks" \
"1d"
fi
done
done
}
# Function to sync source and destination directories to multiple storage units HDD 5400RPM
sync_specific_sources_and_destinations_to_multi_stores_5400rpms() {
USE_CHECKSUM=false
# Sync each source directory to specific backup destinations
for SRC in "${SRC_DIRS[@]}"; do
case "$SRC" in
"localsrc1:/mnt/tns-google-drive-smb" | "localsrc2:/mnt/tns-maindatads-smb" | "localsrc3:/mnt/tns-app-configs-smb")
DEST_DIR="${DEST_DIRS_MULTI_STORE_5400RPM[1]}"
;;
"localsrc4:/mnt/tns-proxbkps-ds-1")
DEST_DIR="${DEST_DIRS_MULTI_STORE_5400RPM[0]}"
;;
*)
log "<<<ERROR>>> Unknown source directory: $SRC. Skipping backup sync."
continue
;;
esac
if check_directories "$SRC" "$DEST_DIR/$(basename "$SRC")"; then
# Working 1 for SSD with below parameters + transfers=8 and checkers=8:
# rclone_sync "$SRC" "$DEST_DIR/$(basename "$SRC")" "$USE_CHECKSUM" "1G" "150M" "8G"
rclone_sync "$SRC" "$DEST_DIR/$(basename "$SRC")" \
"$USE_CHECKSUM" \
"32M" \
"" \
"8G" \
"/home/usertemp/rclone_cache_temp/tmp_upload" \
"/home/usertemp/rclone_cache_temp/chunks" \
"1d"
fi
done
}
# Function to check sources and destinations
check_sources_and_destinations_to_single_stores_ssd_nvme() {
# Check source and destination directories to single storage units SSD and NVME
for DEST_DIR in "${DEST_DIRS_SINGLE_STORE_SSD_NVME[@]}"; do
for SRC in "${SRC_DIRS[@]}"; do
check_files_rclone_check "$SRC" "$DEST_DIR/$(basename "$SRC")" "1"
# Ensure that only one rclone check runs at a time
wait
# rclone check does not have bandwidth limiting, so some throttling can be done with sleep:
# DELAY="0.5" # Delay in seconds between checks (adjust to control "throttling").
# sleep "$DELAY"
done
done
}
check_sources_and_destinations_to_single_stores_7200rpm() {
# Check source and destination directories to single storage units HDD 7200RPM
for DEST_DIR in "${DEST_DIRS_SINGLE_STORE_7200RPM[@]}"; do
for SRC in "${SRC_DIRS[@]}"; do
check_files_rclone_check "$SRC" "$DEST_DIR/$(basename "$SRC")" ""
done
done
}
check_sources_and_destinations_to_multi_stores_5400rpms(){
# Check source and destination directories to multiple storage units HDD 5400RPM
for SRC in "${SRC_DIRS[@]}"; do
case "$SRC" in
"localsrc1:/mnt/tns-google-drive-smb" | "localsrc2:/mnt/tns-maindatads-smb" | "localsrc3:/mnt/tns-app-configs-smb")
DEST_DIR="${DEST_DIRS_MULTI_STORE_5400RPM[0]}"
;;
"localsrc4:/mnt/tns-proxbkps-ds-1")
DEST_DIR="${DEST_DIRS_MULTI_STORE_5400RPM[1]}"
;;
*)
log "<<<ERROR>>> Unknown source directory: $SRC. Skipping backup sync."
continue
;;
esac
check_files_rclone_check "$SRC" "$DEST_DIR/$(basename "$SRC")" ""
done
}
print_debug_info_1(){
for DEST_DIR in "${DEST_DIRS_SINGLE_STORE_SSD_NVME[@]}"; do
for SRC in "${SRC_DIRS[@]}"; do
log "<<<DEBUG>>> _single_stores_ssd_nvme = SOURCE: "$SRC" | DESTINATION: "$DEST_DIR/$(basename "$SRC")" ..."
done
done
for DEST_DIR in "${DEST_DIRS_SINGLE_STORE_7200RPM[@]}"; do
for SRC in "${SRC_DIRS[@]}"; do
log "<<<DEBUG>>> _single_stores_7200rpm = SOURCE: "$SRC" | DESTINATION: "$DEST_DIR/$(basename "$SRC")" ..."
done
done
for SRC in "${SRC_DIRS[@]}"; do
case "$SRC" in
"localsrc1:/mnt/tns-google-drive-smb" | "localsrc2:/mnt/tns-maindatads-smb" | "localsrc3:/mnt/tns-app-configs-smb")
DEST_DIR="${DEST_DIRS_MULTI_STORE_5400RPM[1]}"
;;
"localsrc4:/mnt/tns-proxbkps-ds-1")
DEST_DIR="${DEST_DIRS_MULTI_STORE_5400RPM[0]}"
;;
*)
log "<<<ERROR>>> Unknown source directory: $SRC. Skipping backup sync."
continue
;;
esac
log "<<<DEBUG>>> _multi_stores_5400rpms = SOURCE: "$SRC" | DESTINATION: "$DEST_DIR/$(basename "$SRC")" ..."
done
}
################################ Program execution start and function calling.
# Capture start time
START_TIME=$(date +%s)
START_TIMESTAMP=$(get_timestamp)
# Start logging
log "<<<INFO>>> Starting sync process at $START_TIMESTAMP"
# Calling function to sync source and destination directories to single storage units SSD and NVME
sync_sources_and_destinations_to_single_stores_ssd_nvme
# Calling function to sync source and destination directories to single storage units HDD 7200RPM
# sync_sources_and_destinations_to_single_stores_7200rpm
# Calling function to check sources and destinations to single storage units SSD and NVME
check_sources_and_destinations_to_single_stores_ssd_nvme
# Calling function to check sources and destinations to single storage units HDD 7200RPM
# check_sources_and_destinations_to_single_stores_7200rpm
# Calling function to sync source and destination directories to multiple storage units HDD 5400RPM
# sync_specific_sources_and_destinations_to_multi_stores_5400rpms
# Calling function to check sources and destinations to to multiple storage units HDD 5400RPM
# check_sources_and_destinations_to_multi_stores_5400rpms
print_debug_info_1
# Capture end time
END_TIME=$(date +%s)
END_TIMESTAMP=$(get_timestamp)
ELAPSED_TIME=$((END_TIME - START_TIME))
ELAPSED_HOURS=$((ELAPSED_TIME / 3600))
ELAPSED_MINUTES=$(((ELAPSED_TIME % 3600) / 60))
ELAPSED_SECONDS=$((ELAPSED_TIME % 60))
# Log end time and elapsed time
log "<<<INFO>>> Sync process finished at $END_TIMESTAMP"
log "<<<INFO>>> Total time taken: ${ELAPSED_HOURS}h ${ELAPSED_MINUTES}m ${ELAPSED_SECONDS}s"
Run the command 'rclone version' and share the full output of the command.
rclone --version
rclone v1.60.1-DEV
- os/version: ubuntu 24.04 (64 bit)
- os/kernel: 6.8.0-38-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.22.2
- go/linking: dynamic
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
None - the source and destinations are locally mounted.
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone check "$src" "$dest" \
--checkers="$checkers" \
--fast-list \
--multi-thread-streams=0 \
--buffer-size=0 \
--one-way \
--checksum \
--log-file="$LOG_FILE" \
--log-level=DEBUG \
--retries 3 \
--retries-sleep 3s \
--progress
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
### Source directories
[localsrc1]
type = local
[localsrc2]
type = local
[localsrc3]
type = local
[localsrc4]
type = local
################################
# Destination directories for single storage units
[localdest1]
type = local
encoding = Asterisk,BackQuote,BackSlash,Colon,CrLf,Ctl,Del,Dollar,Dot,DoubleQuote,Hash,InvalidUtf8,LeftCrLfHtVt,LeftPeriod,LeftSpace,LeftTilde,LtGt,Percent,Pipe,Question,RightCrLfHtVt,RightPeriod,RightSpace,Semicolon,SingleQuote,Slash,SquareBracket
[localdest4]
type = local
encoding = Asterisk,BackQuote,BackSlash,Colon,CrLf,Ctl,Del,Dollar,Dot,DoubleQuote,Hash,InvalidUtf8,LeftCrLfHtVt,LeftPeriod,LeftSpace,LeftTilde,LtGt,Percent,Pipe,Question,RightCrLfHtVt,RightPeriod,RightSpace,Semicolon,SingleQuote,Slash,SquareBracket
################################
# Destination directories for multiple storage units
[localdest2]
type = local
encoding = Asterisk,BackQuote,BackSlash,Colon,CrLf,Ctl,Del,Dollar,Dot,DoubleQuote,Hash,InvalidUtf8,LeftCrLfHtVt,LeftPeriod,LeftSpace,LeftTilde,LtGt,Percent,Pipe,Question,RightCrLfHtVt,RightPeriod,RightSpace,Semicolon,SingleQuote,Slash,SquareBracket
[localdest3]
type = local
encoding = Asterisk,BackQuote,BackSlash,Colon,CrLf,Ctl,Del,Dollar,Dot,DoubleQuote,Hash,InvalidUtf8,LeftCrLfHtVt,LeftPeriod,LeftSpace,LeftTilde,LtGt,Percent,Pipe,Question,RightCrLfHtVt,RightPeriod,RightSpace,Semicolon,SingleQuote,Slash,SquareBracket
################################
A log from the command that you were trying to run with the -vv flag
The log is in pastebin:
The log files from the rclone sync or rclone check commands don't have anything which seems to point to any issue.
When the crash happens, the log file just stops at the point of the file which it was currently processing.
There don't seem to be any errors in the logs of the Ubuntu Server machine as well.
