What is the problem you are having with rclone?
Unable to clone from Zoho to s3. Here is our workflow:
- (One time only) get Zoho tokens using rclone config tool
Then once every 15 minutes:
-
Fetch rclone configuration from our s3
-
Launch rclone
-
Upload rclone configuration to our s3 (tokens may have changed)
We should not reach Zoho limiation as stated in the documention we are limited to 10 tokens for 10 minutes.
This stopped working arround last week but has been working well for years.
What I tried:
-
Generate new fresh token with rclone config tool
-
Creating a new oauth app on zoho side and getting new tokens
Run the command 'rclone version' and share the full output of the command.
We use a custom docker image based on rclone/rclone:1.65, the image only add aws cli from alpine packages
/app # rclone version
rclone v1.65.2
- os/version: alpine 3.19.0 (64 bit)
- os/kernel: 5.10.0-21-amd64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.6
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
Zoho
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone sync $SOURCE_NAME:$SOURCE_PATH $DESTINATION_NAME:$BUCKET_NAME/$DESTINATION_PATH --filter-from filters --config rclone.conf
The full script is
#!/usr/bin/env sh
set -e
# Fetch rclone configuration from encrypted bucket
aws s3 cp s3://$BUCKET_NAME/configuration/rclone.conf .
# Synchronize from zoho to s3
rclone sync $SOURCE_NAME:$SOURCE_PATH $DESTINATION_NAME:$BUCKET_NAME/$DESTINATION_PATH --filter-from filters --config rclone.conf
# Configuration may have changed after running, mostly because of
# token refreshment
aws s3 cp rclone.conf s3://$BUCKET_NAME/configuration/
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[zoho]
type = zoho
region = eu
token = {"access_token":XXX,"token_type":"Zoho-oauthtoken","refresh_token":XXX,"expiry":"2024-02-16T13:14:00.521231+01:00"}
root_folder_id = XXX
[s3]
type = s3
provider = AWS
env_auth = true
access_key_id = XXX
secret_access_key = XXX
region = eu-west-3
location_constraint = eu-west-3
acl = private
bucket_acl = private
A log from the command that you were trying to run with the -vv flag
2024/02/23 11:11:15 DEBUG : rclone: Version "v1.65.2" starting with parameters ["rclone" "sync" "zoho:OUT/" "s3:[REDACTED]/files/" "--filter-from" "filters" "--config" "rclone.conf" "-vv" "--log-file=full_logs.txt"]
2024/02/23 11:11:15 DEBUG : Creating backend with remote "zoho:OUT/"
2024/02/23 11:11:15 DEBUG : Using config file from "/app/rclone.conf"
2024/02/23 11:11:15 DEBUG : zoho: Loaded invalid token from config file - ignoring
2024/02/23 11:11:20 DEBUG : zoho: Token refresh failed try 1/5: oauth2: "invalid_code"
2024/02/23 11:11:21 DEBUG : zoho: Loaded invalid token from config file - ignoring
2024/02/23 11:11:21 DEBUG : zoho: Token refresh failed try 2/5: oauth2: "invalid_code"
2024/02/23 11:11:22 DEBUG : zoho: Loaded invalid token from config file - ignoring
2024/02/23 11:11:22 DEBUG : zoho: Token refresh failed try 3/5: oauth2: "invalid_code"
2024/02/23 11:11:23 DEBUG : zoho: Loaded invalid token from config file - ignoring
2024/02/23 11:11:23 DEBUG : zoho: Token refresh failed try 4/5: oauth2: "invalid_code"
2024/02/23 11:11:24 DEBUG : zoho: Loaded invalid token from config file - ignoring
2024/02/23 11:11:24 DEBUG : zoho: Token refresh failed try 5/5: oauth2: "invalid_code"
2024/02/23 11:11:25 DEBUG : zoho: Loaded invalid token from config file - ignoring
2024/02/23 11:11:25 DEBUG : zoho: Token refresh failed try 1/5: oauth2: "invalid_code"
2024/02/23 11:11:26 DEBUG : zoho: Loaded invalid token from config file - ignoring
2024/02/23 11:11:26 DEBUG : zoho: Token refresh failed try 2/5: oauth2: "invalid_code"
2024/02/23 11:11:27 DEBUG : zoho: Loaded invalid token from config file - ignoring
2024/02/23 11:11:27 DEBUG : zoho: Token refresh failed try 3/5: oauth2: "invalid_code"
2024/02/23 11:11:28 DEBUG : zoho: Loaded invalid token from config file - ignoring
2024/02/23 11:11:28 DEBUG : zoho: Token refresh failed try 4/5: oauth2: "invalid_code"
2024/02/23 11:11:29 DEBUG : zoho: Loaded invalid token from config file - ignoring
2024/02/23 11:11:29 DEBUG : zoho: Token refresh failed try 5/5: oauth2: "invalid_code"
2024/02/23 11:11:30 Failed to create file system for "zoho:OUT/": couldn't list files: Get "https://workdrive.zoho.eu/api/v1/files/[REDACTED]/files?page%5Blimit%5D=10&page%5Boffset%5D=0": couldn't fetch token: oauth2: "invalid_code"
Thanks for your time working on this awesome tool, I hope we can find what's wrong here.