Problem syncing to/from Seafile source

lost_in_the_woods · June 17, 2023, 6:05pm

What is the problem you are having with rclone?

I'm trying to use rclone to back up a Seafile server running in Docker on the same machine. I'm having trouble syncing between the Seafile source and a local directory. Rclone can list the files in the Seafile source, and it can run syncs that delete files on that source, but it cannot copy files from that source to a local directory (or from a local directory to the source).

The error message mentions "Access token not found", but I'm not sure if that refers to a 2FA token or something else. Since rclone can list and delete files on the Seafile source, I assume it's not a 2FA thing, and 2FA is not enabled on the Seafile server in any case.

Note that the Seafile server is running 10.0.1, which is newer than mentioned on rclone's Seafile page.

Run the command 'rclone version' and share the full output of the command.

$ rclone version
rclone v1.62.2
- os/version: debian 12.0 (64 bit)
- os/kernel: 6.1.0-9-amd64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.20.2
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Seafile

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

richard@little:~$ rclone sync seafile:Launchy/test\ sync /home/richard/test/
2023/06/17 11:48:20 ERROR : test.txt: Failed to copy: failed to open source object: HTTP error 403 (403 Forbidden) returned body: "Access token not found\n"
2023/06/17 11:48:20 ERROR : Local file system at /home/richard/test: not deleting files as there were IO errors
2023/06/17 11:48:20 ERROR : Local file system at /home/richard/test: not deleting directories as there were IO errors
2023/06/17 11:48:20 ERROR : Attempt 1/3 failed with 1 errors and: failed to open source object: HTTP error 403 (403 Forbidden) returned body: "Access token not found\n"
2023/06/17 11:48:20 ERROR : test.txt: Failed to copy: failed to open source object: HTTP error 403 (403 Forbidden) returned body: "Access token not found\n"
2023/06/17 11:48:20 ERROR : Local file system at /home/richard/test: not deleting files as there were IO errors
2023/06/17 11:48:20 ERROR : Local file system at /home/richard/test: not deleting directories as there were IO errors
2023/06/17 11:48:20 ERROR : Attempt 2/3 failed with 1 errors and: failed to open source object: HTTP error 403 (403 Forbidden) returned body: "Access token not found\n"
2023/06/17 11:48:22 ERROR : test.txt: Failed to copy: failed to open source object: HTTP error 403 (403 Forbidden) returned body: "Access token not found\n"
2023/06/17 11:48:22 ERROR : Local file system at /home/richard/test: not deleting files as there were IO errors
2023/06/17 11:48:22 ERROR : Local file system at /home/richard/test: not deleting directories as there were IO errors
2023/06/17 11:48:22 ERROR : Attempt 3/3 failed with 1 errors and: failed to open source object: HTTP error 403 (403 Forbidden) returned body: "Access token not found\n"
2023/06/17 11:48:22 Failed to sync: failed to open source object: HTTP error 403 (403 Forbidden) returned body: "Access token not found\n"

Listing the contents of that directory does work:

$ rclone lsf seafile:Launchy/test\ sync
test.txt

and Rclone can even delete from the Seafile directory:

$ rclone sync /home/richard/test seafile:Launchy/test\ sync -vv
2023/06/17 11:35:16 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "sync" "/home/richard/test" "seafile:Launchy/test sync" "-vv"]
2023/06/17 11:35:16 DEBUG : Creating backend with remote "/home/richard/test"
2023/06/17 11:35:16 DEBUG : Using config file from "/home/richard/.config/rclone/rclone.conf"
2023/06/17 11:35:16 DEBUG : Creating backend with remote "seafile:Launchy/test sync"
2023/06/17 11:35:16 DEBUG : Seafile server version 10.0.1
2023/06/17 11:35:16 DEBUG : seafile library 'Launchy' path 'test sync': Waiting for checks to finish
2023/06/17 11:35:16 DEBUG : seafile library 'Launchy' path 'test sync': Waiting for transfers to finish
2023/06/17 11:35:16 DEBUG : Waiting for deletions to finish
2023/06/17 11:35:16 INFO  : test.txt: Deleted
2023/06/17 11:35:16 INFO  : There was nothing to transfer
2023/06/17 11:35:16 INFO  :
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:                 1 / 1, 100%
Deleted:                1 (files), 0 (dirs)
Elapsed time:         0.5s

2023/06/17 11:35:16 DEBUG : 3 go routines active

The rclone config contents with secrets removed.

[seafile]
type = seafile
url = http://192.168.x.x/
user = xxx
pass = xxx
2fa = false

[backblaze]
(snipped)

A log from the command with the `-vv` flag

richard@little:~$ rclone sync seafile:Launchy/test\ sync /home/richard/test/ -vv
2023/06/17 11:33:45 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "sync" "seafile:Launchy/test sync" "/home/richard/test/" "-vv"]
2023/06/17 11:33:45 DEBUG : Creating backend with remote "seafile:Launchy/test sync"
2023/06/17 11:33:45 DEBUG : Using config file from "/home/richard/.config/rclone/rclone.conf"
2023/06/17 11:33:45 DEBUG : Seafile server version 10.0.1
2023/06/17 11:33:45 DEBUG : Creating backend with remote "/home/richard/test/"
2023/06/17 11:33:45 DEBUG : fs cache: renaming cache item "/home/richard/test/" to be canonical "/home/richard/test"
2023/06/17 11:33:45 DEBUG : Local file system at /home/richard/test: Waiting for checks to finish
2023/06/17 11:33:45 DEBUG : Local file system at /home/richard/test: Waiting for transfers to finish
2023/06/17 11:33:48 ERROR : test.txt: Failed to copy: failed to open source object: HTTP error 403 (403 Forbidden) returned body: "Access token not found\n"
2023/06/17 11:33:48 ERROR : Local file system at /home/richard/test: not deleting files as there were IO errors
2023/06/17 11:33:48 ERROR : Local file system at /home/richard/test: not deleting directories as there were IO errors
2023/06/17 11:33:48 ERROR : Attempt 1/3 failed with 1 errors and: failed to open source object: HTTP error 403 (403 Forbidden) returned body: "Access token not found\n"
2023/06/17 11:33:48 DEBUG : Local file system at /home/richard/test: Waiting for checks to finish
2023/06/17 11:33:48 DEBUG : Local file system at /home/richard/test: Waiting for transfers to finish
2023/06/17 11:33:48 ERROR : test.txt: Failed to copy: failed to open source object: HTTP error 403 (403 Forbidden) returned body: "Access token not found\n"
2023/06/17 11:33:48 ERROR : Local file system at /home/richard/test: not deleting files as there were IO errors
2023/06/17 11:33:48 ERROR : Local file system at /home/richard/test: not deleting directories as there were IO errors
2023/06/17 11:33:48 ERROR : Attempt 2/3 failed with 1 errors and: failed to open source object: HTTP error 403 (403 Forbidden) returned body: "Access token not found\n"
2023/06/17 11:33:48 DEBUG : Local file system at /home/richard/test: Waiting for checks to finish
2023/06/17 11:33:48 DEBUG : Local file system at /home/richard/test: Waiting for transfers to finish
2023/06/17 11:33:49 ERROR : test.txt: Failed to copy: failed to open source object: HTTP error 403 (403 Forbidden) returned body: "Access token not found\n"
2023/06/17 11:33:49 ERROR : Local file system at /home/richard/test: not deleting files as there were IO errors
2023/06/17 11:33:49 ERROR : Local file system at /home/richard/test: not deleting directories as there were IO errors
2023/06/17 11:33:49 ERROR : Attempt 3/3 failed with 1 errors and: failed to open source object: HTTP error 403 (403 Forbidden) returned body: "Access token not found\n"
2023/06/17 11:33:49 INFO  :
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         4.3s

2023/06/17 11:33:49 DEBUG : 9 go routines active
2023/06/17 11:33:49 Failed to sync: failed to open source object: HTTP error 403 (403 Forbidden) returned body: "Access token not found\n"

Does anyone have any suggestions?

asdffdsa · June 17, 2023, 8:35pm

hello and welcome to the forum,

https://github.com/rclone/rclone/issues/6126

for a deeper look, add --dump=headers --retries=1

lost_in_the_woods · June 18, 2023, 6:16am

Interesting - not sure if that's the same as the error I'm getting or not.

Here are the headers in a Pastebin rclone log - Pastebin.com

It looks like it starts by making a request to my server, but then hits docs.seafile.com for some reason.

bstrmbrg · June 18, 2023, 6:32am

It looks like there are multiple sync commands. Shouldn't it be more like:

rclone sync seafile:Launchy/test /home/richard/test

Assuming you have a folder called "Launchy" in Seafile, and another folder called "test" under that.

lost_in_the_woods · June 18, 2023, 6:54am

The folder is actually called "test sync" and the space is escaped by the backslash

bstrmbrg · June 18, 2023, 7:00am

If it's on the same server and you only want to backup to another local folder you can do that without a remote by doing something like:

rclone sync /path/to/seafile/persistentdata /path/to/local/folder

Depends a little on what you are trying to achieve.

bstrmbrg · June 18, 2023, 7:07am

Also, can you try the command using paths that do not have spaces?

lost_in_the_woods · June 18, 2023, 7:29am

Seafile doesn't make libraries locally accessible unless you mount them read-only through FUSE, see FUSE extension - Seafile Admin Manual. It might work for backup, but I'm concerned about the server rebooting and rclone trying to sync from an unmounted directory and wiping out the backup.

I tried renaming the test directory and nothing changed.

$ rclone sync seafile:Launchy/test /home/richard/test/ --retries=1
2023/06/18 01:20:19 ERROR : test.txt: Failed to copy: failed to open source object: HTTP error 403 (403 Forbidden) returned body: "Access token not found\n"
2023/06/18 01:20:19 ERROR : Local file system at /home/richard/test: not deleting files as there were IO errors
2023/06/18 01:20:19 ERROR : Local file system at /home/richard/test: not deleting directories as there were IO errors
2023/06/18 01:20:19 ERROR : Attempt 1/1 failed with 1 errors and: failed to open source object: HTTP error 403 (403 Forbidden) returned body: "Access token not found\n"
2023/06/18 01:20:19 Failed to sync: failed to open source object: HTTP error 403 (403 Forbidden) returned body: "Access token not found\n"

bstrmbrg · June 18, 2023, 8:05am

I was bored so I fired up a Seafile Docker and it works for me. I did the following:

Start seafile
rclone config, go through wizard to configure remote
rclone sync seafile:/test /home/olympen/test_seafile -vv
Output:

2023/06/18 10:00:14 DEBUG : rclone: Version "v1.61.1" starting with parameters ["rclone" "sync" "seafile:/test" "/home/olympen/test_seafile" "-vv"]
2023/06/18 10:00:14 DEBUG : Creating backend with remote "seafile:/test"
2023/06/18 10:00:14 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/06/18 10:00:14 DEBUG : Seafile server version 10.0.1
2023/06/18 10:00:14 DEBUG : fs cache: renaming cache item "seafile:/test" to be canonical "seafile:test"
2023/06/18 10:00:14 DEBUG : Creating backend with remote "/home/olympen/test_seafile"
2023/06/18 10:00:14 DEBUG : Local file system at /home/olympen/test_seafile: Waiting for checks to finish
2023/06/18 10:00:14 DEBUG : Local file system at /home/olympen/test_seafile: Waiting for transfers to finish
2023/06/18 10:00:15 INFO : test.xlsx: Copied (new)
2023/06/18 10:00:15 DEBUG : Waiting for deletions to finish
2023/06/18 10:00:15 INFO :
Transferred: 8.376 KiB / 8.376 KiB, 100%, 0 B/s, ETA -
Transferred: 1 / 1, 100%
Elapsed time: 1.0s

Rclone config:

[seafile]
type = seafile
url = http://10.23.2.150:62348/
user = XXX
pass = lLWQ1KXVHSRzS68sd1vpy0F2TqxLYlY
library = Testlibrary

Have you really done rclone config? Because the password is encrypted

bstrmbrg · June 18, 2023, 8:11am

After deleting excel file from seafile gui:

root@olympen:/home/olympen/test_seafile# rclone sync seafile:/test /home/olympen/test_seafile -vv
2023/06/18 10:09:58 DEBUG : rclone: Version "v1.61.1" starting with parameters ["rclone" "sync" "seafile:/test" "/home/olympen/test_seafile" "-vv"]
2023/06/18 10:09:58 DEBUG : Creating backend with remote "seafile:/test"
2023/06/18 10:09:58 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/06/18 10:09:58 DEBUG : Seafile server version 10.0.1
2023/06/18 10:09:59 DEBUG : fs cache: renaming cache item "seafile:/test" to be canonical "seafile:test"
2023/06/18 10:09:59 DEBUG : Creating backend with remote "/home/olympen/test_seafile"
2023/06/18 10:09:59 DEBUG : Local file system at /home/olympen/test_seafile: Waiting for checks to finish
2023/06/18 10:09:59 DEBUG : Local file system at /home/olympen/test_seafile: Waiting for transfers to finish
2023/06/18 10:09:59 DEBUG : Waiting for deletions to finish
2023/06/18 10:09:59 INFO : test.xlsx: Deleted
2023/06/18 10:09:59 INFO : There was nothing to transfer
2023/06/18 10:09:59 INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Checks: 1 / 1, 100%
Deleted: 1 (files), 0 (dirs)
Elapsed time: 0.6s

lost_in_the_woods · June 18, 2023, 10:20am

Looking at your rclone config, it looks like your test was in library mode. I'm using the other mode, pointing to the root of the server.

Just to be sure it wasn't a config issue, I removed my rclone.conf, changed my Seafile password, removing all special characters, and reran rclone config. The new rclone.conf file is below. I still got the same error when trying to sync a file from Seafile to the local folder, and rclone was still able to delete the file from Seafile. The successful deletion makes me think that the username and password are correct and that the issue is somewhere else.

[seafile_new]
type = seafile
url = http://192.168.1.77/
user = XXX
pass = cuQb5QZVKoUdKTliy_Q8DS8DigU

Here's a pastebin of the headers from the successful deletion, if that helps rclone successful log - Pastebin.com

bstrmbrg · June 18, 2023, 11:26am

I am not familiar with seafile so I'm not sure if this is how you want it but try editing /shared/seafile/conf/seahub_settings.py and set the local IP:port in SERVICE_URL and FILE_SERVER_ROOT

Now I can sync my Seafile root or any subfolder:

root@olympen:/opt/docker/data/seafile/app/nginx/conf# rclone sync seafile: /home/olympen/test_seafile -vv
2023/06/18 13:25:48 DEBUG : rclone: Version "v1.61.1" starting with parameters ["rclone" "sync" "seafile:" "/home/olympen/test_seafile" "-vv"]
2023/06/18 13:25:48 DEBUG : Creating backend with remote "seafile:"
2023/06/18 13:25:48 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/06/18 13:25:48 DEBUG : Seafile server version 10.0.1
2023/06/18 13:25:48 DEBUG : Creating backend with remote "/home/olympen/test_seafile"
2023/06/18 13:25:48 DEBUG : Local file system at /home/olympen/test_seafile: Waiting for checks to finish
2023/06/18 13:25:48 DEBUG : Local file system at /home/olympen/test_seafile: Waiting for transfers to finish
2023/06/18 13:25:49 INFO  : test3/safsdgfd.xlsx: Copied (new)
2023/06/18 13:25:49 INFO  : My Library/seafile-tutorial.doc: Copied (new)
2023/06/18 13:25:49 INFO  : Testlibrary/test.docx: Copied (new)
2023/06/18 13:25:49 DEBUG : Waiting for deletions to finish
2023/06/18 13:25:49 INFO  : safsdgfd.xlsx: Deleted
2023/06/18 13:25:49 INFO  :
Transferred:      315.045 KiB / 315.045 KiB, 100%, 0 B/s, ETA -
Checks:                 1 / 1, 100%
Deleted:                1 (files), 0 (dirs)
Transferred:            3 / 3, 100%
Elapsed time:         1.4s

2023/06/18 13:25:49 DEBUG : 7 go routines active
root@olympen:/opt/docker/data/seafile/app/nginx/conf#
root@olympen:/opt/docker/data/seafile/app/nginx/conf#
root@olympen:/opt/docker/data/seafile/app/nginx/conf#
root@olympen:/opt/docker/data/seafile/app/nginx/conf#
root@olympen:/opt/docker/data/seafile/app/nginx/conf#
root@olympen:/opt/docker/data/seafile/app/nginx/conf# ls /home/olympen/test_seafile
'My Library'   test3   Testlibrary
root@olympen:/opt/docker/data/seafile/app/nginx/conf#

Is that what you wanted?

EDIT: Updated rclone config:

[seafile]
type = seafile
url = http://10.23.2.150:62348/
user = XXX
pass = lLWQ1KXVHSRzS68sd1vpy0F2TqxLYlY

lost_in_the_woods · June 18, 2023, 11:55am

Thank you! That was it. I changed my seahub_settings.py to point to the server and now syncs work properly.

bstrmbrg · June 18, 2023, 11:59am

No problem :). I am actually looking for some kind of self hosted cloud storage solution myself. I have tried Nextcloud a little but it feels kind of glitchy. How is Seafile compared to Nextcloud?

lost_in_the_woods · June 18, 2023, 12:43pm

I've only used Nextcloud a little bit several versions ago, but it felt like it was trying to do everything at once and only did an OK job at each. Seafile only does file sync, but it does it well.

Things I like about Seafile:

The client syncs fast but doesn't use a lot of resources (on Windows).
I can share a library with someone else.
Library files can be stored on device or left on the server, which is great for mobile use.
I've used it for several months with no issues.

Things I don't like about Seafile:

The only way to access library contents from the server is either through the read-only FUSE mount or through a WebDAV extension.
The built-in backup system doesn't back up the underlying files, making restoration and backup validation really annoying. Using rclone fixes this, but then you have to have at least one rclone backend per user, and possibly more if there are encrypted libraries. Doing it this way does mean that the server admin can see everything, but if you're self-hosting it you're probably OK with that.
The server needs garbage collection, which can't be done while it's running if you're using the Community Edition.

bstrmbrg · June 19, 2023, 2:39pm

I did some testing with Seafile and it appears to be more suited to my needs than Nextcloud which is too big and complicated for my needs. I am currently using my rclone mount (Google Drive) for storing the libraries/data since I want to utilize unlimited storage and not worry about backups etc. It was a pain to get it working due to file permissions in docker but finally I got it working.

system · June 22, 2023, 2:39pm

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.