Is anyone using Rclone with Joplin, recommendations for mount settings

Help and Support
1

What is the problem you are having with rclone?

I am looking to ue rclone with Joplin. Joplin has built-in sync capabilities with webdav. Joplin can also sync to a local filesystem. Does anyone have any experience syncing Joplin to a local filesystem (rclone mount) and letting rclone do the syncing?

I tried it with the below mounting settings and it mainly appeared to work. The transfer stopped once with a Joplin API error (see the corresponding rclone error) however I was able to start it again and it finished successfully.

What is your rclone version (output from rclone version)

rclone v1.54.0

  • os/arch: linux/amd64
  • go version: go1.15.7

Which OS you are using and how many bits (eg Windows 7, 64 bit)

KDE Neon 5.21.4, 64bit

Which cloud storage system are you using? (eg Google Drive)

Nextcloud (Webdav)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

/usr/bin/rclone mount \
>     --config=rclone.conf \
>     -v \
>     --cache-dir=/tmp/rclone \
>     --uid=1000 \
>     --dir-perms=0777 \
>     --gid=1000 \
>     --vfs-cache-mode=writes \
>     --attr-timeout=1s \
> remote: /path

A log from the command with the -vv flag

2021/05/11 13:33:31 ERROR : id.md: Failed to copy: Put "https://host/remote.php/dav/files/id": http2: Transport: cannot retry err [http2: Transport received Server's graceful shutdown GOAWAY] after Request.Body was written; define Request.GetBody to avoid this error

2021/05/11 13:33:31 ERROR : id.md: vfs cache: failed to upload try #1, will retry in 10s: vfs cache: failed to transfer file from cache to remote: Put "https://host/remote.php/dav/files/id": http2: Transport: cannot retry err [http2: Transport received Server's graceful shutdown GOAWAY] after Request.Body was written; define Request.GetBody to avoid this error

Joplin is a note taking app and it syncs small .md files for each note. Are my rclone mount settings appropriate for this usecase? Should I add or remove anything?

2

I am testing another sync. One error I am seeing often is:

<7>DEBUG : file.txt: vfs cache: delaying writeback as --transfers exceeded

Is this something to be concerned about, can I adjust the settings to prevent it?

3

That looks like the upload failed for some reason. Since you are using --vfs-cache-mode=writes it will get retried shortly.

That is OK - rclone won't transfer more than --transfers (4 by default) files at once so it will queue them.

4

Thanks for the reply.

I am trying the same application with a similar config cross-platform on Android without vfs caching. I am seeing this error:

Date ERROR: file: WriteFileHandle: Can't open for write without O_TRUNC on existing fie without --vfs-cache-mode >- writes.

I would prefer not to use vfs caching if possible on Android due to resource constraints. Is this message telling me that I cannot write at all without using vfs caching? What is O_TRUNC in this context and how does it relate to vfs caching?

5

It's how whatever application you are using is trying to write the file out and there isn't anyway around it (other than changing the application perhaps).

6

Thanks for the reply. Have you seen this error message before? Isn't filesystem access and operations handled at a lower level by the OS? Are you saying that rclone mount is not compatible with the way certain applications call OS resources to write files? Do you have any further information or are aware of documentation on the specifics so I can look for a solution?

I found this man page which describes O_TRUNC:

https://www.man7.org/linux/man-pages/man2/open.2.html

O_TRUNC
              If the file already exists and is a regular file and the
              access mode allows writing (i.e., is O_RDWR or O_WRONLY)
              it will be truncated to length 0.  If the file is a FIFO
              or terminal device file, the O_TRUNC flag is ignored.
              Otherwise, the effect of O_TRUNC is unspecified.

In this case, yes the file does exits, it is a regular file and it has write permissions (0644). Is rclone here asking for O_TRUNC to be set on this file by the application in order to allow it to be written to?

7

Turning on vfs-cache-mode writes would fix the issue.

It's how the application is writing the file and nothing in rclone would fix that unfortunately.

It's like running a copy command as that would work fine, but something like ffmpeg would not as it needs writes.

8

The higher you raise --vfs-cache-mode the more compatible rclone becomes. with --vfs-cache-mode off rclone can't go back and adjust data it has already written.

The details are in the vfs cache mode docs.

9

I've been using Joplin for a few months now. I use Joplin's 'local' sync. But I use Syncthing to sync a shared folder including my Joplin notes to several local devices:

  • 2 android devices to their internal encrypted storage
  • 1 Raspberry pi 'server' to encrypted home directory
  • 1 Windows laptop to bitlocker drive

All devices can also connect via ZeroTier. Then I use rclone on a scheduled task every few hours to sync it to a Backblaze bucket from the Windows laptop.

In your scenario you do not have a local copy of your notes at all, so network is required to access them?

10

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.