Rclone v1.65.0 release

Releases
1

Rclone 1.65.0 has been released. Find it in the rclone downloads or use rclone selfupdate to upgrade.

Some highlights of the release:

  • New Backends
  • New commands
    • serve s3: Let rclone act as an S3 compatible server (experimental)
    • nfsmount: macOS mount using NFS without FUSE
    • serve nfs: to serve a remote for use by nfsmount and other uses
  • New Features
    • --metadata-mapper to transform metatadata with a user supplied program
    • Use less memory when doing multithread uploads

Thank you to all the contributors to this release, those who contributed code or doc fixes (36 people) or made issues or answered questions in the forum - your help is much appreciated!

If you’re working in the industry and you feel your company could benefit from a support contract please get in touch at sales@rclone.com or if you’re interested in advertising with us then please drop us a message to sponsorship@rclone.com.

Please consider donating or sponsoring to keep the project sustainable.

v1.65.0 - 2023-11-26

See commits

  • New backends
  • New commands
    • serve s3: Let rclone act as an S3 compatible server (Mikubill, Artur Neumann, Saw-jan, Nick Craig-Wood)
    • nfsmount: mount command to provide mount mechanism on macOS without FUSE (Saleh Dindar)
    • serve nfs: to serve a remote for use by nfsmount (Saleh Dindar)
  • New Features
    • install.sh: Clean up temp files in install script (Jacob Hands)
    • build
      • Update all dependencies (Nick Craig-Wood)
      • Refactor version info and icon resource handling on windows (albertony)
    • doc updates (albertony, alfish2000, asdffdsazqqq, Dimitri Papadopoulos, Herby Gillot, Joda Stößer, Manoj Ghosh, Nick Craig-Wood)
    • Implement --metadata-mapper to transform metatadata with a user supplied program (Nick Craig-Wood)
    • Add ChunkWriterDoesntSeek feature flag and set it for b2 (Nick Craig-Wood)
    • lib/http: Export basic go string functions for use in --template (Gabriel Espinoza)
    • makefile: Use POSIX compatible install arguments (Mina Galić)
    • operations
      • Use less memory when doing multithread uploads (Nick Craig-Wood)
      • Implement --partial-suffix to control extension of temporary file names (Volodymyr)
    • rc
      • Add operations/check to the rc API (Nick Craig-Wood)
      • Always report an error as JSON (Nick Craig-Wood)
      • Set Last-Modified header for files served by --rc-serve (Nikita Shoshin)
    • size: Dont show duplicate object count when less than 1k (albertony)
  • Bug Fixes
    • fshttp: Fix --contimeout being ignored (你知道未来吗)
    • march: Fix excessive parallelism when using --no-traverse (Nick Craig-Wood)
    • ncdu: Fix crash when re-entering changed directory after rescan (Nick Craig-Wood)
    • operations
      • Fix overwrite of destination when multi-thread transfer fails (Nick Craig-Wood)
      • Fix invalid UTF-8 when truncating file names when not using --inplace (Nick Craig-Wood)
    • serve dnla: Fix crash on graceful exit (wuxingzhong)
  • Mount
    • Disable mount for freebsd and alias cmount as mount on that platform (Nick Craig-Wood)
  • VFS
    • Add --vfs-refresh flag to read all the directories on start (Beyond Meat)
    • Implement Name() method in WriteFileHandle and ReadFileHandle (Saleh Dindar)
    • Add go-billy dependency and make sure vfs.Handle implements billy.File (Saleh Dindar)
    • Error out early if can't upload 0 length file (Nick Craig-Wood)
  • Local
    • Fix copying from Windows Volume Shadows (Nick Craig-Wood)
  • Azure Blob
    • Add support for cold tier (Ivan Yanitra)
  • B2
    • Implement "rclone backend lifecycle" to read and set bucket lifecycles (Nick Craig-Wood)
    • Implement --b2-lifecycle to control lifecycle when creating buckets (Nick Craig-Wood)
    • Fix listing all buckets when not needed (Nick Craig-Wood)
    • Fix multi-thread upload with copyto going to wrong name (Nick Craig-Wood)
    • Fix server side chunked copy when file size was exactly --b2-copy-cutoff (Nick Craig-Wood)
    • Fix streaming chunked files an exact multiple of chunk size (Nick Craig-Wood)
  • Box
    • Filter more EventIDs when polling (David Sze)
    • Add more logging for polling (David Sze)
    • Fix performance problem reading metadata for single files (Nick Craig-Wood)
  • Drive
    • Add read/write metadata support (Nick Craig-Wood)
    • Add support for SHA-1 and SHA-256 checksums (rinsuki)
    • Add --drive-show-all-gdocs to allow unexportable gdocs to be server side copied (Nick Craig-Wood)
    • Add a note that --drive-scope accepts comma-separated list of scopes (Keigo Imai)
    • Fix error updating created time metadata on existing object (Nick Craig-Wood)
    • Fix integration tests by enabling metadata support from the context (Nick Craig-Wood)
  • Dropbox
    • Factor batcher into lib/batcher (Nick Craig-Wood)
    • Fix missing encoding for rclone purge (Nick Craig-Wood)
  • Google Cloud Storage
    • Fix 400 Bad request errors when using multi-thread copy (Nick Craig-Wood)
  • Googlephotos
    • Implement batcher for uploads (Nick Craig-Wood)
  • Hdfs
    • Added support for list of namenodes in hdfs remote config (Tayo-pasedaRJ)
  • HTTP
    • Implement set backend command to update running backend (Nick Craig-Wood)
    • Enable methods used with WebDAV (Alen Šiljak)
  • Jottacloud
    • Add support for reading and writing metadata (albertony)
  • Onedrive
    • Implement ListR method which gives --fast-list support (Nick Craig-Wood)
      • This must be enabled with the --onedrive-delta flag
  • Quatrix
    • Add partial upload support (Oksana Zhykina)
    • Overwrite files on conflict during server-side move (Oksana Zhykina)
  • S3
    • Add Linode provider (Nick Craig-Wood)
    • Add docs on how to add a new provider (Nick Craig-Wood)
    • Fix no error being returned when creating a bucket we don't own (Nick Craig-Wood)
    • Emit a debug message if anonymous credentials are in use (Nick Craig-Wood)
    • Add --s3-disable-multipart-uploads flag (Nick Craig-Wood)
    • Detect looping when using gcs and versions (Nick Craig-Wood)
  • SFTP
    • Implement --sftp-copy-is-hardlink to server side copy as hardlink (Nick Craig-Wood)
  • Smb
    • Fix incorrect about size by switching to github.com/cloudsoda/go-smb2 fork (Nick Craig-Wood)
    • Fix modtime of multithread uploads by setting PartialUploads (Nick Craig-Wood)
  • WebDAV
    • Added an rclone vendor to work with rclone serve webdav (Adithya Kumar)
8 Likes
3

This is great news for MacOS users and potentially perfect alternative to FUSE.

What exactly is nfsmount command? Is it documented? I can not find anything in docs.

Or I have to manually spin rclone serve nfs and then mount it from OS level?

4

Regarding the read-only nfsmount restriction for no cache, is it possible to make it so you can move and delete? Just not write? Or is that part of the limitations?

5

To see the docs you'll need to run

rclone nfsmount --help

On macOS.

I only realised when writing this forum post that it doesn't have docs. There is reason for that is the machine that generates the docs runs Linux and this command doesn't exist on Linux!

I'll fix this in due course.

You can do that , but nfsmount automates that process.

From what I remember writing files needs the cache so move and delete should be ok.

6

It doesn't work with them. Should I create a feature request?

8

What is the bucket name provided by rclone serve s3

9 
$ rclone --version
rclone v1.65.0
- os/version: darwin 14.1.1 (64 bit)
- os/kernel: 23.1.0 (x86_64)
- os/type: darwin
- os/arch: amd64
- go/version: go1.21.4
- go/linking: dynamic
- go/tags: cmount


$ rclone nfsmount --help
Error: unknown command "nfsmount" for "rclone"
Run 'rclone --help' for usage.
You could use 'rclone selfupdate' to get latest features.

2023/11/27 05:55:17 Fatal error: unknown command "nfsmount" for "rclone"

Thank you for this explanation.

10

You create whatever buckets you need in the normal way.

11

The build constraints for rclone nfsmount are this

//go:build darwin && !cmount
// +build darwin,!cmount

You will be using a cmount build if you are using the official rclone so it won't be in there at all! I think that is a mistake

I've corrected that here which should have rclone nfsmount in.

v1.66.0-beta.7552.fc07df85c.fix-nfsmount on branch fix-nfsmount (uploaded in 15-30 mins)

Along with making it work on all unix OSes which means the docs will appear!

I don't think I did a very good job reviewing this, so I've marked it as experimental also - will be interested to see if you can make it work.

1 Like
12

I will definitely test it including macOS characters encoding quirks.

13

FYI - automated build does not create osx-amd64 file.

I can build it manually though without issues.

14

It should do, it just failed. It should build OK though!

15

Could you clarify the use case for this? Thanks!

16

It allows you to prime or pre-read all the directory structure as part of the mount command rather than running a post command after.

If you want to direcetory structure to be walked, you can use this flag.

17

Hi there, thanks for the new release.
This is the output of installation in container ran with 'ubuntu' image:

curl https://rclone.org/install.sh | bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4734  100  4734    0     0   2912      0  0:00:01  0:00:01 --:--:--  2913
Archive:  rclone-current-linux-amd64.zip
   creating: tmp_unzip_dir_for_rclone/rclone-v1.65.0-linux-amd64/
  inflating: tmp_unzip_dir_for_rclone/rclone-v1.65.0-linux-amd64/README.txt  [text]  
  inflating: tmp_unzip_dir_for_rclone/rclone-v1.65.0-linux-amd64/rclone  [binary]
  inflating: tmp_unzip_dir_for_rclone/rclone-v1.65.0-linux-amd64/git-log.txt  [text]  
  inflating: tmp_unzip_dir_for_rclone/rclone-v1.65.0-linux-amd64/rclone.1  [text]  
  inflating: tmp_unzip_dir_for_rclone/rclone-v1.65.0-linux-amd64/README.html  [text]  
mandb not found. The rclone man docs will not be installed.
bash: line 200: errors: No such file or directory

 has successfully installed.
Now run "rclone config" for setup. Check https://rclone.org/docs/ for more details

I started to see this error right after the release. Everything was correct yesterday. So I guess something changed in the meaning of script.

1 Like
18

I found this is caused by install.sh: Clean up temp files in install script · jahands/rclone@fb95483 · GitHub but still dont know why and how to fix.

Got it

install.sh first create tmp_dir and enter to it by

tmp_dir=$(mktemp -d 2>/dev/null || mktemp -d -t 'rclone-install.XXXXXXXXXX')
cd "$tmp_dir"

However, bash redirection to a file errors for variable assignment happens after cleanup tmp_dir like below

#cleanup
rm -rf "$tmp_dir"

version=$(rclone --version 2>>errors | head -n 1)
1 Like
19

Thanks for tracking it down.

I've fixed that here

I'll push that fix out in the point release.

I think it is harmless so it can wait until then.

20

Thanks, mate. Definitely, it can wait.

21

Are local directories broken for Windows in 1.65?

If I do the below command on 1.65 I get Directory Not Found, but on 1.63 it lists the dirs. I am also having an issue creating local directories.

 .\rclone.exe lsd C:\

If you'd like me to open a new topic, please let me know.