Rclone v1.66 release

Releases
1

Rclone 1.66.0 has been released. Find it in the rclone downloads or use rclone selfupdate to upgrade.

Some highlights of the release:

  • Rclone will now sync directory modification times if the backend supports it.
  • Rclone will now sync directory metadata if the backend supports it when -M/--metadata is in use.
    • See the overview and look for the D flags in the Metadata column to see which backends support it.
  • Bisync has received many updates see below for more details or bisync's changelog

Thank you to all the contributors to this release, those who contributed code or doc fixes (34 people) or made issues or answered questions in the forum - your help is much appreciated!

If you’re working in the industry and you feel your company could benefit from a support contract please get in touch at sales@rclone.com or if you’re interested in advertising with us then please drop us a message to sponsorship@rclone.com.

Please consider donating or sponsoring to keep the project sustainable.

v1.66.0 - 2024-03-10

See commits

  • Removed backends
    • amazonclouddrive: Remove Amazon Drive backend code and docs (Nick Craig-Wood)
  • New Features
    • backend
      • Add description field for all backends (Paul Stern)
    • build
      • Update to go1.22 and make go1.20 the minimum required version (Nick Craig-Wood)
      • Fix CVE-2024-24786 by upgrading google.golang.org/protobuf (Nick Craig-Wood)
    • check: Respect --no-unicode-normalization and --ignore-case-sync for --checkfile (nielash)
    • cmd: Much improved shell auto completion which reduces the size of the completion file and works faster (Nick Craig-Wood)
    • doc updates (albertony, ben-ba, Eli, emyarod, huajin tong, Jack Provance, kapitainsky, keongalvin, Nick Craig-Wood, nielash, rarspace01, rzitzer, Tera, Vincent Murphy)
    • fs: Add more detailed logging for file includes/excludes (Kyle Reynolds)
    • lsf
      • Add --time-format flag (nielash)
      • Make metadata appear for directories (Nick Craig-Wood)
    • lsjson: Make metadata appear for directories (Nick Craig-Wood)
    • rc
      • Add srcFs and dstFs to core/stats and core/transferred stats (Nick Craig-Wood)
      • Add operations/hashsum to the rc as rclone hashsum equivalent (Nick Craig-Wood)
      • Add config/paths to the rc as rclone config paths equivalent (Nick Craig-Wood)
    • sync
      • Optionally report list of synced paths to file (nielash)
      • Implement directory sync for mod times and metadata (Nick Craig-Wood)
      • Don't set directory modtimes if already set (nielash)
      • Don't sync directory modtimes from backends which don't have directories (Nick Craig-Wood)
  • Bug Fixes
    • backend
      • Make backends which use oauth implement the Shutdown and shutdown the oauth properly (rkonfj)
    • bisync
      • Handle unicode and case normalization consistently (nielash)
      • Partial uploads known issue on local/ftp/sftp has been resolved (unless using --inplace) (nielash)
      • Fixed handling of unicode normalization and case insensitivity, support for --fix-case, --ignore-case-sync, --no-unicode-normalization (nielash)
      • Bisync no longer fails to find the correct listing file when configs are overridden with backend-specific flags. (nielash)
    • nfsmount
      • Fix exit after external unmount (nielash)
      • Fix --volname being ignored (nielash)
    • operations
      • Fix renaming a file on macOS (nielash)
      • Fix case-insensitive moves in operations.Move (nielash)
      • Fix TestCaseInsensitiveMoveFileDryRun on chunker integration tests (nielash)
      • Fix TestMkdirModTime test (Nick Craig-Wood)
      • Fix TestSetDirModTime for backends with SetDirModTime but not Metadata (Nick Craig-Wood)
      • Fix typo in log messages (nielash)
    • serve nfs: Fix writing files via Finder on macOS (nielash)
    • serve restic: Fix error handling (Michael Eischer)
    • serve webdav: Fix --baseurl without leading / (Nick Craig-Wood)
    • stats: Fix race between ResetCounters and stopAverageLoop called from time.AfterFunc (Nick Craig-Wood)
    • sync
      • --fix-case flag to rename case insensitive dest (nielash)
      • Use operations.DirMove instead of sync.MoveDir for --fix-case (nielash)
    • systemd: Fix detection and switch to the coreos package everywhere rather than having 2 separate libraries (Anagh Kumar Baranwal)
  • Mount
    • Fix macOS not noticing errors with --daemon (Nick Craig-Wood)
    • Notice daemon dying much quicker (Nick Craig-Wood)
  • VFS
    • Fix unicode normalization on macOS (nielash)
  • Bisync
    • Copies and deletes are now handled in one operation instead of two (nielash)
    • --track-renames and --backup-dir are now supported (nielash)
    • Final listings are now generated from sync results, to avoid needing to re-list (nielash)
    • Bisync is now much more resilient to changes that happen during a bisync run, and far less prone to critical errors / undetected changes (nielash)
    • Bisync is now capable of rolling a file listing back in cases of uncertainty, essentially marking the file as needing to be rechecked next time. (nielash)
    • A few basic terminal colors are now supported, controllable with --color (AUTO|NEVER|ALWAYS) (nielash)
    • Initial listing snapshots of Path1 and Path2 are now generated concurrently, using the same "march" infrastructure as check and sync, for performance improvements and less risk of error. (nielash)
    • --resync is now much more efficient (especially for users of --create-empty-src-dirs) (nielash)
    • Google Docs (and other files of unknown size) are now supported (with the same options as in sync) (nielash)
    • Equality checks before a sync conflict rename now fall back to cryptcheck (when possible) or --download, (nielash)
      instead of of --size-only, when check is not available.
    • Bisync now fully supports comparing based on any combination of size, modtime, and checksum, lifting the prior restriction on backends without modtime support. (nielash)
    • Bisync now supports a "Graceful Shutdown" mode to cleanly cancel a run early without requiring --resync. (nielash)
    • New --recover flag allows robust recovery in the event of interruptions, without requiring --resync. (nielash)
    • A new --max-lock setting allows lock files to automatically renew and expire, for better automatic recovery when a run is interrupted. (nielash)
    • Bisync now supports auto-resolving sync conflicts and customizing rename behavior with new --conflict-resolve, --conflict-loser, and --conflict-suffix flags. (nielash)
    • A new --resync-mode flag allows more control over which version of a file gets kept during a --resync. (nielash)
    • Bisync now supports --retries and --retries-sleep (when --resilient is set.) (nielash)
    • Clarify file operation directions in dry-run logs (Kyle Reynolds)
  • Local
    • Fix cleanRootPath on Windows after go1.21.4 stdlib update (nielash)
    • Implement setting modification time on directories (nielash)
    • Implement modtime and metadata for directories (Nick Craig-Wood)
    • Fix setting of btime on directories on Windows (Nick Craig-Wood)
    • Delete backend implementation of Purge to speed up and make stats (Nick Craig-Wood)
    • Support metadata setting and mapping on server side Move (Nick Craig-Wood)
  • Cache
    • Implement setting modification time on directories (if supported by wrapped remote) (nielash)
    • Implement setting metadata on directories (Nick Craig-Wood)
  • Crypt
    • Implement setting modification time on directories (if supported by wrapped remote) (nielash)
    • Implement setting metadata on directories (Nick Craig-Wood)
    • Improve handling of undecryptable file names (nielash)
    • Add missing error check spotted by linter (Nick Craig-Wood)
  • Azure Blob
    • Implement --azureblob-delete-snapshots (Nick Craig-Wood)
  • B2
    • Clarify exactly what --b2-download-auth-duration does in the docs (Nick Craig-Wood)
  • Chunker
    • Implement setting modification time on directories (if supported by wrapped remote) (nielash)
    • Implement setting metadata on directories (Nick Craig-Wood)
  • Combine
    • Implement setting modification time on directories (if supported by wrapped remote) (nielash)
    • Implement setting metadata on directories (Nick Craig-Wood)
    • Fix directory metadata error on upstream root (nielash)
    • Fix directory move across upstreams (nielash)
  • Compress
    • Implement setting modification time on directories (if supported by wrapped remote) (nielash)
    • Implement setting metadata on directories (Nick Craig-Wood)
  • Drive
    • Implement setting modification time on directories (nielash)
    • Implement modtime and metadata setting for directories (Nick Craig-Wood)
    • Support metadata setting and mapping on server side Move,Copy (Nick Craig-Wood)
  • FTP
    • Fix mkdir with rsftp which is returning the wrong code (Nick Craig-Wood)
  • Hasher
    • Implement setting modification time on directories (if supported by wrapped remote) (nielash)
    • Implement setting metadata on directories (Nick Craig-Wood)
    • Fix error from trying to stop an already-stopped db (nielash)
    • Look for cached hash if passed hash unexpectedly blank (nielash)
  • Imagekit
    • Updated docs and web content (Harshit Budhraja)
    • Updated overview - supported operations (Harshit Budhraja)
  • Mega
    • Fix panic with go1.22 (Nick Craig-Wood)
  • Netstorage
    • Fix Root to return correct directory when pointing to a file (Nick Craig-Wood)
  • Onedrive
    • Add metadata support (nielash)
  • Opendrive
    • Fix moving file/folder within the same parent dir (nielash)
  • Oracle Object Storage
    • Support backend restore command (Nikhil Ahuja)
    • Support workload identity authentication for OKE (Anders Swanson)
  • Protondrive
    • Fix encoding of Root method (Nick Craig-Wood)
  • Quatrix
    • Fix Content-Range header (Volodymyr)
    • Add option to skip project folders (Oksana Zhykina)
    • Fix Root to return correct directory when pointing to a file (Nick Craig-Wood)
  • S3
    • Add --s3-version-deleted to show delete markers in listings when using versions. (Nick Craig-Wood)
    • Add IPv6 support with option --s3-use-dual-stack (Anthony Metzidis)
    • Copy parts in parallel when doing chunked server side copy (Nick Craig-Wood)
    • GCS provider: fix server side copy of files bigger than 5G (Nick Craig-Wood)
    • Support metadata setting and mapping on server side Copy (Nick Craig-Wood)
  • Seafile
    • Fix download/upload error when FILE_SERVER_ROOT is relative (DanielEgbers)
    • Fix Root to return correct directory when pointing to a file (Nick Craig-Wood)
  • SFTP
    • Implement setting modification time on directories (nielash)
    • Set directory modtimes update on write flag (Nick Craig-Wood)
    • Shorten wait delay for external ssh binaries now that we are using go1.20 (Nick Craig-Wood)
  • Swift
    • Avoid unnecessary container versioning check (Joe Cai)
  • Union
    • Implement setting modification time on directories (if supported by wrapped remote) (nielash)
    • Implement setting metadata on directories (Nick Craig-Wood)
  • WebDAV
    • Reduce priority of chunks upload log (Gabriel Ramos)
    • owncloud: Add config owncloud_exclude_shares which allows to exclude shared files and folders when listing remote resources (Thomas MĂĽller)
12 Likes
3

Thank you all but special thx to @nielash for making mounting on macOS finally working 100%.

After years of unresolved issues now NFC/NFD problem has been fixed.

In addition rclone nfsmount allows running mount without problematic macFUSE support.

Great day for macOS users:)

5 Likes
4

Was this feature not released? I didn't find it in the changelog, just to confirm.

5

Nope. You can tell as the issue is still open so not even merged yet.

6

I missed merging that - sorry! I'll take a look tomorrow.

2 Likes
7

A big thank!! Awesome job @nielash (also, from a mac user).

2 Likes
8

Will it be included in the upcoming version 1.66.1?

9

@cberni I merged that now. It is too big for 1.66.1 it will have to wait from 1.67 I think.

1 Like
10

Another shoutout to @nielash (and of course @ncw) for the work on bisync. I am the author of syncrclone and I may even move over to bisync. It is on my todo list to update my comparison shortly.

I still prefer the compare-current-then-resolve-with-past (syncrclone) to use-past-to-generate-diffs-then-resolve (bisync) but they yield the same thing in the end; it's just that --resync is implicit and interruptions don't lead to as many redundant transfers.

But there are real benefits to using the native solution!

1 Like
11

Thank you, @jwink3101! And thanks again for your insights which were especially helpful with the new --compare feature! :slightly_smiling_face:

Recovering from interruptions was a major focus of v1.66 and Bisync should now handle this a lot better than before. With the following flag combo, I think you should find that you almost never need to --resync:

--resilient --recover --max-lock 2m --conflict-resolve newer

We considered adding an actual --auto-resync option (and may still at some point), but user feedback was that --recover was a safer way of accomplishing the same goal (without risk of files getting inadvertently un-deleted.)

If you have a chance to try it out, I welcome your feedback!

1 Like
12

putting in my obligatory comment about remote caching. am i still the only one who's interested in a local cache of the remote file system for escapting I/O throttling? :slight_smile:

13

You can always develop it, find someone to develop it or sponsor it if you really want it. Not sure why you'd post to the release thread as it's not really relevant.

14

I have a vfscache backend on a dev branch which is the solution to this problem. One day I will get round to finishing it I'm sure!

1 Like
15

thanks @ncw! waiting for that one day :smirk: boo @Animosity022 :-1:

16

I know. I hate trying to be a good mod and keeping things on topic and avoiding things unrelated to the release. @ncw is generally too nice and repsonds to everyone so it's a lose lose sometimes for me.

So a healthy boo to you as well for that comment.

4 Likes
17

in the words of the great theologian charlie sheen: I understand :grin: no for real though, keep up the good work; we appreciate you :sunglasses: