Rclone 1.56.0 release

Rclone 1.56.0 has been released. Find it in the rclone downloads or use rclone selfupdate to upgrade.

Some highlights of the release:

Thank you to all the contributors to this release, those who contributed code or doc fixes (46 people) or made issues or answered questions in the forum - your help is much appreciated!

My apologies for this release being a lot later than planned. Unfortunately I failed to make the release before I went on holiday, then I needed a bit of time off rclone.

If you (or your company) is enjoying rclone, please consider donating or sponsoring to keep the project sustainable, or hire me to add a feature you really want!

v1.56.0 - 2021-07-20

See commits

  • New backends
  • New commands
  • Deprecations
    • dbhashsum: Remove command deprecated a year ago (Ivan Andreev)
    • cache: Deprecate cache backend (Ivan Andreev)
  • New Features
    • rework config system so it can be used non-interactively via cli and rc API.
      • See docs in config create
      • This is a very big change to all the backends so may cause breakages - please file bugs!n
    • librclone ​port the rclone RC as a C library (lewisxy) (Nick Craig-Wood)
      • Link a C-API rclone shared object into your project
      • Use the RC as an in memory interface
      • Python example supplied
      • Also supports Android and gomobile
    • fs
      • Add --disable-http2 for global http2 disable (Nick Craig-Wood)
      • Make --dump imply -vv (Alex Chen)
      • Use binary prefixes for size and rate units (albertony)
      • Use decimal prefixes for counts (albertony)
      • Add google search widget to rclone.org (Ivan Andreev)
    • accounting: Calculate rolling average speed (Haochen Tong)
    • atexit: Terminate with non-zero status after receiving signal (Michael Hanselmann)
    • build
      • Only run event-based workflow scripts under rclone repo with manual override (Mathieu Carbou)
      • Add Android build with gomobile (x0b)
    • check: Log the hash in use like cryptcheck does (Nick Craig-Wood)
    • version: Print os/version, kernel and bitness (Ivan Andreev)
    • config
      • Prevent use of Windows reserved names in config file name (albertony)
      • Create config file in windows appdata directory by default (albertony)
      • Treat any config file paths with filename notfound as memory-only config (albertony)
      • Delay load config file (albertony)
      • Replace defaultConfig with a thread-safe in-memory implementation (Chris Macklin)
      • Allow config create and friends to take key=value parameters (Nick Craig-Wood)
      • Fixed issues with flags/options set by environment vars. (Ole Frost)
    • fshttp: Implement graceful DSCP error handling (Tyson Moore)
    • lib/http - provides an abstraction for a central http server that services can bind routes to (Nolan Woods)
      • Add --template config and flags to serve/data (Nolan Woods)
      • Add default 404 handler (Nolan Woods)
    • link: Use "off" value for unset expiry (Nick Craig-Wood)
    • oauthutil: Raise fatal error if token expired without refresh token (Alex Chen)
    • rcat: Add --size flag for more efficient uploads of known size (Nazar Mishturak)
    • serve sftp: Add --stdio flag to serve via stdio (Tom)
    • sync: Don't warn about --no-traverse when --files-from is set (Nick Gaya)
    • test makefiles
      • Add --seed flag and make data generated repeatable (Nick Craig-Wood)
      • Add log levels and speed summary (Nick Craig-Wood)
  • Bug Fixes
    • accounting: Fix startTime of statsGroups.sum (Haochen Tong)
    • cmd/ncdu: Fix out of range panic in delete (buengese)
    • config
      • Fix issues with memory-only config file paths (albertony)
      • Fix in memory config not saving on the fly backend config (Nick Craig-Wood)
    • fshttp: Fix address parsing for DSCP (Tyson Moore)
    • ncdu: Update termbox-go library to fix crash (Nick Craig-Wood)
    • oauthutil: Fix old authorize result not recognised (Cnly)
    • operations: Don't update timestamps of files in --compare-dest (Nick Gaya)
    • selfupdate: fix archive name on macos (Ivan Andreev)
  • Mount
    • Refactor before adding serve docker (Antoine GIRARD)
  • VFS
    • Add cache reset for --vfs-cache-max-size handling at cache poll interval (Leo Luan)
    • Fix modtime changing when reading file into cache (Nick Craig-Wood)
    • Avoid unnecessary subdir in cache path (albertony)
    • Fix that umask option cannot be set as environment variable (albertony)
    • Do not print notice about missing poll-interval support when set to 0 (albertony)
  • Local
    • Always use readlink to read symlink size for better compatibility (Nick Craig-Wood)
    • Add --local-unicode-normalization (and remove --local-no-unicode-normalization) (Nick Craig-Wood)
    • Skip entries removed concurrently with List() (Ivan Andreev)
  • Crypt
    • Support timestamped filenames from --b2-versions (Dominik Mydlil)
  • B2
    • Don't include the bucket name in public link file prefixes (Jeffrey Tolar)
    • Fix versions and .files with no extension (Nick Craig-Wood)
    • Factor version handling into lib/version (Dominik Mydlil)
  • Box
    • Use upload preflight check to avoid listings in file uploads (Nick Craig-Wood)
    • Return errors instead of calling log.Fatal with them (Nick Craig-Wood)
  • Drive
    • Switch to the Drives API for looking up shared drives (Nick Craig-Wood)
    • Fix some google docs being treated as files (Nick Craig-Wood)
  • Dropbox
    • Add --dropbox-batch-mode flag to speed up uploading (Nick Craig-Wood)
    • Set visibility in link sharing when --expire is set (Nick Craig-Wood)
    • Simplify chunked uploads (Alexey Ivanov)
    • Improve "own App IP" instructions (Ivan Andreev)
  • Fichier
    • Check if more than one upload link is returned (Nick Craig-Wood)
    • Support downloading password protected files and folders (Florian Penzkofer)
    • Make error messages report text from the API (Nick Craig-Wood)
    • Fix move of files in the same directory (Nick Craig-Wood)
    • Check that we actually got a download token and retry if we didn't (buengese)
  • Filefabric
    • Fix listing after change of from field from "int" to int. (Nick Craig-Wood)
  • FTP
    • Make upload error 250 indicate success (Nick Craig-Wood)
  • GCS
    • Make compatible with gsutil's mtime metadata (database64128)
    • Clean up time format constants (database64128)
  • Google Photos
    • Fix read only scope not being used properly (Nick Craig-Wood)
  • HTTP
    • Replace httplib with lib/http (Nolan Woods)
    • Clean up Bind to better use middleware (Nolan Woods)
  • Jottacloud
    • Fix legacy auth with state based config system (buengese)
    • Fix invalid url in output from link command (albertony)
    • Add no versions option (buengese)
  • Onedrive
    • Add list_chunk option (Nick Gaya)
    • Also report root error if unable to cancel multipart upload (Cnly)
    • Fix failed to configure: empty token found error (Nick Craig-Wood)
    • Make link return direct download link (Xuanchen Wu)
  • S3
    • Add --s3-no-head-object (Tatsuya Noyori)
    • Remove WebIdentityRoleProvider to fix crash on auth (Nick Craig-Wood)
    • Don't check to see if remote is object if it ends with / (Nick Craig-Wood)
    • Add SeaweedFS (Chris Lu)
    • Update Alibaba OSS endpoints (Chuan Zh)
  • SFTP
    • Fix performance regression by re-enabling concurrent writes (Nick Craig-Wood)
    • Expand tilde and environment variables in configured known_hosts_file (albertony)
  • Tardigrade
    • Upgrade to uplink v1.4.6 (Caleb Case)
    • Use negative offset (Caleb Case)
    • Add warning about too many open files (acsfer)
  • WebDAV
    • Fix sharepoint auth over http (Nick Craig-Wood)
    • Add headers option (Antoon Prins)
5 Likes

Wow! This is quite the release.

Can you expound more on --onedrive-list-chunk?

Is that per directory? Or the whole remote? Woudl setting it to a number >> # of files be the same as ListR support?

I have a about 125,000 files in OneDrive across many folders (photo library) so walking them all is very slow. I would love to speed that up with this setting.

Also, I assume:

  • selfupdate: fix archive name on macos (Ivan Andreev)

is why it isn't working on macOS? But it was great to run on my linux machines. Yay!


Thanks for everyone's hard work!

Rclone fetches the listing in 1000 item chunks normally. You can now play with that 1000 number - increasing it might get you a bit more performance with large directories, decreasing it might avoid timeouts.

So it is per directory and it doesn't give you ListR support, alas.

Yes I think so.

I am not sure if this counts as a bug or a change but in 1.55.1, the hash for DropBox was called DropboxHash and now it is dropbox.

Was this an intended change? It breaks some of my workflows that reuse a prior hash but it is far from insurmountable (I will either rehash or manually rename in the saved JSON) but it is good to know these things if possible

it's intended
old names are removed from docs and reports but will be silently supported in all commands e.g. rclone hashsum DropboxHash blah:duh for at least a year.
please update your scripts if you hardcoded these names. if you can't update, stay with 1.55.1

1 Like

I don't have it hard coded. It just uses older lsjson results to compare newer. But I will update it.

Is there a reason this change was not documented? It is cleaner and I like it but the names are more than just internal; they can be user facing so I would have expected to see that.

1 Like

new hash names are mentioned at rclone hashsum

i don't know why the patch is not in changelog Documentation

frankly, i don't even know who where or how produces changelog. i just submit pull requests.

1 Like

@ncw
Do we have a forum topic/section with announcements from developers?
You could ask developers to post headsup notes there at the time you accept their patch for merge, 1-3 months before it gets released, at your discretion.

The first draft of the changelog is made with make_changelog.py but this only looks at the first line of the commit.

I then spend an hour tidying the changelog, removing irrelevant detail for users and expanding on important stuff.

I did miss the hash name change though - sorry!

That is a good idea. A sort of beta channel heads-up. How about "Upcoming changes"?

What do you think @Animosity022 ?

1 Like

I think that's a great idea!