Rclone 1.43 release

Releases
1

Rclone 1.43 has been released after a slightly extended release cycle.

Highlights include the Jottacloud backend; the copyurl command; the –progress/-P flag for interactive stats; a reworked backend config system which can configure all the options in the config file, as flags or as environment variables and many more improvements and fixes!

Download from: https://rclone.org/downloads/

Thank you to all the contributors to this release, those who contributed code (25 people!) or made issues or answered questions in the forum - your help is much appreciated!

Update if you are using the hubic or azureblob backends then please update to v1.43.1 (see Rclone 1.43.1 release). If you aren’t, no need to upgrade.

v1.43 - 2018-09-01

  • New backends
    • Jottacloud (Sebastian Bünger)
  • New commands
    • copyurl: copies a URL to a remote (Denis)
  • New Features
    • Reworked config for backends (Nick Craig-Wood)
      • All backend config can now be supplied by command line, env var or config file
      • Advanced section in the config wizard for the optional items
      • A large step towards rclone backends being usable in other go software
      • Allow on the fly remotes with :backend: syntax
    • Stats revamp
      • Add --progress/-P flag to show interactive progress (Nick Craig-Wood)
      • Show the total progress of the sync in the stats (Nick Craig-Wood)
      • Add --stats-one-line flag for single line stats (Nick Craig-Wood)
    • Added weekday schedule into --bwlimit (Mateusz)
    • lsjson: Add option to show the original object IDs (Fabian Möller)
    • serve webdav: Make Content-Type without reading the file and add --etag-hash (Nick Craig-Wood)
    • build
      • Build macOS with native compiler (Nick Craig-Wood)
      • Update to use go1.11 for the build (Nick Craig-Wood)
    • rc
      • Added core/stats to return the stats (reddi1)
    • version --check: Prints the current release and beta versions (Nick Craig-Wood)
  • Bug Fixes
    • accounting
      • Fix time to completion estimates (Nick Craig-Wood)
      • Fix moving average speed for file stats (Nick Craig-Wood)
    • config: Fix error reading password from piped input (Nick Craig-Wood)
    • move: Fix --delete-empty-src-dirs flag to delete all empty dirs on move (ishuah)
  • Mount
    • Implement --daemon-timeout flag for OSXFUSE (Nick Craig-Wood)
    • Fix mount --daemon not working with encrypted config (Alex Chen)
    • Clip the number of blocks to 2^32-1 on macOS - fixes borg backup (Nick Craig-Wood)
  • VFS
    • Enable vfs-read-chunk-size by default (Fabian Möller)
    • Add the vfs/refresh rc command (Fabian Möller)
    • Add non recursive mode to vfs/refresh rc command (Fabian Möller)
    • Try to seek buffer on read only files (Fabian Möller)
  • Local
    • Fix crash when deprecated --local-no-unicode-normalization is supplied (Nick Craig-Wood)
    • Fix mkdir error when trying to copy files to the root of a drive on windows (Nick Craig-Wood)
  • Cache
    • Fix nil pointer deref when using lsjson on cached directory (Nick Craig-Wood)
    • Fix nil pointer deref for occasional crash on playback (Nick Craig-Wood)
  • Crypt
    • Fix accounting when checking hashes on upload (Nick Craig-Wood)
  • Amazon Cloud Drive
    • Make very clear in the docs that rclone has no ACD keys (Nick Craig-Wood)
  • Azure Blob
    • Add connection string and SAS URL auth (Nick Craig-Wood)
    • List the container to see if it exists (Nick Craig-Wood)
    • Port new Azure Blob Storage SDK (sandeepkru)
    • Added blob tier, tier between Hot, Cool and Archive. (sandeepkru)
    • Remove leading / from paths (Nick Craig-Wood)
  • B2
    • Support Application Keys (Nick Craig-Wood)
    • Remove leading / from paths (Nick Craig-Wood)
  • Box
    • Fix upload of > 2GB files on 32 bit platforms (Nick Craig-Wood)
    • Make --box-commit-retries flag defaulting to 100 to fix large uploads (Nick Craig-Wood)
  • Drive
    • Add --drive-keep-revision-forever flag (lewapm)
    • Handle gdocs when filtering file names in list (Fabian Möller)
    • Support using --fast-list for large speedups (Fabian Möller)
  • FTP
    • Fix Put mkParentDir failed: 521 for BunnyCDN (Nick Craig-Wood)
  • Google Cloud Storage
    • Fix index out of range error with --fast-list (Nick Craig-Wood)
  • Jottacloud
    • Fix MD5 error check (Oliver Heyme)
    • Handle empty time values (Martin Polden)
    • Calculate missing MD5s (Oliver Heyme)
    • Docs, fixes and tests for MD5 calculation (Nick Craig-Wood)
    • Add optional MimeTyper interface. (Sebastian Bünger)
    • Implement optional About interface (for df support). (Sebastian Bünger)
  • Mega
    • Wait for events instead of arbitrary sleeping (Nick Craig-Wood)
    • Add --mega-hard-delete flag (Nick Craig-Wood)
    • Fix failed logins with upper case chars in email (Nick Craig-Wood)
  • Onedrive
    • Shared folder support (Yoni Jah)
    • Implement DirMove (Cnly)
    • Fix rmdir sometimes deleting directories with contents (Nick Craig-Wood)
  • Pcloud
    • Delete half uploaded files on upload error (Nick Craig-Wood)
  • Qingstor
    • Remove leading / from paths (Nick Craig-Wood)
  • S3
    • Fix index out of range error with --fast-list (Nick Craig-Wood)
    • Add --s3-force-path-style (Nick Craig-Wood)
    • Add support for KMS Key ID (bsteiss)
    • Remove leading / from paths (Nick Craig-Wood)
  • Swift
    • Add storage_policy (Ruben Vandamme)
    • Make it so just storage_url or auth_token can be overidden (Nick Craig-Wood)
    • Fix server side copy bug for unusal file names (Nick Craig-Wood)
    • Remove leading / from paths (Nick Craig-Wood)
  • WebDAV
    • Ensure we call MKCOL with a URL with a trailing / for QNAP interop (Nick Craig-Wood)
    • If root ends with / then don’t check if it is a file (Nick Craig-Wood)
    • Don’t accept redirects when reading metadata (Nick Craig-Wood)
    • Add bearer token (Macaroon) support for dCache (Nick Craig-Wood)
    • Document dCache and Macaroons (Onno Zweers)
    • Sharepoint recursion with different depth (Henning)
    • Attempt to remove failed uploads (Nick Craig-Wood)
  • Yandex
    • Fix listing/deleting files in the root (Nick Craig-Wood)
11 Likes
2

Thanks for this.

Just to confirm, how will the vfs-read-chunk-size, since its now enabled by default, option affect those who are using the cache backend instead? Will it work alongside it without any issues or is there an extra config change needed for it?

3

It shouldn’t make any difference to cache users. It will probably help there too!

4

Thanks, I love copyurl

5

I really love the new detailed -vv log.

Transferred: 246.945M / 151.308 GBytes, 0%, 123.136 kBytes/s, ETA 357h20m24s
Errors: 0
Checks: 4634 / 4634, 100%
Transferred: 9 / 9527, 0%
Elapsed time: 34m13.5s

6

rclone just gets better and better… thanks NCW!

7

I must take the chance to clarify that rclone NOW could handle empty folders.

Thank you NCW

ps:
I tested syncing using 1.43 to sync dir1 to dir2
/test/dir1/0-empty folder to /test/dir2

rclone CAN now sync the 0-empty folder to dir2, which is too great to be true.

verion before 1.40? wont copy that empty folder and it hurts very much as
i think sync = same and wont expect any loss, even those are empty folders.

8

@ncw

Thanks for the update :slight_smile: as always love your work and effort you put into this.

Can we update and keep the existing config files?

Thanks

Morphy

1 Like
9

Absolutely!

I try not to break the config files, though occasionally it is necessary to reconfig a backend (that will be necessary in v1.44 for onedrive when we swap to microsoft graph for instance). rclone will tell you what to do in that case.

1 Like
10

Cheers :slight_smile:

Thanks for the reply .

Morphy

11

Hey there ncw, I am a huge fan of your work. I was pretty stoked to read about the “:backend:” features. Would it be possible to create additional flags for “–drive-token” and “–drive-team_drive”? If I could get these, I would be able to greatly simplify my drive to drive transfer scripts.

12

Hi @ncw

When I download from https://rclone.org/downloads/ and install the new version … after running rclone version it states its still version 1.42 ??

13

LOL forget my post… freaking hell… Was in the wrong folder lmao :stuck_out_tongue:

1 Like
14

It was my aim to get all the config as config flags, but I missed all the ones to do with oauth: client_id, client_secret and token (and maybe one other).

I seem to have decided that team_drive should be hidden - not sure why. That does in fact already exist at --drive-team-drive it just doesn’t appear in the help!

Can you please make a new issue on github with this in and I’ll make sure we get those added - thanks :smiley: