Restoring directory from the Cloud

Help and Support
1

STOP and READ USE THIS TEMPLATE NO EXCEPTIONS - By not using this, you waste your time, our time and really hate puppies. Please remove these two lines and that will confirm you have read them.

What is the problem you are having with rclone?

I am trying to restore a directory to a dedicated TrueNAS share. The subdirectories and files get restored to the root directory. Parent directory does not get restored.

Run the command 'rclone version' and share the full output of the command.

rclone v1.67.0

  • os/version: debian 12.6 (64 bit)
  • os/kernel: 6.6.44-production+truenas (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.22.7
  • go/linking: dynamic
  • go/tags: none

No, I am on 1.67 and I seem to be unable to run sudo rclone selfupdate - I guess this has been blocked in the latest release of TrueNAS Electric Eel or I at least have to run this as root, which I can' at the moment.

Which cloud storage system are you using? (eg Google Drive)

Microsoft OneDrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy --progress --no-update-dir-modtime --multi-thread-streams=8 --transfers 16 --create-empty-src-dirs ONEDRIVE:/ABC /mnt/POOL1/ONEDRIVE-SHARE/

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[ONEDRIVE]
type = onedrive
token = XXX
drive_id = XXX
drive_type = personal

A log from the command that you were trying to run with the -vv flag

I am trying to bring one directory back to NAS, I don't have it locally. I created a dataset and a share on Truenas for this single directory. Copying back to NAS from Onedrive works fine except for the fact that I get everything what is inside that directory written to the root of the NAS share. My expectation was that ABC will be created inside /mnt/POOL1/ONEDRIVE-SHARE with everything what inside it. Instead ABC does not get created, however everything what is inside it is dumped into the root of the target on the NAS.

I wanted to ask whether this is expected behavior and I should first make a target directory on NAS share and append it to the destination in my command? Like:

rclone copy --progress --no-update-dir-modtime --multi-thread-streams=8 --transfers 16 --create-empty-src-dirs ONEDRIVE:/ABC /mnt/POOL1/ONEDRIVE-SHARE/ABC

Or may be I just miss some flags?
Thank you very much.

3

welcome to the forum,

that is the correct way to do it.

4

This is how rclone works...As per docs:

Note that it is always the contents of the directory that is synced, not the directory itself. So when source:path is a directory, it's the contents of source:path that are copied, not the directory name and contents.

You need:

rclone copy ONEDRIVE:ABC /mnt/POOL1/ONEDRIVE-SHARE/ABC
5

In addition. Create your own client_id or you will suffer from Microsoft throttling your traffic a lot. When done recreate your remote using your own client_id.

You can experiment with this value but default 4 is probably enough. High values will work for a moment and then you will be throttled.

It is possible with TrueNAS appliance but simply download latest rclone to some directory and use from there. Also for OneDrive I recommend the latest beta (v1.69) as it contains fix for Onedrive some throttling issues.

6

Thank you for clearing my doubts.

7

This is super helpful. Thank you for sharing knowledge, friend!

8

Hello @kapitainsky - Following your advise I have ventured into creating my ClientID. One question I wanted to ask - in your post suggesting the use of own ClientID you said When done recreate your remote using your own client_id.

I read the guide and tried a couple of times to edit the remote rather than creating a new one. Both time I failed - Microsoft did not like the ClientIDs and said something like ID does not exist or not enabled yada yada.

Question - do you suggest recreating vs editing? This is just for my information and knowledge.

In fact when I deleted the remote and created it afresh with the ID and secret I succeeded.

9

You could edit but then some extra steps are needed as client_id must be bound to token. In theory running rclone config reconnect remote_name: should do the job but it failed me in the past. Most likely it was problem between chair and keyboard. So as lazy person I use the path of least resistance.

Recreating remote takes 1min max so this is the easiest approach and result is guaranteed:)

1 Like