Copy Files From SFTP remote to SharePoint Remote

Help and Support
1

I am loving rClone thus far and have quite a large amount of data to do in an initial lift and drop. +- 250GB WAV files on an SFTP endpoint that I would like to move to a Sharepoint Library.

I am using the latest stable rclone release.

RCLone is running on an AWS EC2 instance (t2.micro) and have successfuly set up both remote endpoints and can confirm that i am able to successfully list, copy and sync between remotes.

before i go ahead and attempt the initial lift and drop I was hoping there would be some sort of strategy I could employ to make the lift and drop reliable and a the same time preserve the current sftp folder structure.

My current folder structure is /root folder/subfolders(9150 in total) and each of these contain multiple WAV files. The plan is to do the initial lift and drop and then run a cron to copy any new files dropped into the sftp server. The TTL on the files on the sftp server will be reduced to 7 days so I would need to ensure that all new files are copied before the TTL expires on the SFTP server over which we do not have any control.

Any tips, suggestions or comment would be greatly appreciated.

Thanks in advance.

2

Rclone will preserve your folder structure so probably something like this will do the trick.

rclone copy  sftp:"/root folder/" sharepoint:"root folder"

You should just be able to re-run the rclone copy to pick up new files. It won't copy file which have been transferred already.

3

Thanks @ncw

Will give this a go and see what results i get.

1 Like
4

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.