What's the most efficient way of syncing/copying between two Sharepoint mounts?

Help and Support
1

What is the problem you are having with rclone?

We're migrating our Sharepoint Documents via rclone and rsnyc between two Microsoft tenants. I've mounted two Sharepoint (as WebDAV) sites for testing( from source tenant to destination tenant). I thought that turning off the vfs-cache mode to off would make things faster. It took almost 2 hours to transfer 19GB of data. I had to use --tpslimit because of microsoft requests limits.

What's the efficient way to mount? (Should I use vfs-cache mode writes?) or even if I should mount it, I'm not so familiar with the copy, sync and dedupe command, but they seem a replacement for rsync.

Because of the how large one of our sites (close to 3TB), we probably have to do one pass then do a sync after because of the deltas.

Run the command 'rclone version' and share the full output of the command.

rclone v1.59.1

  • os/version: ubuntu 22.04 (64 bit)
  • os/kernel: 5.15.0-1020-aws (x86_64)
  • os/type: linux
  • os/arch: amd64

Which cloud storage system are you using? (eg Google Drive)

Microsoft Sharepoint but mounted as WebDAV

The command you were trying to run (eg rclone copy /tmp remote:tmp)

For mounting the Sharepoint sites. As for the source site, it's pretty much the same.

rclone --vfs-cache-mode off mount "EVP_Test_Site": /mnt/EVP_Test_Sharepoint/ --allow-other -vvv --tpslimit=10 --log-file=/mnt/rclone_log_destination.txt

For the rsync command, I used

rsync --delete -ar SOURCE_PATH DESTINATION_PATH --log-file=/mnt/rsync_log.txt --info=progress2

The rclone config contents with secrets removed.

type = webdav
vendor = sharepoint
2

You will get better performance doing an rclone sync rather than using rsync on the two mounts.

So

rclone sync --dry-run "source:" "dest:" -vv --tpslimit=10 --log-file=transfer.log

where one of those source or dest is "EVP_Test_Site": - I don't know what the other should be.

Remove --dry-run when happy.

3

Thank you for the reply! I'm actually testing rclone copy right now with --drun-run flag. I'm planning to do rclone copy first then sync for the deltas after. Do you think that's reasonable?

4

That is what I'd do. rclone copy is safe. rclone sync can delete stuff you wanted so try with --dry-run first.

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.