Slow file move from source to cloud using vfs-cache

What is the problem you are having with rclone?

Very slow moves of files

Run the command 'rclone version' and share the full output of the command.

rclone v1.61.1

  • os/version: debian 11.6 (64 bit)
  • os/kernel: 5.15.39-1-pve (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.19.4
  • go/linking: static
  • go/tags: none
#RCLONE SERVICE CONFIG
[Unit]
Description=RClone Service
PartOf=gmedia.service
RequiresMountsFor=/gmedia

[Service]
Type=notify
Environment=RCLONE_CONFIG=/home/USER/.config/rclone/rclone.conf
ExecStart=/usr/bin/rclone mount gdrive_linux_crypt: /gmedia/ \
   --allow-other \
   --cache-dir=/docker/vfs_cache \
   --attr-timeout 1000h \
   --buffer-size 256M \
   --dir-cache-time 1000h \
   --log-level INFO \
   --log-file /var/log/rclone/rclone-linux.log \
   --vfs-cache-mode full \
   --vfs-cache-max-size 200G \
   --vfs-fast-fingerprint \
   --vfs-read-chunk-size 32M \
   --vfs-write-back 5m \
   --vfs-cache-max-age 24h \
   --disable-http2 \
   --tpslimit 12 \
   --tpslimit-burst 0
ExecStop=/bin/fusermount -uz /gmedia
Restart=on-failure
User=USER
Group=GROUP

[Install]
WantedBy=multi-user.target

Are you on the latest version of rclone? You can validate by checking the version listed here: Rclone downloads
--> Yes

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Just moving a file from local storage to the rclone mount 
mv filename /gmedia/Location/destinationfilename

The rclone config contents with secrets removed.

[gdrive_linux]
type = drive
client_id = redacted
client_secret = redacted
scope = drive
token = redacted

A log from the command with the -vv flag

I don't have any log entries for what I was doing as it was just a move using mv

So let me explain my setup here. I don't run plex on the same box as my docker containers. I use proxmox and I have 2 Hosts. 1 host has my plex LXC on it and then my other host has my docker lxc on it.
I used to download everything to one location and then I would NFS share that location with plex and use mergerfs on both plex as well as my docker server so plex could see the files after sonarr/radarr moved the files to their temporary location until midnight rolled around when my upload script would fire off and move everything up to the drive.
After reading about how vfs-cache-mode full works I liked the idea of things just going up to the drive as quickly as possible instead of at the end of the day.
So my sonarr has been backlogged severely this week and I've been trying to figure out why I thought it might be disk usage contention between it and another VM so I stopped what I thought would be causing the problem with that other VM. No change
So then today I manually moved a file for Sonarr to see for myself what was going on. As it was just a mv it should've been pretty instant I thought since I was moving it from where it was stored at to the drive and the vfs-cache is located on the same mount point as where the file was at so I figured the system would just move it like moving anything else normally and it'd be quick.
The file I moved was 10GB in size and it took over 10 mins for it to end up in the vfs-cache. I confirmed that while I did move it to /gdrive I looked in the cache dir and there it is just chilling in there.
I've checked my systemd scripts against what Animosity has (thank you for sharing with all of us) and everything looks fine. Yeah I have mine set to write back after 5m because I want plex to pick up the file once it sees it.
Thank you all for your help here and I look forward to finding a solution here. If I need to make the /gmedia mount within the /docker mount point thats fine I don't mind doing that but I just figured since the end location was on the same mount as the original location it would move it nice and fast.

You could try to copy the same file directly to the cache and see the speed.

That would help rule out the disk being the bottleneck.

Otherwise, you'd want to repeat it with a log file and share the log.

Generally, the slow point here would be the disk configured for cache as it's just copying it locally and there is very little rclone 'work' being done.

Ok so doing a local mv of a file to the cache dir it was instant. Did another copy of it and here is the log file for it. Its for Poker Face S01E01

2023/02/17 03:09:23 INFO  : 1TV Shows/The Middle/Season 5/The Middle - S05E11 - War of the Hecks WEBDL-1080p.mkv: vfs cache: removed cache file as Removing old cache file not in use
2023/02/17 03:09:23 INFO  : vfs cache RemoveNotInUse (maxAge=0, emptyOnly=false): item 1TV Shows/The Middle/Season 5/The Middle - S05E11 - War of the Hecks WEBDL-1080p.mkv was removed, freed 21033205 bytes
2023/02/17 03:09:23 INFO  : vfs cache: cleaned: objects 4209 (was 4272) in use 2813, to upload 2805, uploading 4, total size 199.985Gi (was 201.378Gi)
2023/02/17 03:09:41 INFO  : 1TV Shows/Poker Face (2023)/Season 1/Poker Face (2023) - S01E01 - Dead Man's Hand WEBDL-1080.mkv: vfs cache: queuing for upload in 5m0s
2023/02/17 03:09:45 INFO  : 1TV Shows/Who's the Boss!/Season 2/Who's the Boss! - S02E09 - Thanksgiving at Mrs. Rossini's WEBRip-480p.en.srt: vfs cache: queuing for upload in 5m0s

But once again this move took around 10mins to complete vs doing just a move the other way.

Debug log as that INFO doesn't show anything helpful unfortunately.

Oh sorry about that. Ok let me change the log level and I'll get it. My apologies @Animosity022

Ok so the logs are full of this while its moving the file

2023/02/17 06:43:42 DEBUG : 1TV Shows/Chicago Med/: >Attr: attr=valid=1000h0m0s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2023/02/17 06:43:44 DEBUG : 1TV Shows/Chicago Fire/Season 9/Chicago Fire - S09E06 - Blow This Up Somehow HDTV-1080p.en.srt: vfs cache: starting upload
2023/02/17 06:43:46 DEBUG : 1TV Shows/Chicago Med/Season 8/: Lookup: name="Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv"
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/: >Lookup: node=<nil>, err=no such file or directory
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/: Lookup: name="Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv"
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/: >Lookup: node=<nil>, err=no such file or directory
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/: Lookup: name="Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv"
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/: >Lookup: node=<nil>, err=no such file or directory
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/: Create: name="Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv"
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv: Open: flags=O_WRONLY|O_CREATE|O_EXCL
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv: newRWFileHandle:
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv(0xc001d26b80): openPending:
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv: vfs cache: truncate to size=0 (not needed as size correct)
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8: Added virtual directory entry vAddFile: "Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv"
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv(0xc001d26b80): >openPending: err=<nil>
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv: >newRWFileHandle: err=<nil>
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8: Added virtual directory entry vAddFile: "Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv"
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv: >Open: fd=1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv (rw), err=<nil>
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/: >Create: node=1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv, handle=&{1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv (rw)}, err=<nil>
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv: Attr:
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv: >Attr: a=valid=1000h0m0s ino=0 size=0 mode=-rw-r--r--, err=<nil>
2023/02/17 06:43:47 DEBUG : &{1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv (rw)}: Write: len=131072, offset=0
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv(0xc001d26b80): _writeAt: size=131072, off=0
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv(0xc001d26b80): >_writeAt: n=131072, err=<nil>
2023/02/17 06:43:47 DEBUG : &{1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv (rw)}: >Write: written=131072, err=<nil>
2023/02/17 06:43:47 DEBUG : &{1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv (rw)}: Write: len=131072, offset=131072
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv(0xc001d26b80): _writeAt: size=131072, off=131072
2023/02/17 06:43:47 DEBUG : 1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv(0xc001d26b80): >_writeAt: n=131072, err=<nil>
2023/02/17 06:43:47 DEBUG : &{1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv (rw)}: >Write: written=131072, err=<nil>

But the thing that is concerning is that its been trying to upload since last night

2023/02/17 17:37:33 DEBUG : vfs cache RemoveNotInUse (maxAge=86400000000000, emptyOnly=false): item 1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv not removed, freed 0 bytes
2023/02/17 17:37:33 INFO  : vfs cache: cleaned: objects 3 (was 3) in use 3, to upload 1, uploading 2, total size 8.739Gi (was 8.739Gi)
2023/02/17 17:38:23 DEBUG : Google drive root 'media/store': Checking for changes on remote
2023/02/17 17:38:33 DEBUG : vfs cache RemoveNotInUse (maxAge=86400000000000, emptyOnly=false): item 1TV Shows/Yellowstone (2018)/Season 1/Yellowstone (2018) - S01E01 - Daybreak WEBDL-1080p.mkv not removed, freed 0 bytes
2023/02/17 17:38:33 DEBUG : vfs cache RemoveNotInUse (maxAge=86400000000000, emptyOnly=false): item 1TV Shows/Chicago Fire/Season 9/Chicago Fire - S09E06 - Blow This Up Somehow HDTV-1080p.en.srt not removed, freed 0 bytes
2023/02/17 17:38:33 DEBUG : vfs cache RemoveNotInUse (maxAge=86400000000000, emptyOnly=false): item 1TV Shows/Chicago Med/Season 8/Chicago Med - S08E13 -It's an Ill Wind That Blows Nobody Good WEBDL-1080p.mkv not removed, freed 0 bytes
2023/02/17 17:38:33 INFO  : vfs cache: cleaned: objects 3 (was 3) in use 3, to upload 1, uploading 2, total size 8.739Gi (was 8.739Gi)
2023/02/17 17:38:33 DEBUG : : Statfs:
2023/02/17 17:38:33 DEBUG : : >Statfs: stat={Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2023/02/17 17:38:36 DEBUG : pacer: low level retry 9/10 (error Post "https://www.googleapis.com/upload/drive/v3/files?alt=json&fields=id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CexplicitlyTrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink%2CshortcutDetails%2CexportLinks%2CresourceKey&supportsAllDrives=true&uploadType=resumable&upload_id=ADPycds94M4qBt4JE8mYQ1nN-4EXnL19JeamPh7TvD8jIjcmUNlUjyuP_BIxuOXkvG2h1JW_dH48zU21WMOznuax5EeSqC9TsnJW": write tcp 192.168.90.2:58188->142.250.74.138:443: i/o timeout)
2023/02/17 17:38:36 DEBUG : pacer: Rate limited, increasing sleep to 1.715316134s
2023/02/17 17:38:36 DEBUG : bknu8r6rf5icb0qlf4i3n1v8sg/54jhdp4dugss437bv2j9rj34lo/3haktacfi8ptbrg2282hpmb2do/guj4533pba6u45rnc752q0v0npu8fmaeg6v4s6k2lq44npio1rfk7uketvv9t6jj98na4b2bd6skhjfk14kfd910qu3jjkt41olegrmm9phpuampvn8iq6ppds264frb: Sending chunk 0 length 8388608
2023/02/17 17:39:23 DEBUG : Google drive root 'media/store': Checking for changes on remote
2023/02/17 17:39:24 DEBUG : pacer: Reducing sleep to 0s

So I'm trying to figure out why its getting a timeout. The drive is mounted properly but it just cannot upload so I'm trying to figure out why its getting a timeout when uploading.

That's a network timeout so more a local issue type thing.

Yeah I'm trying to figure out what is causing this. Digging through my firewall logs and they all look ok so its gotta be something on the box so digging.

Ok so for the rclone mount I ended up putting it directly on the proxmox host and then mounting the directory into the LXC container directly. This fixed the upload issue. I'll be able to provide an update here in a bit with how the file moving goes once I get done moving my docker configs to the new mount point that I created for them as well since they are now all on the same mount point within the host. It'll be a few before I can get back here to provide and update today. Rsync is working hard on transferring everything.

So I can confirm that now that I have everything back up and running, I'm still seeing the slow move times. The logs show exactly what they showed up above where when writing the file directly to the mount point it basically acts like its copying it there instead of just updating the file location in the filesystem. If you write directly to the vfs_cache its instant. I don't know what the ramifications would be of writing directly to the cache as I don't know if rclone would pick up on the file change.

If you cleaned up the logs, please share a new set.

If the logs still have networking errors in them, that will be your problem though.

Ok let me get a new set of logs that no longer have the network error and turn the debug back on. I can def confirm that there is no more upload issue things are flowing up quite nicely.

You'd break things if you write directly to the cache location.

Ok was finally able to enable DEBUG mode
So when the file begins its move we get

2023/03/01 00:12:58 DEBUG : : >Statfs: stat={Blocks:274877906944 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2023/03/01 00:12:58 DEBUG : 1TV Shows/Magnum P.I/Season 1/: Lookup: name="Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv"
2023/03/01 00:12:58 DEBUG : 1TV Shows/Magnum P.I/Season 1/: >Lookup: node=<nil>, err=no such file or directory
2023/03/01 00:12:58 DEBUG : 1TV Shows/Magnum P.I/Season 1/: Lookup: name="Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv"
2023/03/01 00:12:58 DEBUG : 1TV Shows/Magnum P.I/Season 1/: >Lookup: node=<nil>, err=no such file or directory
2023/03/01 00:12:58 DEBUG : 1TV Shows/Magnum P.I/Season 1/: Lookup: name="Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv"
2023/03/01 00:12:58 DEBUG : 1TV Shows/Magnum P.I/Season 1/: >Lookup: node=<nil>, err=no such file or directory
2023/03/01 00:12:58 DEBUG : 1TV Shows/Magnum P.I/Season 1/: Lookup: name="Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv"
2023/03/01 00:12:58 DEBUG : 1TV Shows/Magnum P.I/Season 1/: >Lookup: node=<nil>, err=no such file or directory
2023/03/01 00:12:58 DEBUG : 1TV Shows/Magnum P.I/Season 1/: Lookup: name="Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv"
2023/03/01 00:12:58 DEBUG : 1TV Shows/Magnum P.I/Season 1/: >Lookup: node=<nil>, err=no such file or directory
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: Lookup: name="Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv"
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: >Lookup: node=<nil>, err=no such file or directory
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: Lookup: name="Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv"
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: >Lookup: node=<nil>, err=no such file or directory
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: Lookup: name="Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv"
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: >Lookup: node=<nil>, err=no such file or directory
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: Lookup: name="Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv"
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: >Lookup: node=<nil>, err=no such file or directory
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: Lookup: name="Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv"
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: >Lookup: node=<nil>, err=no such file or directory
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: Lookup: name="Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv"
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: >Lookup: node=<nil>, err=no such file or directory
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: Lookup: name="Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv"
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: >Lookup: node=<nil>, err=no such file or directory
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: Lookup: name="Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv"
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: >Lookup: node=<nil>, err=no such file or directory
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: Create: name="Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv"
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv: Open: flags=O_RDWR|O_CREATE|O_EXCL
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv: newRWFileHandle:
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv(0xc00498da40): openPending:
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv: vfs cache: truncate to size=0 (not needed as size correct)
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1: Added virtual directory entry vAddFile: "Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv"
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv(0xc00498da40): >openPending: err=<nil>
2023/03/01 00:12:59 DEBUG : bknu8r6rf5icb0qlf4i3n1v8sg/gto2ab3aha8hh413birt0ibog8/el47ahm8i2b188bppahb2esi9g/a5poopkcodo9s9na9ln39rf1k34crbsmg1ovmt3m9498m0bor6dkaal83tr7e6te7gn3vacr9si7dova4l91i1frj1d2cpckhojjhp8: Sending chunk 1912602624 length 8388608
2023/03/01 00:12:59 DEBUG : bknu8r6rf5icb0qlf4i3n1v8sg/gto2ab3aha8hh413birt0ibog8/el47ahm8i2b188bppahb2esi9g/a5poopkcodo9s9na9ln39rf1k34crbsmg1ovmt3m9498m0bor6dkaal83tr7e6te7gn3vacr9si7dova4l91i1frj1d2cpckhojjhp8: Sending chunk 1920991232 length 8388608
2023/03/01 00:12:59 DEBUG : bknu8r6rf5icb0qlf4i3n1v8sg/gto2ab3aha8hh413birt0ibog8/el47ahm8i2b188bppahb2esi9g/a5poopkcodo9s9na9ln39rf1k34crbsmg1ovmt3m9498m0bor6dkaal83tr7e6te7gn3vacr9si7dova4l91i1frj1d2cpckhojjhp8: Sending chunk 1929379840 length 8388608
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv: >newRWFileHandle: err=<nil>
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1: Added virtual directory entry vAddFile: "Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv"
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv: >Open: fd=1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv (rw), err=<nil>
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/: >Create: node=1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv, handle=&{1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv (rw)}, err=<nil>
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv: Attr:
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv: >Attr: a=valid=1000h0m0s ino=0 size=0 mode=-rw-r--r--, err=<nil>

Then once it starts to actually move it you can see the off size increases

2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv: Attr:
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv: >Attr: a=valid=1000h0m0s ino=0 size=0 mode=-rw-r--r--, err=<nil>
2023/03/01 00:12:59 DEBUG : &{1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv (rw)}: Write: len=65536, offset=0
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv(0xc00498da40): _writeAt: size=65536, off=0
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv(0xc00498da40): >_writeAt: n=65536, err=<nil>
2023/03/01 00:12:59 DEBUG : &{1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv (rw)}: >Write: written=65536, err=<nil>
2023/03/01 00:12:59 DEBUG : &{1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv (rw)}: Write: len=65536, offset=65536
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv(0xc00498da40): _writeAt: size=65536, off=65536
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv(0xc00498da40): >_writeAt: n=65536, err=<nil>
2023/03/01 00:12:59 DEBUG : &{1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv (rw)}: >Write: written=65536, err=<nil>
2023/03/01 00:12:59 DEBUG : &{1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv (rw)}: Write: len=65536, offset=131072
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv(0xc00498da40): _writeAt: size=65536, off=131072
2023/03/01 00:12:59 DEBUG : 1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv(0xc00498da40): >_writeAt: n=65536, err=<nil>
2023/03/01 00:12:59 DEBUG : &{1TV Shows/Magnum P.I/Season 1/Magnum P.I. - S01E05 - Sudden Death WEBDL-1080p.mkv (rw)}: >Write: written=65536, err=<nil>

These logs will repeat over and over until the file is completely moved.

Can you share the full log and not just the snippet?

Do you have a place I can send a 3G file?

Correction after compressing it down its down to 70MB tar.gz up.

EDIT: I just DM'd you the link to the log file. Thanks.

@Animosity022 was just checking in that you were able to get the log file

Yes.

Always best to share a log as I'm missing all the top part which shows version, flags, all the good details I really want to see before digging into the log.

The time spent is when it copies to your local cache disk for a 3GB file, it takes

2023/03/01 20:49:34 DEBUG - start of the copy
2023/03/01 20:52:30 DEBUG - end of the copy

So 3GB file takes about 3 minutes to copy which seems pretty long.

That would be the slowness.

Yeah I figured it'd just be a straight direct copy but it seems to no matter where its at write it bit by bit until its in there. Even if all of the locations are on the same mount point.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.