My streams seem much more stable on my new server, but I’ve got 2 questions;
Using your ACD Speedtest, (with Crypt) - I seem to get about a 40s delay before Rclone will begin to copy the file, once it starts its consistently 30MiB/s+, but it takes way too long to begin. It seems to be the same with Plex too
Have you noticed that Rclone seems to be noticeably slower when copying through the unionfs mount then when copying directy from the rclone mount point? do you have that issue?
Thanks!
Everything is slower when copying trough unionfs, iam getting flaky results as well.
What about start up times for transferring files? There will be like a 40 second pause after your script downloads a few KB, then it’ll begin downloading fast. (This is with a 3.1GB file)
Dunno for me i sometimes have that inital pause directly on acdcrypt and sometimes on unionfs, its some weird behavior from acd.
Would you care to share this script?
Regarding UnionFS speed… this may help, I’m still testing, add this to your Union FS mount options:
-o sync_read
This changes the default read behaviour of unionfs from asynchronous to synchronous, which I bleieve it more in line with the way rclone works and the way Plex prepares its files for stremaing.
Iam also checking it with direct_io it seems it improves performance on file access.
Sometimes it can take 10 to 30 seconds before file is actually being copied ( copyspeedtest ) and since I was trying with direct_io i never encounter it .. could be a fluke as it does not make sense.
direct_io
This option disables the use of page cache (file content cache)
in the kernel for this filesystem. This has several affects:
1. Each read(2) or write(2) system call will initiate one or more
read or write operations, data will not be cached in the kernel.
2. The return value of the read() and write() system calls will
correspond to the return values of the read and write
operations. This is useful for example if the file size is not
known in advance (before reading it).
Mind sharing your mount parameters again 
You’re always on the bleeding edge I’ve been a bit out of the loop. I see there’s --buffer-size now are people using that?
Here are mine
rclone mount --allow-other --max-read-ahead=2G --acd-templink-threshold=0 --dir-cache-time=60m --checkers=16 --stats=0 --quiet
unionfs-fuse -o allow_other,cow,max_readahead=2000000000
Iam currently using
rclone mount
--read-only
--allow-other
--acd-templink-threshold 0
--stats 10s
--buffer-size 1G
-v
--log-file=/home/plex/logs/mountacdcrypt.log
acdcrypt: $MPOINT &
Did not notice any difference with or without -v and logging the advantage with this info log is that you can tail with what speed if file getting transferred. eg
tail -f /home/plex/logs/mountacdcrypt.log | grep ETA
and for unionfs
unionfs-fuse \ -o cow,allow_other,direct_io,auto_cache,sync_read \ /storage/local=RW:/mnt/acdcrypt=RO \ $MPOINT
@toomuchio the max read ahead and checkers wont do anything so you can drop them.
With current unionfs mount its a bit contradictory the direct_io and auto_cache together since
auto_cache
This option enables automatic flushing of the data cache on
open(2). The cache will only be flushed if the modification time
or the size of the file has changed.
while direct_io disables page cache
hi jpat0000,
I installed node-gdrive-fuse with the changes for mtime and ctime. For some reason, I do not get valid ctime or mtime. When I do ls -lh, I see Jan 1, 1970 for the ctime for all files and folders.
I’m not sure why the fix didn’t work for me.
edit: nevermind. It looks like this issue was fixed in the latest snapshot so when I modified the files, I broke the mtime and ctime edits that were made elswhere.
Hey @Ajki, are you still pointing plex at your rclone mount directly to avoid the flakiness with fuse speeds or are you actually pointing plex to the fused directory now?
Iam pointing it to fuse and it seems direct_io solved the flakiness with unionfs, the main problem is ACD and not really the fuse. I did several tests with local fuse and while read is almost on pair the write speed is a bit slower but still you will hit bandwith limit way before you hit that one.
This is read speed of local files unionfs compared to local copy
As for speed from ACD with or without fuse its more about amazon none consistent speeds then fuse it self.
few minutes later same test ( rclone remounted so cache is cleared )
Awesome, thank you for all your help and testing!
Hey, I have a few question regarding mounting with rclone.
If I try to mount with --buffer-size=1G I get an unknown flag error, rclone version 1.35 on Ubuntu. Is this only available in beta?
@Ajki: I’m in germany and have an Amazon US Account, is therer really an difference between the US and the DE Account speedwise? Therefore it would be better for me to make a switch?
Why the–read-only flag? Sonarr, Sub-Zero and Couchpotato aren’t working right with that
@Karl_Gustav yes there is speed difference since Amazon EU have datacenter in Ireland and .com in US. I also started with amazon.com but had around 50% worst performance compared to DE one.
If you are from Germany you should get amazon.de account.
As for buffer its only in beta and as for read-only you should use unionfs so everything copied into it its stored in other folder locally from which you upload with rclone.
Do you have any tips for my switch, approximately 8 TB to transfer? Could I just ask them to switch my account from US to DE? I won’t get blocked 
You can migrate account but they will tell you that you need to move data on your own eg its better you open amazon.de drive with new email account and get 3 months free.
My suggestion would be to make that amazon.de acount add it to rclone as remote then get yourself cheap VPS on https://www.scaleway.com
Install Ubuntu 16.04 , wget latest rclone beta and copy your rclone config
make simple bash script acdus2acdde.sh
and put it in crontab -e
This will make sure that script would be run every minute, but dont worry it wont allow to instances at same time.
IMPORTANT. Sync remote drives directly and not secret/crypt ones.
Once all your files are synced ( 3 days top's for 8TB ) just check the log, remove it from crontab and change the sync part with
/usr/bin/rclone check acdus: acdde: -v --log-file=/path/to log file/
Run it once and when its finished check the logs if all files are ok.
Now when you have everything synced open rclone.conf find entry with type = crypt and change remote = acdde:/myencryptedfiles ( just the remote part if it was acd now change it to the new registered remote )
And thats it you don't need to change anything else.
A little off topic but has anyone looked into using https://github.com/trapexit/mergerfs instead of unionfs ? Seems like it has some pretty cool features. Probably helpful to those who have a primary SSD drive and a secondary SATA drive.
jtame said he is testing OverlayFS for it.
Personally when it comes to read speed there is not much difference if you are using unionfs or pointing plex directly ( eg for streaming )
Check the test on 1GB file being copied from UnionFS folder or directly from RW folder
Only 0.10 sec difference for 1GB file and even that could easily be due some server task that triggered during that time.
p.s. Write speed should be slower
