I'm using the "--vfs-cache-mode writes" property on mount command to download "directly" from JDownloader to a Google Drive remote and encripted folder. It seems to be working fine so far, but I've noticed that at remote Download directory there are some .part files. Here's an example:
Seems like every iso file has the correct size but the .iso.part files size is 0.
It could be the way JDownloader works, but there are some files where .iso files are 0 and the .iso.part files have a size that could be the correct and expected size for an iso file. Maybe they didn't close properly?
So I don't know if the files are in a good shape or JDownloader is doing something that rclone is not able to support!
Should I remove the .iso.part files that are equal to zero and forget about it? and therefore...
Should I rename the .iso.part files that it's size aren't zero and delete the .iso with zero size?
Or maybe I'm doing something wrong on mount?
Thank you very much in advance,
What is your rclone version (output from rclone version)
1.51.0
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Linux (QNAP) 64 bit.
Which cloud storage system are you using? (eg Google Drive)
Google Drive.
The command you were trying to run (eg rclone copy /tmp remote:tmp)
Thank you for your (fast!!) response. I'm trying to reproduce the issue, but on the first download the only strange thing are all those ".iso.part" with 0 size for every normal iso file downloaded.
I think those ".part" files are generated by JDownloader as a container for each file that is being downloaded and aren't complete. Maybe when the download ends, JDownloader changes or rename the filename removing that ".part". I don't know why they are being uplodaded too to the remote when the download finishes.
Another question: Is it normal not to have the files in the cache directory during the download? I'm looking at "/share/homes/user/Downloads/cache-rclone" and there are no files during the download. I can see vfs/drivecrypt directory but always empty. Same if I change "--vfs-cache-max-age 0h0m0s" to "--vfs-cache-max-age 1h0m0s"
Here's the log, I can't see anything important or to worry about (or anything related to the download of the files itself) but every downloaded file still has its own ".part duplicate" file with 0 size. I don't know if I delete all the ".part" files the "good" files will be in a good shape or not. So weird.
2020/05/06 21:02:06 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone" "mount" "-vv" "drivecrypt:Downloads" "/share/Public/Rclonedrives/Downloads" "--config=/share/CACHEDEV1_DATA/homes/user/.config/rclone/rclone.conf" "--allow-other" "--buffer-size" "256M" "--dir-cache-time" "72h" "--umask" "002" "--vfs-cache-mode" "writes" "--vfs-cache-max-age" "1h0m0s" "--cache-dir" "/share/homes/user/Downloads/cache-rclone" "--rc-addr=localhost:5575"]
2020/05/06 21:02:06 DEBUG : Using config file from "/share/CACHEDEV1_DATA/homes/user/.config/rclone/rclone.conf"
2020/05/06 21:02:09 DEBUG : Encrypted drive 'drivecrypt:Downloads': Mounting on "/share/Public/Rclonedrives/Downloads"
2020/05/06 21:02:09 DEBUG : vfs cache root is "/share/homes/user/Downloads/cache-rclone/vfs/drivecrypt/Downloads"
2020/05/06 21:02:09 DEBUG : Adding path "vfs/forget" to remote control registry
2020/05/06 21:02:09 DEBUG : Adding path "vfs/refresh" to remote control registry
2020/05/06 21:02:09 DEBUG : Adding path "vfs/poll-interval" to remote control registry
2020/05/06 21:02:09 DEBUG : : Root:
2020/05/06 21:02:09 DEBUG : : >Root: node=/, err=
2020/05/06 21:03:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:04:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:05:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:06:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:07:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:08:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:09:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:10:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:11:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:12:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:13:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:14:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:15:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:16:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:17:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:18:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:19:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:20:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:21:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:22:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:22:09 DEBUG : drive: Loaded fresh token from config file
2020/05/06 21:22:09 DEBUG : drive: Saved new token in config file
2020/05/06 21:23:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:24:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:25:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:26:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:27:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:28:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:29:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:30:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:31:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:32:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:33:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:34:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
2020/05/06 21:35:09 DEBUG : Google drive root 'server/d879ljafm5lho0qm4tlf71ke0o': Checking for changes on remote
You're very clear, but the log doesn't seems to show any movement from the downloads.. or any reference to the file or the path.. I'm going to try with another download.
Well, after some downloads seems like the mix Jdownloader/rclone is doing what I said only with .iso files. I've downloaded some .rar files without that problem and can decompress them so they are working as they should.
On the other hand I can't see the download debug info with the option -vv, but I can see the rest of the processes. Maybe that's because I'm mounting a drivecrypt:Downloads remote folder when this remote is already mounted on another folder. Here's what I've done:
First mount: rclone mount -vv drivecrypt: /share/Public/Rclonedrives/drivecrypt
Second mount: rclone mount -vv drivecrypt:Downloads /share/Public/Rclonedrives/Downloads
And maybe the problem with the JDownloader and rclone also comes from here.
I'm not able to umount the First mount to try because it's doing a long backup right now, so I'll try at another time.
Searching an alternative I've found that I can use Download Station on my QNAP which allows me to assign a Download folder and a folder to move the file once downloaded. Seems very similar to what rclone does with cache. But then I thought, what's the difference between the rclone move and the Linux move command who surely that's the command used by Download Station? I'm doing this question just to be sure if there's an Internet connexion issue or an energy drop this does not affect the integrity of the files that are moving between local and remote.
Is there a problem to move files to a remote with the Linux command "mv" (Download Station way) instead of "rclone move"? or could there be some kind of problems in case of loss connection?
Is there a better application than Download Station to do what I want and ensure the integrity of the data?
Thank you Calisro, I'll try with the --log-file to generate the file and to see why JDownloader is uploading the iso file plus a file with ".part" extension to zero size to remote. Then, in therms of file integrity and recovery in strange situations like power supply drop or Internet connection interruption, what do you think is better uploading to a remote?
Finally, the problem has disappeared. Maybe the new JDownloader update does the trick, I don't know exactly. Everything's fine now! Thanks a lot for all your support!