Re-start terminated job

Help and Support
1

What is the problem you are having with rclone?

Below the template

What is your rclone version (output from rclone version)

latest, rclone v1.53.3

  • os/arch: windows/amd64
  • go version: go1.15.5

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount crypt: Y: --vfs-cache-mode full --cache-dir="[deleted]"

Hello. When i terminate job (i mounted google drive, uploaded some files and closed it by using ctrl+c, it asked me if i want terminate job so i clicked yes), is it possible to restart that job after closing that CMD window? Or, is it automatic?

E.g. when i remount google drive, will it continue with uploading files?

1 Like
2

what operating system are you using?

3

Windows. I don't think it really matters.

4

rclone mount only uploads what it is asked it to, does not upload anything on it own.

so the question is, when you manually restart the mount, will that app notice the mount is mounted again and re-start the upload process.

5

Not sure why folks fight the help template.

It matters what OS.
It matters what version you are using.
It matters what mount command you are using.
It matters what backend you are using and what rclone.conf you have setup.

The answer to your question is that it depends on all the above and what is setup.

6

Well, we probably misunderstood each other. My theory is, when i copy the files to google drive mount, it will temporarily copy to cache folder in file system. But when i terminate the job when the files are already copied to cache folder, it will be there until the cache time passes. And my question is, when i remount that google drive, it will spot there are some files in cache folder so it should start upload the files maybe?

7

not sure, as if i am going to use a mount for copying, instead of rclone copy, i do not use cache.

i often run multiple mounts to the same remote.
one mount for streaming, mounted read-only, vfs cache mode full, optimized for priming the mount before a jellyfin media scan.
one mount for copying files, no vfs cache

8

Edited. Thought rclone works nearly the same on every OS.

9

That is the goal but unfortunately not the reality as Windows uses WinFSP and Linux is fuse based which they operate close, but not always the same.

With full mode, you copy to a cache area first, which leaves it on disk. It then uploads it to your remote, the question I think is if there are files there being uploaded and you stop the mount, what happens to that upload that was in flight and the remainder of the uploads? Is that it the question?

10

Yes, exactly. I think, the files will remain in the cache folder because rclone isn't running so the time doesn't pass and that files can't be deleted. So when i mount gdrive again (so the rclone is running and time passes) it technically should realize there's some files in cache folder but aren't on google drive yet?

11

Assuming you don't clean up the cache file areas, it should resume the upload when it's started back up.

An example log looks like:

2020/12/30 10:06:19 DEBUG : rclone: Version "v1.53.3" finishing with parameters ["rclone" "mount" "-vv" "GD:" "/home/felix/test" "--vfs-cache-mode" "full" "-vv"]
felix@gemini:~$ rclone mount -vv GD: /home/felix/test --vfs-cache-mode full -vv
2020/12/30 10:06:28 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "mount" "-vv" "GD:" "/home/felix/test" "--vfs-cache-mode" "full" "-vv"]
2020/12/30 10:06:28 DEBUG : Creating backend with remote "GD:"
2020/12/30 10:06:28 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2020/12/30 10:06:28 DEBUG : vfs cache: root is "/home/felix/.cache/rclone/vfs/GD"
2020/12/30 10:06:28 DEBUG : vfs cache: metadata root is "/home/felix/.cache/rclone/vfs/GD"
2020/12/30 10:06:28 DEBUG : Creating backend with remote "/home/felix/.cache/rclone/vfs/GD"
2020/12/30 10:06:28 DEBUG : test.out: vfs cache: truncate to size=6471811072
2020/12/30 10:06:28 DEBUG : test.out: vfs cache: setting modification time to 2020-12-30 10:05:34.643640186 -0500 EST
2020/12/30 10:06:28 INFO  : test.out: vfs cache: queuing for upload in 5s
2020/12/30 10:06:28 DEBUG : : Added virtual directory entry vAdd: "test.out"
2020/12/30 10:06:28 DEBUG : test1G.out: vfs cache: truncate to size=1073741824
2020/12/30 10:06:28 DEBUG : test1G.out: vfs cache: setting modification time to 2020-12-30 10:06:06.648994377 -0500 EST
2020/12/30 10:06:28 INFO  : test1G.out: vfs cache: queuing for upload in 5s
2020/12/30 10:06:28 DEBUG : : Added virtual directory entry vAdd: "test1G.out"
2020/12/30 10:06:28 DEBUG : Google drive root '': Mounting on "/home/felix/test"
2020/12/30 10:06:28 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item test.out not removed, freed 0 bytes
2020/12/30 10:06:28 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item test1G.out not removed, freed 0 bytes
2020/12/30 10:06:28 INFO  : vfs cache: cleaned: objects 2 (was 2) in use 2, to upload 2, uploading 0, total size 7.027G (was 7.027G)
2020/12/30 10:06:28 DEBUG : : Root:
2020/12/30 10:06:28 DEBUG : : >Root: node=/, err=<nil>
2020/12/30 10:06:33 DEBUG : test.out: vfs cache: starting upload
2020/12/30 10:06:33 DEBUG : test1G.out: vfs cache: starting upload
2020/12/30 10:06:34 DEBUG : test.out: Sending chunk 0 length 1073741824
2020/12/30 10:06:34 DEBUG : test1G.out: Sending chunk 0 length 1073741824
^C2020/12/30 10:06:38 INFO  : Signal received: interrupt
2020/12/30 10:06:38 DEBUG : vfs cache: cleaner exiting
2020/12/30 10:06:38 INFO  : Exiting...

and if it starts back up, you can see it starts the upload again.

felix@gemini:~$ rclone mount -vv GD: /home/felix/test --vfs-cache-mode full -vv
2020/12/30 10:07:56 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "mount" "-vv" "GD:" "/home/felix/test" "--vfs-cache-mode" "full" "-vv"]
2020/12/30 10:07:56 DEBUG : Creating backend with remote "GD:"
2020/12/30 10:07:56 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2020/12/30 10:07:56 DEBUG : vfs cache: root is "/home/felix/.cache/rclone/vfs/GD"
2020/12/30 10:07:56 DEBUG : vfs cache: metadata root is "/home/felix/.cache/rclone/vfs/GD"
2020/12/30 10:07:56 DEBUG : Creating backend with remote "/home/felix/.cache/rclone/vfs/GD"
2020/12/30 10:07:56 DEBUG : test.out: vfs cache: truncate to size=6471811072
2020/12/30 10:07:56 DEBUG : test.out: vfs cache: setting modification time to 2020-12-30 10:05:34.643640186 -0500 EST
2020/12/30 10:07:56 INFO  : test.out: vfs cache: queuing for upload in 5s
2020/12/30 10:07:56 DEBUG : : Added virtual directory entry vAdd: "test.out"
2020/12/30 10:07:56 DEBUG : test1G.out: vfs cache: truncate to size=1073741824
2020/12/30 10:07:56 DEBUG : test1G.out: vfs cache: setting modification time to 2020-12-30 10:06:06.648994377 -0500 EST
2020/12/30 10:07:56 INFO  : test1G.out: vfs cache: queuing for upload in 5s
2020/12/30 10:07:56 DEBUG : : Added virtual directory entry vAdd: "test1G.out"
2020/12/30 10:07:56 DEBUG : Google drive root '': Mounting on "/home/felix/test"
2020/12/30 10:07:56 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item test.out not removed, freed 0 bytes
2020/12/30 10:07:56 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item test1G.out not removed, freed 0 bytes
2020/12/30 10:07:56 INFO  : vfs cache: cleaned: objects 2 (was 2) in use 2, to upload 2, uploading 0, total size 7.027G (was 7.027G)
2020/12/30 10:07:56 DEBUG : : Root:
2020/12/30 10:07:56 DEBUG : : >Root: node=/, err=<nil>
2020/12/30 10:08:01 DEBUG : test.out: vfs cache: starting upload
2020/12/30 10:08:01 DEBUG : test1G.out: vfs cache: starting upload
2020/12/30 10:08:01 DEBUG : test1G.out: Sending chunk 0 length 1073741824
2020/12/30 10:08:01 DEBUG : test.out: Sending chunk 0 length 1073741824
12

Yep. That's what i wanted to know. Thank you

13

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.