Rclone stopped uploading (but still alive) after MacBook Pro woke up from sleep

MaxC · November 4, 2021, 12:18am

What is the problem you are having with rclone?

After my M1 Max laptop running MacOS Monterey 12.0.1 went to sleep (with lid open), the Rclone (v1.56.2) upload session is still alive, but no longer uploading. I know it's not the latest Rclone (1.57.0 is out now), but it's the one currently stuck. The "Elapsed time" is still going. Is it possible not to lose this upload?

Transferred:       87.928Gi / 172.373 GiByte, 51%, 0 Byte/s, ETA -1s
Checks:                 3 / 3, 100%
Transferred:            1 / 4, 25%
Elapsed time:   9h21m20.3s
Transferring:
 *                         ******************.7z: 80% /31.562Gi, 0/s, 0s
 *        *******************************.tar.gz: 32% /77.789Gi, 0/s, 0s
 *    **************************************.zip: 50% /50.956Gi, 0/s, 0s

What is your rclone version (output from `rclone version`)

❯ rclone --version
rclone v1.56.2
- os/version: darwin 12.0.1 (64 bit)
- os/kernel: 21.1.0 (arm64)
- os/type: darwin
- os/arch: arm64
- go/version: go1.17.2
- go/linking: dynamic
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Backblaze B2 (via crypt)

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone copy --transfers=50 -P --fast-list --log-level=ERROR \
  --b2-account=[redacted] --b2-key=[redacted] \
  "Path/To/Dir" :crypt:DestDir \
  --crypt-remote=:b2:[redacted]/Encrypted \
  --crypt-filename-encryption=off \
  --crypt-directory-name-encryption=false \
  --crypt-password=[redacted] \
  --crypt-password2=[redacted]

The rclone config contents with secrets removed.

No config. The above command is self-sufficient.

A log from the command with the `-vv` flag

N/A

Animosity022 · November 4, 2021, 12:21am

Doubtful as I assume at this point all the sessions got closed/timed out. I'd assume it will error out at some point or potentially retry but the log level is pretty low to figure out what is going on.

MaxC · November 4, 2021, 12:29am

Thank you, yeah I didn't get my hopes up, but figured it's worth asking. This isn't the first time it happened, but last time it was with lid closed. Was hoping keeping lid open would keep it running.

Do you know of any guide for ensuring upload stability on laptops or unstable connections?

Animosity022 · November 4, 2021, 12:35am

A few flags may help:

    --low-level-retries int                Number of low level retries to do (default 10)

and

  --retries int                          Retry operations this many times if they fail (default 3)

Depending on how bad the connection is, you can turn them up, but the downside is, it does take longer to fail since it keeps trying things.

MaxC · November 4, 2021, 12:49am

Ah makes sense for unstable connections. I imagine this still wouldn't cross the "sleep" boundary. Right now rclone is still running (about 2 hours since it got stuck), which suggests that retries aren't pertinent here. Doesn't seem like it will ever be timing out. Perhaps it's some kind of MacOS's resume-after-sleep mechanism that it's not compatible with.

Animosity022 · November 4, 2021, 12:56am

I can't say I'm sure how sleeping/resuming would work.

You could probably test but setting the log level a little higher, using --log-file something.log and close/open it and see what gets logged when it resumes.

MaxC · November 4, 2021, 1:20am

Holy crap, it just resumed. I randomly kept it on with no hope, and it just decided to continue on its own.

Transferred:   	   89.505Gi / 172.373 GiByte, 52%, 3.291 MiByte/s, ETA 7h9m42s
Checks:                 3 / 3, 100%
Transferred:            1 / 4, 25%
Elapsed time:  10h42m28.3s
Transferring:
 *                         ******************.7z: 82% /31.562Gi, 1.329Mi/s, 1h12m29s
 *        *******************************.tar.gz: 32% /77.789Gi, 473.168Ki/s, 32h20m29s
 *    **************************************.zip: 51% /50.956Gi, 1.500Mi/s, 4h40m49s

Wat.

So I guess to anyone finding this thread: keep the thing on for like 3 hours and it should be… okay?

MaxC · November 4, 2021, 1:34am

In an effort to prevent a wild goose chase (since I "conveniently" disabled logs ) — I just realized that there could be another explanation.

This could have been caused by a switch from wired to wifi connection. Before laptop fell asleep, I unplugged it from my thunderbolt monitor that also serves as a hub to a wired network adapter. Rclone might have stopped working at that time. Eventually it might've "realized" that it needs to use another network interface.

Just another possibility.

MaxC · November 4, 2021, 1:42pm

Morning update: I woke up to find rclone in this status:

Transferred:      189.702Gi / 186.326 GiByte, 102%, 0 Byte/s, ETA -
Checks:                 3 / 3, 100%
Transferred:            3 / 4, 75%
Elapsed time:  22h39m17.3s
Transferring:
 *        *******************************.tar.gz:104% /77.789Gi, 0/s, -

104% seems like a bug. As before, the timer keeps going. In B2 interface the file doesn't show up as completed yet, there's only the stub saying "started large file" for this file. The others are completed successfully. I guess I'll keep waiting.

Animosity022 · November 4, 2021, 1:46pm

Nah, retries. If it has to retry a bunch with errors, it will transfer more than you had started with.

Log files really make these questions easy as you can see the issue in there.

MaxC · November 4, 2021, 3:28pm

Update: it finished.

Moral of the story: when rclone is sitting there and doing nothing, give it some personal space, don't poke it, it will eventually come around by itself. It just needs some rest, some alone time.

Animosity022 · November 4, 2021, 3:33pm

You can trace a couple things as well to see what's going on.

I would use lsof and you can find the process.

Works pretty much the same on Linux/Mac as you need sudo for lsof:

felix@gemini:~$ ps -ef | grep rclone
felix        718       1  1 11:09 ?        00:00:25 /usr/bin/rclone mount gcrypt: /GD --allow-other --dir-cache-time 5000h --attr-timeout 5000h --log-file /opt/rclone/logs/rclone.log --log-level NOTICE --poll-interval 10s --umask 002 --user-agent someappname101 --rc --rc-addr :5572 --rc-no-auth --cache-dir=/tmp/cache --drive-pacer-min-sleep 10ms --drive-pacer-burst 200 --vfs-cache-mode full --vfs-cache-max-size 500G --vfs-cache-max-age 5000h --vfs-cache-poll-interval 5m --vfs-read-ahead 2G --bwlimit-file 32M
felix       4577    4373  0 11:32 pts/0    00:00:00 grep rclone
felix@gemini:~$ sudo lsof -p 718
COMMAND PID  USER   FD      TYPE             DEVICE SIZE/OFF     NODE NAME
rclone  718 felix  cwd       DIR                8,2     4096        2 /
rclone  718 felix  rtd       DIR                8,2     4096        2 /
rclone  718 felix  txt       REG                8,2 41893888 39068567 /usr/bin/rclone
rclone  718 felix    0r      CHR                1,3      0t0        4 /dev/null

You can see what connections it has open from the process level. I'd imagine, it pauses and eventually reconnects based on the retry.

system · December 4, 2021, 3:34pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.