Machine ran out of local storage to cache copy files

Help and Support
1

What is the problem you are having with rclone?

The files I want to copy between mounted drive and mounted drive exceed the local drive storage, hence causing this io error.
16acf6ab-8037-425e-98a3-db7fa7d55d42

I have tried not to use vfs-cache-mode full, but it just became painfully slow to do anything, is there anything I could do about this or is the only solution is just to increase the local storage.

Rclone version

rclone v1.55.0-beta.5241.0ad38dd6f

  • os/arch: windows/amd64
  • go version: go1.16

Which cloud storage system are you using? (eg Google Drive)

AWS S3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

I am using nssm to manage the service but this command looks like:

rclone --config "C:\\rclone\\rclone.conf" mount s3:{bucket_name}/research/shared P: --volname \\\\ml-lab\\research\\shared --vfs-cache-mode full --network-mode

The rclone config contents with secrets removed.

[s3]
type = s3
provider = AWS
env_auth = false
region = us-east-1
server_side_encryption = AES256
access_key_id = 
secret_access_key = 
session_token =
2

hello and welcome to the forum,

the screen snippet shows an 0x8007045D, not an error due to lack of free space.
what makes you think the lack of free space is the issue?

3

Thanks,

I had managed to reproduce the error multiple times. The error only occurred when the disk space available is lesser than the files I want to copy, hence, led me to believe it is a disk space caching error. But I am open to any suggestion on what this error could be. Thanks in advance.

4

You have a beta version so you'd want to update the latest stable.

You'd want to generate a debug log file and share the full log.

If you don't have enough storage to use full mode, you are correct as you can't copy a file as it makes a local copy and uploads it.

If you remove full, depending on your app, it may or may not work (again log file would clearly show this) which is why we ask for a log file..

5

without a debug log, hard to know exactly what is going on.

if you do not use --vfs-cache-mode, then rclone will not use local storage to cache the file.
and this is what i see, that rclone copies the file without using local stoarge.

DEBUG : zork/file.txt: md5 = 225921b9f014321f3bde2bab02d2a6a8 OK
INFO  : zork/file.txt: Copied (new)

but as @Animosity022 mentioned, without the cache, you might see this

ERROR : zork/file.txt: WriteFileHandle: Truncate: Can't change size without --vfs-cache-mode >= writes
6

Since I am using windows registry to host the rclone, is there any feasible way to get the debug log?

7

Since I am using windows registry to host the rclone, is there any feasible way to get the debug log?

8

sorry, not understanding, how you are using windows registry to host the rclone?
and why would that affect using a rclone debug log?

to produce a debug log, --log-level=DEBUG --log-file=c:\path\to\local\folder\rclone.log
change the log file location

9

Ahh, ok, mistaken debug log with the flag -vv. Will provide a debug log in the next reply. Thanks.

how you are using windows registry to host the rclone?

Host might not be the right word, I use nssm to turn it into a service and run it from script

10

ok, that has nothing to do with your issue.

11

The debug log, controlc, have to use controlc as it exceeded the limit in pastebin

12

You still have some odd beta version running. Please update and use the latest stable.

1 Like
13

if you do not have enough free space to use rclone mount and cannot add some storage, external usb or whatever,
then use rclone copy/move/sync.

also, need to fix this issue s3 issue.

Failed to copy: s3 upload: 403 Forbidden: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied

and need to update rclone to v1.57.0

14

OK, I will update rclone.

if you do not have enough free space to use rclone mount and cannot add some storage, external usb or whatever,

But I thought mounting as a network drive solves this issue? Sorry if I am being ignorant.

15

the only advantage, that i know about, for network mode, is faster navigation using windows explorer.

that windows os will not try to create thumbnails and perhaps some other such minor tweaks.

1 Like
16

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.