Rclone size on storj returns 0 incorrectly

Help and Support
1

What is the problem you are having with rclone?

Can't get the real size of a bucket in storj. I have no folders in the bucket.

image
image1246×863 45.3 KB

Run the command 'rclone version' and share the full output of the command.

rclone version
rclone v1.60.1

  • os/version: debian 11.5 (64 bit)
  • os/kernel: 4.4.180+ (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.19.3
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

storj (41)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone size storj:docker-appdata
Total objects: 0 (0)
Total size: 0 B (0 Byte)

A log from the command with the -vv flag

rclone size storj:docker-appdata -vv
2023/03/19 10:19:53 DEBUG : rclone: Version "v1.60.1" starting with parameters ["rclone" "size" "storj:docker-appdata" "-vv"]
2023/03/19 10:19:53 DEBUG : Creating backend with remote "storj:docker-appdata"
2023/03/19 10:19:53 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/03/19 10:19:53 DEBUG : FS sj://docker-appdata: connecting...
2023/03/19 10:19:53 DEBUG : FS sj://docker-appdata: connected: <nil>
2023/03/19 10:19:53 DEBUG : FS sj://docker-appdata: ls -R ./
2023/03/19 10:19:53 DEBUG : FS sj://docker-appdata: OBJ ls -R ./ ("docker-appdata", "")
Total objects: 0 (0)
Total size: 0 B (0 Byte)
2023/03/19 10:19:55 DEBUG : 5 go routines active

Same result even if I add /

rclone size storj:docker-appdata/ -vv
2023/03/19 10:30:23 DEBUG : rclone: Version "v1.60.1" starting with parameters ["rclone" "size" "storj:docker-appdata/" "-vv"]
2023/03/19 10:30:23 DEBUG : Creating backend with remote "storj:docker-appdata/"
2023/03/19 10:30:23 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/03/19 10:30:23 DEBUG : FS sj://docker-appdata: connecting...
2023/03/19 10:30:23 DEBUG : FS sj://docker-appdata: connected: <nil>
2023/03/19 10:30:23 DEBUG : fs cache: renaming cache item "storj:docker-appdata/" to be canonical "storj:docker-appdata"
2023/03/19 10:30:23 DEBUG : FS sj://docker-appdata: ls -R ./
2023/03/19 10:30:23 DEBUG : FS sj://docker-appdata: OBJ ls -R ./ ("docker-appdata", "")
Total objects: 0 (0)
Total size: 0 B (0 Byte)
2023/03/19 10:30:25 DEBUG : 5 go routines active

Thanks in advance for any help!

2

Hmm...

Try the latest rclone see if that makes a difference.

Does rclone ls show any files?

How about rclone lsf?

Also try

rclone ls --disable ListR storj:docker-appdata
3

Hi,

I've tried rclone ls --disable ListR storj:docker-appdata

But nothing got returned.

I've also updated to the latest version and ran the command again:

rclone size storj:docker-appdata/ -vv
2023/03/19 21:10:08 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "size" "storj:docker-appdata/" "-vv"]
2023/03/19 21:10:08 DEBUG : Creating backend with remote "storj:docker-appdata/"
2023/03/19 21:10:08 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/03/19 21:10:08 DEBUG : FS sj://docker-appdata: connecting...
2023/03/19 21:10:08 DEBUG : FS sj://docker-appdata: connected: <nil>
2023/03/19 21:10:08 DEBUG : fs cache: renaming cache item "storj:docker-appdata/" to be canonical "storj:docker-appdata"
2023/03/19 21:10:08 DEBUG : FS sj://docker-appdata: ls -R ./
2023/03/19 21:10:08 DEBUG : FS sj://docker-appdata: OBJ ls -R ./ ("docker-appdata", "")
Total objects: 0 (0)
Total size: 0 B (0 Byte)
2023/03/19 21:10:09 DEBUG : 5 go routines active
4

Can you list a different bucket? Try creating one, uploading something and listing it.

Just trying to work out if your storj: config is wrong or if this is a bug.

What do you get if you do

rclone size storj:docker-appdata -vv --dump bodies

That will print more stuff, but I don't think it is very useful on storj.

Have you tried the S3 gateway?

5

Hi,

Here is teh result of the query you asked me to execute:

rclone size storj:docker-appdata -vv --dump bodies
2023/03/22 18:53:33 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "size" "storj:docker-appdata" "-vv" "--dump" "bodies"]
2023/03/22 18:53:33 DEBUG : Creating backend with remote "storj:docker-appdata"
2023/03/22 18:53:33 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/03/22 18:53:33 DEBUG : FS sj://docker-appdata: connecting...
2023/03/22 18:53:33 DEBUG : FS sj://docker-appdata: connected: <nil>
2023/03/22 18:53:33 DEBUG : FS sj://docker-appdata: ls -R ./
2023/03/22 18:53:33 DEBUG : FS sj://docker-appdata: OBJ ls -R ./ ("docker-appdata", "")
Total objects: 0 (0)
Total size: 0 B (0 Byte)
2023/03/22 18:53:34 DEBUG : 5 go routines active

Created a test bucket with a folder, same result:

image
image1237×351 14.1 KB 

rclone size storjtest:test/folder-name/ -vv
2023/03/22 18:59:54 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "size" "storjtest:test/folder-name/" "-vv"]
2023/03/22 18:59:54 DEBUG : Creating backend with remote "storjtest:test/folder-name/"
2023/03/22 18:59:54 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/03/22 18:59:54 DEBUG : FS sj://test/folder-name: connecting...
2023/03/22 18:59:54 DEBUG : FS sj://test/folder-name: connected: <nil>
2023/03/22 18:59:55 DEBUG : fs cache: renaming cache item "storjtest:test/folder-name/" to be canonical "storjtest:test/folder-name"
2023/03/22 18:59:55 DEBUG : FS sj://test/folder-name: ls -R ./
2023/03/22 18:59:55 DEBUG : FS sj://test/folder-name: OBJ ls -R ./ ("test", "folder-name")
Total objects: 0 (0)
Total size: 0 B (0 Byte)
2023/03/22 18:59:55 DEBUG : 5 go routines active

Let me know if you need something more,
thanks,
j

6

Can you post your config with secrets redacted please?

7

Hi,

Is this sufficient?

[storjtest]
type = storj
access_grant = 1cAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
8

Just checking if anyone has an idea?

9

For some reason rclone cannot see the files in your storj.

The most obvious thing is that rclone isn't looking in the right place, so spelling of bucket, access credentials, that kind of thing.

No-one has reported anything odd with Storj and rclone so its most likely to be something like that, rather than a bug.

Try uploading a file with rclone and see if you see it in the web interface.

10

Hi there,

The bucket name is correct, e.g. if I misspell the name I get a different error:

root@Synology-NAS:/# rclone size storj:docker-appdataa
2023/04/09 08:19:26 Failed to size: directory not found

Try uploading a file with rclone and see if you see it in the web interface.

I have tried that. I've created a completely new bucket, folder, uploaded a 100MB+ file and other results are still the same.

image
image991×426 17.8 KB

Below notice the different error output if I use an existing vs wrong bucket name and/or folder name.

Correct bucket names:

root@Synology-NAS:/# rclone size storj:docker-appdata -vv
2023/04/09 08:24:26 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "size" "storj:docker-appdata" "-vv"]
2023/04/09 08:24:26 DEBUG : Creating backend with remote "storj:docker-appdata"
2023/04/09 08:24:26 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/04/09 08:24:26 DEBUG : FS sj://docker-appdata: connecting...
2023/04/09 08:24:26 DEBUG : FS sj://docker-appdata: connected: <nil>
2023/04/09 08:24:26 DEBUG : FS sj://docker-appdata: ls -R ./
2023/04/09 08:24:26 DEBUG : FS sj://docker-appdata: OBJ ls -R ./ ("docker-appdata", "")
Total objects: 0 (0)
Total size: 0 B (0 Byte)
2023/04/09 08:24:27 DEBUG : 5 go routines active
root@Synology-NAS:/# rclone size storj:test -vv
2023/04/09 08:24:35 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "size" "storj:test" "-vv"]
2023/04/09 08:24:35 DEBUG : Creating backend with remote "storj:test"
2023/04/09 08:24:35 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/04/09 08:24:35 DEBUG : FS sj://test: connecting...
2023/04/09 08:24:35 DEBUG : FS sj://test: connected: <nil>
2023/04/09 08:24:35 DEBUG : FS sj://test: ls -R ./
2023/04/09 08:24:35 DEBUG : FS sj://test: OBJ ls -R ./ ("test", "")
Total objects: 0 (0)
Total size: 0 B (0 Byte)
2023/04/09 08:24:35 DEBUG : 5 go routines active
root@Synology-NAS:/# rclone size storj:test -vv
2023/04/09 08:25:16 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "size" "storj:test" "-vv"]
2023/04/09 08:25:16 DEBUG : Creating backend with remote "storj:test"
2023/04/09 08:25:16 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/04/09 08:25:16 DEBUG : FS sj://test: connecting...
2023/04/09 08:25:16 DEBUG : FS sj://test: connected: <nil>
2023/04/09 08:25:16 DEBUG : FS sj://test: ls -R ./
2023/04/09 08:25:16 DEBUG : FS sj://test: OBJ ls -R ./ ("test", "")
Total objects: 0 (0)
Total size: 0 B (0 Byte)

Incorrect bucket names:


2023/04/09 08:25:16 DEBUG : 5 go routines active
root@Synology-NAS:/# rclone size storja:test -vv
2023/04/09 08:25:24 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "size" "storja:test" "-vv"]
2023/04/09 08:25:24 DEBUG : Creating backend with remote "storja:test"
2023/04/09 08:25:24 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/04/09 08:25:24 Failed to create file system for "storja:test": didn't find section in config file
root@Synology-NAS:/# rclone size storj:testa -vv
2023/04/09 08:25:33 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "size" "storj:testa" "-vv"]
2023/04/09 08:25:33 DEBUG : Creating backend with remote "storj:testa"
2023/04/09 08:25:33 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/04/09 08:25:33 DEBUG : FS sj://testa: connecting...
2023/04/09 08:25:33 DEBUG : FS sj://testa: connected: <nil>
2023/04/09 08:25:33 DEBUG : FS sj://testa: ls -R ./
2023/04/09 08:25:33 DEBUG : FS sj://testa: OBJ ls -R ./ ("testa", "")
2023/04/09 08:25:34 DEBUG : 5 go routines active
2023/04/09 08:25:34 Failed to size: directory not found

Correct bucket name and folder combination:

root@Synology-NAS:/# rclone size storj:test -vv
2023/04/09 08:25:47 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "size" "storj:test" "-vv"]
2023/04/09 08:25:47 DEBUG : Creating backend with remote "storj:test"
2023/04/09 08:25:47 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/04/09 08:25:47 DEBUG : FS sj://test: connecting...
2023/04/09 08:25:47 DEBUG : FS sj://test: connected: <nil>
2023/04/09 08:25:47 DEBUG : FS sj://test: ls -R ./
2023/04/09 08:25:47 DEBUG : FS sj://test: OBJ ls -R ./ ("test", "")
Total objects: 0 (0)
Total size: 0 B (0 Byte)
2023/04/09 08:25:48 DEBUG : 5 go routines active
root@Synology-NAS:/# rclone size storj:test/folder-name -vv
2023/04/09 08:26:01 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "size" "storj:test/folder-name" "-vv"]
2023/04/09 08:26:01 DEBUG : Creating backend with remote "storj:test/folder-name"
2023/04/09 08:26:01 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2023/04/09 08:26:01 DEBUG : FS sj://test/folder-name: connecting...
2023/04/09 08:26:01 DEBUG : FS sj://test/folder-name: connected: <nil>
2023/04/09 08:26:02 DEBUG : FS sj://test/folder-name: ls -R ./
2023/04/09 08:26:02 DEBUG : FS sj://test/folder-name: OBJ ls -R ./ ("test", "folder-name")
Total objects: 0 (0)
Total size: 0 B (0 Byte)
2023/04/09 08:26:02 DEBUG : 5 go routines active
root@Synology-NAS:/#
11

Try uploading a file with rclone and see if you see it in the web interface.

Hold on - I misread this sentence, let me try that

12

I think I've figured it out.

image
image415×547 28.8 KB

I think I've created 2 different set of access credentials and they can't read each-others data.

Thanks for helping me figure this out.

13

Ah ha! That makes sense. Well done for figuring out out.

14

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.