Unable to mount Storj buckets to Linux. No files visible even if mount succeeds

arthaus · June 15, 2022, 8:20pm

What is the problem you are having with rclone?

Unable to see contents of storj buckets after mounting.

I have been trying to get my storj mounted on Linux (Debian 11.3). I'm able to follow the config wizard and make a connection to my storj buckets. Problem is in the mounting part. Sometimes the mount goes through successfully and sometimes the terminal freezes, but I can never see the contents of my buckets. If I'm lucky sometimes I'm able to see the buckets, but never whats in it. I'm not using rclone as root.

Would really appreciate any insight into what's wrong, if there's something I'm doing incorrectly or whether its a bug.

Run the command 'rclone version' and share the full output of the command.

rclone v1.58.1

os/version: debian 11.3 (64 bit)
os/kernel: 5.10.0-13-amd64 (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.17.9
go/linking: static
go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Storj DCS

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone mount arthaus_storj3 /etc/levideo --allow-other --log-file=/etc/rclonelog/storjmountlog.txt --log-level DEBUG

The rclone config contents with secrets removed.


[arthaus_storj3]
type = storj
access_grant = [my access grant key]

A log from the command with the `-vv` flag

2022/06/15 20:08:04 DEBUG : rclone: Version "v1.58.1" starting with parameters ["rclone" "mount" "arthaus_storj3" "/etc/levideo" "--allow-other" "--log-file=/etc/rclonelog/storjmountlog.txt" "--log-level" "DEBUG"]
2022/06/15 20:08:04 DEBUG : Creating backend with remote "arthaus_storj3"
2022/06/15 20:08:04 DEBUG : Using config file from "/home/vsuvarna/.config/rclone/rclone.conf"
2022/06/15 20:08:04 DEBUG : fs cache: renaming cache item "arthaus_storj3" to be canonical "/home/arthaus_storj3"
2022/06/15 20:08:04 INFO  : Local file system at /home/arthaus_storj3: poll-interval is not supported by this remote
2022/06/15 20:08:04 DEBUG : Local file system at /home/arthaus_storj3: Mounting on "/etc/levideo"
2022/06/15 20:08:04 DEBUG : : Root: 
2022/06/15 20:08:04 DEBUG : : >Root: node=/, err=<nil>
2022/06/15 20:09:25 DEBUG : : forgetting directory cache
2022/06/15 20:12:23 INFO  : Signal received: terminated
2022/06/15 20:12:23 ERROR : /etc/levideo: Failed to unmount: signal: hangup
2022/06/15 20:12:23 INFO  : Exiting...

Animosity022 · June 15, 2022, 8:22pm

You need a ":" after a remote name.

To

rclone mount arthaus_storj3: /etc/levideo --allow-other --log-file=/etc/rclonelog/storjmountlog.txt --log-level DEBUG

arthaus · June 15, 2022, 8:39pm

Thank you animosity for such a quick reply. I tried adding the : but I'm still not able to get it to work. Everytime I enter the mount command, the terminal stops responding. My SSH has also stopped working.

Here's the most recent log from the updated mount command. Forgive me for attaching an image instead of pasting the code. My SSH isn't working so the screenshot is from a graphical shell.

Animosity022 · June 15, 2022, 8:48pm

The mount command won't ever return a command prompt as it's "running".

You have to decide how you want it run it in the background if that is your goal.

Some folks use a service file.
Some folks use --daemon

It's really up to you.

If you run:

rclone ls arthaus_storj3:

It should show you output and if not, run that command and add -vv on the end share the full log.

arthaus · June 15, 2022, 8:59pm

Hi animosity, I gave it another try, this time using the --daemon flag as you suggested. It did not freeze up this time and I have a successful mount. I can see both my buckets listed under the directory, but nothing beyond that. Still can't see any of my sub folders or files.

Here's the current log:

2022/06/15 20:53:58 DEBUG : /: >Lookup: node=arthaus-library/, err=<nil>
2022/06/15 20:53:58 DEBUG : arthaus-library/: Attr: 
2022/06/15 20:53:58 DEBUG : arthaus-library/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2022/06/15 20:54:00 DEBUG : /: Attr: 
2022/06/15 20:54:00 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2022/06/15 20:54:00 DEBUG : /: ReadDirAll: 
2022/06/15 20:54:00 DEBUG : /: >ReadDirAll: item=4, err=<nil>
2022/06/15 20:54:00 DEBUG : /: Attr: 
2022/06/15 20:54:00 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2022/06/15 20:54:00 DEBUG : /: Lookup: name="arthaus-library"
2022/06/15 20:54:00 DEBUG : /: >Lookup: node=arthaus-library/, err=<nil>
2022/06/15 20:54:00 DEBUG : arthaus-library/: Attr: 
2022/06/15 20:54:00 DEBUG : arthaus-library/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2022/06/15 20:54:39 DEBUG : arthaus-library/: Attr: 
2022/06/15 20:54:39 DEBUG : arthaus-library/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2022/06/15 20:54:39 DEBUG : arthaus-library/: ReadDirAll: 
2022/06/15 20:54:39 DEBUG : FS sj://: ls ./arthaus-library
2022/06/15 20:54:39 DEBUG : FS sj://: OBJ ls ./arthaus-library ("arthaus-library", "")
2022/06/15 20:54:39 DEBUG : FS sj://: opts &{Prefix: Cursor: Recursive:false System:true Custom:true}
2022/06/15 20:54:40 DEBUG : arthaus-library/: >ReadDirAll: item=2, err=<nil>
2022/06/15 20:54:50 DEBUG : /: Attr: 
2022/06/15 20:54:50 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2022/06/15 20:54:50 DEBUG : /: Lookup: name="arthaus-library"
2022/06/15 20:54:50 DEBUG : /: >Lookup: node=arthaus-library/, err=<nil>
2022/06/15 20:54:50 DEBUG : arthaus-library/: Attr: 
2022/06/15 20:54:50 DEBUG : arthaus-library/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2022/06/15 20:54:53 DEBUG : /: Attr: 
2022/06/15 20:54:53 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>

Animosity022 · June 15, 2022, 9:02pm

If you could please do that.

arthaus · June 15, 2022, 9:09pm

Sorry to have missed your previous request. Here's the output the ls command with a -vv flag:

<7>DEBUG : rclone: Version "v1.58.1" starting with parameters ["rclone" "ls" "arthaus_storj3:" "-vv"]
<7>DEBUG : rclone: systemd logging support activated
<7>DEBUG : Creating backend with remote "arthaus_storj3:"
<7>DEBUG : Using config file from "/home/vsuvarna/.config/rclone/rclone.conf"
<7>DEBUG : FS sj://: connecting...
<7>DEBUG : FS sj://: connected: <nil>
<7>DEBUG : FS sj://: ls -R ./
<7>DEBUG : FS sj://: BKT ls -R
<7>DEBUG : FS sj://: OBJ ls -R ./arthaus-injest ("arthaus-injest", "")
<7>DEBUG : FS sj://: OBJ ls -R ./arthaus-library ("arthaus-library", "")
<7>DEBUG : 5 go routines active

Thank you for taking the time to help me with this.

Animosity022 · June 15, 2022, 9:12pm

So is that what you would expect?

Looks like two buckets I'd imagine? No files?

Your output also looks bizarre. How are you running that? Normally output looks like:

felix@gemini:~$ rclone ls GD: -vv
2022/06/15 17:12:21 DEBUG : Setting --config "/opt/rclone/rclone.conf" from environment variable RCLONE_CONFIG="/opt/rclone/rclone.conf"
2022/06/15 17:12:21 DEBUG : rclone: Version "v1.58.1" starting with parameters ["rclone" "ls" "GD:" "-vv"]
2022/06/15 17:12:21 DEBUG : Creating backend with remote "GD:"
2022/06/15 17:12:21 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2022/06/15 17:12:22 DEBUG : Google drive root '': 'root_folder_id = 0AGoj85v3xeadUk9PVA' - save this in the config to speed up startup
       -1 Joeisms.docx
      278 hosts
        0 test.tar
       -1 TestDoc.docx
      184 testcopy
   306520 RDManager
1504953150 jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv
       -1 test/testsheet.xlsx
2022/06/15 17:12:22 DEBUG : 10 go routines active

As an example.

arthaus · June 15, 2022, 9:18pm

So is that what you would expect?
Looks like two buckets I'd imagine? No files?

arthaus-injest and arthaus-library are the 2 buckets on my storj account, so that part looks okay. I do have files and folders in both buckets, so that is the part I've been getting stuck on for weeks.

Your output also looks bizarre. How are you running that?

Well, my SSH broke shortly after I made this forum post, so currently I'm depending on a weblish/glish graphical shell. I'll get around to troubleshooting SSH tomorrow.

Animosity022 · June 15, 2022, 9:20pm

@ncw - is there anything else you'd recommend to grab? I'm not super familiar with this backend.

pasnox · June 16, 2022, 11:20am

I would suggest you pass through the S3 gateway instead. It's more simple and usable than the native interface that is mostly for enterprise with big bandwidth.
Also, take care that you use the same passphrase for the various access (web, S3, native), else you will not see them!!!

arthaus · June 16, 2022, 4:27pm

I was able to fix this problem with the help of the kind folk over at Storj. It was a decryption key issue. My account has 2 buckets, so it needs 2 different decryption keys. The solution was to make 2 access grants for rclone and mount the buckets one by one. Now with the correct key, I can finally see the contents of the buckets.

Thank you to animosity for being so patient and helping me troubleshoot this yesterday. Sorry for the false alarm. There's no bug in rclone, its beautiful and perfect as is

pasnox · June 23, 2022, 5:14pm

Just like I said

system · July 23, 2022, 5:15pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.