Cannot access some files stored on internet archive

What is the problem you are having with rclone?

Some file seem not to be accessible in some program. Using Retroarch with o2em libretro core which uses bios file o2rom.bin. I have mounted retroarch bios folder from internet archive. When I have that file on local folder without rclone, there is no problem reading that file.

the command I am using:

params="--no-modtime --attr-timeout 10h --dir-cache-time 10h --poll-interval 10h --vfs-cache-mode full --allow-non-empty --daemon"
rclone mount "archive:retroarch-bios" ~/.config/retroarch/system $params

It is strange that it works on my x86 machine now (Atari VCS).

Run the command 'rclone version' and share the full output of the command.

rclone v1.61.1

  • os/version: ubuntu 22.10 (64 bit)
  • os/kernel: 6.0.0-cadmium (aarch64)
  • os/type: linux
  • os/arch: arm64
  • go/version: go1.19.4
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Internet Archive

The rclone config contents with secrets removed.

[archive]
type = internetarchive
access_key_id = deleted
secret_access_key = deleted

[myrient]
type = ftp
host = ftp.myrient.erista.me
tls = false
explicit_tls = true
pass = myUcMnWBKX9R-Gya--f8j0K26zYNvaWCqyqL
user = anonymous

[thumbnails]
type = http
url = https://thumbnails.libretro.com

Retroarch log

[INFO] [Environ]: SYSTEM_DIRECTORY: "/home/wizzard/.config/retroarch/system/".
[libretro WARN] [O2EM]: o2rom.bin not found, cannot load BIOS

Can you add a log from rclone with -vv when Retroarch tries to access the file please?

1 Like

Thank you very much for the reply!

I mounted that share with -vv flag and this is what was logged:

2023/01/20 17:04:22 DEBUG : rclone: Version "v1.61.1" starting with parameters ["rclone" "mount" "archive:retroarch-bios" "/home/wizzard/.config/retroarch/system" "-vv" "--log-file" "/tmp/rclone.log"]
2023/01/20 17:04:22 DEBUG : Creating backend with remote "archive:retroarch-bios"
2023/01/20 17:04:22 DEBUG : Using config file from "/home/wizzard/.config/rclone/rclone.conf"
2023/01/20 17:04:25 INFO  : Internet Archive item retroarch-bios: poll-interval is not supported by this remote
2023/01/20 17:04:25 NOTICE: Internet Archive item retroarch-bios: --vfs-cache-mode writes or full is recommended for this remote as it can't stream
2023/01/20 17:04:25 DEBUG : Internet Archive item retroarch-bios: Mounting on "/home/wizzard/.config/retroarch/system"
2023/01/20 17:04:25 DEBUG : : Root: 
2023/01/20 17:04:25 DEBUG : : >Root: node=/, err=<nil>
2023/01/20 17:10:05 DEBUG : /: Attr: 
2023/01/20 17:10:05 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/01/20 17:10:08 DEBUG : /: Lookup: name="o2rom.bin"
2023/01/20 17:10:10 DEBUG : /: >Lookup: node=o2rom.bin, err=<nil>
2023/01/20 17:10:10 DEBUG : o2rom.bin: Attr: 
2023/01/20 17:10:10 DEBUG : o2rom.bin: >Attr: a=valid=1s ino=0 size=1024 mode=-rw-rw-r--, err=<nil>

It is possible that given libretro core is opening that bios file some unusual way?

This shows that the file exists, but no attempt to open it or access it.

I guess.

Are you 100% sure it is looking in the right place?

You can try to strace it to see what error it gets back from the OS.

Yes, it am using the same folder with mounted files as I tried with file physically in that folder.
Also, it works with the same mount script on another computer (x86 vs arm, so also that libretro core version is a little different)
Will try that command, thank you.

I logged strace to file and this is what I found about that file:

write(3, "[INFO] [Content]: Updating firmw"..., 150) = 150
statx(AT_FDCWD, "/home/wizzard/.config/retroarch/system/o2rom.bin", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT, STATX_BASIC_STATS, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0664, stx_size=1024, ...}) = 0
statx(AT_FDCWD, "/home/wizzard/.config/retroarch/system/c52.bin", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT, STATX_BASIC_STATS, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0664, stx_size=0, ...}) = 0
statx(AT_FDCWD, "/home/wizzard/.config/retroarch/system/g7400.bin", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT, STATX_BASIC_STATS, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0664, stx_size=0, ...}) = 0
statx(AT_FDCWD, "/home/wizzard/.config/retroarch/system/jopac.bin", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT, STATX_BASIC_STATS, {stx_mask=STATX_BASIC_STATS|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0664, stx_size=0, ...}) = 0
write(3, "[INFO] [Core]: Using content: \"/"..., 87) = 87
write(3, "[INFO] [Core]: Content ran for a"..., 77) = 77
write(3, "[INFO] [Core]: Unloading core..\n", 32) = 32
write(3, "[INFO] [Core]: Unloading core sy"..., 40) = 40
write(3, "[INFO] [Content]: Content succes"..., 87) = 87
write(3, "[INFO] [Content]: Content loadin"..., 83) = 83
write(3, "[INFO] [Environ]: SET_PIXEL_FORM"..., 44) = 44
write(3, "[INFO] [Environ]: SET_INPUT_DESC"..., 41) = 41
write(3, "[INFO] [Environ]: SYSTEM_DIRECTO"..., 78) = 78
stat64("/home/wizzard/.config/retroarch/system/o2rom.bin", 0xffe86ca0) = 0
write(3, "[libretro WARN] [O2EM]: o2rom.bi"..., 62) = 62
write(3, "[INFO] [Content]: Removing tempo"..., 85) = 85
unlink("/tmp/4 en 1 Ligne (France).bin") = 0
write(3, "[ERROR] [Content]: Failed to loa"..., 42) = 42
write(3, "[INFO] [Core]: Content ran for a"..., 77) = 77
write(3, "[INFO] [Core]: Unloading core..\n", 32) = 32
write(3, "[INFO] [Core]: Unloading core sy"..., 40) = 40

Apparently the file was found OK here (the = 0) but

It gave the not found error which is weird.

This could be a permissions problem? Are you running the mount as the same user running the emulator?

Yes, it is really odd, I mount that without root with just my user, also run all the programs as my user. I checked mounted file perms and I have the permissions to read and write it.

I would start tracing the source code of libretro now to see exactly what it is doing to give that warning message...

Yes, I even tried to check that libretro core libretro-o2em/libretro.c at master · libretro/libretro-o2em · GitHub sources to find out how it reads the bios files, but I am not so good at C to find out what is wrong. Plus, I dont know why it does not work on my arm machine and works on x86 machine. I even commented some lines in the sources which I considered problematic and built that libretro core, but it did not help :slight_smile:

The error comes from here

Which in turn calls path_is_valid

Which in turn calls path_stat_cb. This is indirected with lots of implementations for different OSes.

I think this is the code which runs

which looks OK (provided it is the code which is actually running!).

As far as I can see rclone is working - the stat returns sensible results, but libretro rejects it for some reason.

I'm sorry I don't have time to dig any more into the C code.

Does it work on your ARM machine without using rclone mount?

Yes, it works on my arm machine without using rclone mount.

That is indeed interesting that it uses different code for different platforms, I may play with that and try to make a fix for that.

Or I will just not be using this emulator like this. I created an issue in that libretro project. Thank you very much for the help.

1 Like