Can't copy rainbow tables

Pascal666 · November 29, 2023, 8:43pm

Sorry, I don't know much about rclone and don't have access to most of what the below template is asking for. I only have access to Rclone WebUI. No idea if the below is a problem with rclone, Rclone WebUI, or my provider's setup.

What is the problem you are having with rclone?

Using the Explorer tab of Rclone WebUI I can see 8 directories in Index of /rainbow/RTI2/lm/ but when I try to view inside them I see nothing and copying them does not work. I can see and download files from inside them just fine in Chrome.

Run the command 'rclone version' and share the full output of the command.

Sorry, I don't have command line access.
Rclone WebUI says:

**Arch:** amd64
**goVersion:** go1.21.4
**OS:** linux
**Rclone version:** v1.65.0-DEV
**isGit:** true

Which cloud storage system are you using? (eg Google Drive)

http

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

Sorry, no idea what commands Rclone WebUI sends to rclone.

The rclone config contents with secrets removed.

This is the only part I can access:

[lm]
type = http
url = https://freerainbowtables.mirror.garr.it/rainbow/RTI2/lm/

A log from the command with the `-vv` flag

Sorry, I don't have access to this.

kapitainsky · November 29, 2023, 8:51pm

It is highly possible that there is this and many other bugs in rclone WebUI.

And probably more bugs with time passing. This is unfortunately normal situation for abandoned projects.

Until somebody decides to spend time and effort to revive it you can consider rclone WebUI dead.

My recommendation is to use standard rclone command line program, e.g. for issue you described start with:

rclone lsf lm:

to see what is in the root directory of your remote.

Then you can check specific folder:

rclone lsf lm:folder1/folder2

and copy its content to local drive:

rclone copy lm:folder1/folder2 /path/to/local/folder

asdffdsa · November 29, 2023, 8:55pm

welcome to the forum,

i can replicate the issue without using rclone webui.
the issue does not appear to be with the rclone webui.

can list root dir
cannot list files in subdir
tho able to download the files

rclone lsd :http,url='https://freerainbowtables.mirror.garr.it':rainbow/RTI2/lm/
          -1 1999-12-31 19:00:00        -1 lm_all-space#1-7_0
          -1 1999-12-31 19:00:00        -1 lm_all-space#1-7_1
          -1 1999-12-31 19:00:00        -1 lm_all-space#1-7_2
          -1 1999-12-31 19:00:00        -1 lm_all-space#1-7_3
          -1 1999-12-31 19:00:00        -1 lm_lm-frt-cp437-850#1-7_0
          -1 1999-12-31 19:00:00        -1 lm_lm-frt-cp437-850#1-7_1
          -1 1999-12-31 19:00:00        -1 lm_lm-frt-cp437-850#1-7_2
          -1 1999-12-31 19:00:00        -1 lm_lm-frt-cp437-850#1-7_3

rclone ls :http,url='https://freerainbowtables.mirror.garr.it':rainbow/RTI2/lm/ --retries=1 -vv
2023/11/29 15:53:22 DEBUG : rclone: Version "v1.65.0" starting with parameters ["rclone" "ls" ":http,url='https://freerainbowtables.mirror.garr.it':rainbow/RTI2/lm/" "--retries=1" "-vv"]
2023/11/29 15:53:22 DEBUG : Creating backend with remote ":http,url='https://freerainbowtables.mirror.garr.it':rainbow/RTI2/lm/"
2023/11/29 15:53:22 DEBUG : Using config file from "c:\\data\\rclone\\rclone.conf"
2023/11/29 15:53:22 DEBUG : :http: detected overridden config - adding "{1e9yy}" suffix to name
2023/11/29 15:53:22 DEBUG : Root: https://freerainbowtables.mirror.garr.it/rainbow/RTI2/lm/
2023/11/29 15:53:22 DEBUG : fs cache: renaming cache item ":http,url='https://freerainbowtables.mirror.garr.it':rainbow/RTI2/lm/" to be canonical ":http{1e9yy}:rainbow/RTI2/lm/"
2023/11/29 15:53:22 ERROR : lm_all-space#1-7_3: error listing: error listing "lm_all-space#1-7_3/": failed to readDir: HTTP Error: 503 Service Unavailable
2023/11/29 15:53:22 ERROR : lm_lm-frt-cp437-850#1-7_3: error listing: error listing "lm_lm-frt-cp437-850#1-7_3/": failed to readDir: HTTP Error: 503 Service Unavailable
2023/11/29 15:53:22 ERROR : lm_all-space#1-7_1: error listing: error listing "lm_all-space#1-7_1/": failed to readDir: HTTP Error: 503 Service Unavailable
2023/11/29 15:53:22 ERROR : lm_all-space#1-7_0: error listing: error listing "lm_all-space#1-7_0/": failed to readDir: HTTP Error: 503 Service Unavailable
2023/11/29 15:53:22 ERROR : lm_lm-frt-cp437-850#1-7_1: error listing: error listing "lm_lm-frt-cp437-850#1-7_1/": failed to readDir: HTTP Error: 503 Service Unavailable
2023/11/29 15:53:22 ERROR : lm_all-space#1-7_2: error listing: error listing "lm_all-space#1-7_2/": failed to readDir: HTTP Error: 503 Service Unavailable
2023/11/29 15:53:22 ERROR : lm_lm-frt-cp437-850#1-7_0: error listing: error listing "lm_lm-frt-cp437-850#1-7_0/": failed to readDir: HTTP Error: 503 Service Unavailable
2023/11/29 15:53:22 ERROR : lm_lm-frt-cp437-850#1-7_2: error listing: error listing "lm_lm-frt-cp437-850#1-7_2/": failed to readDir: HTTP Error: 503 Service Unavailable

rclone copyurl https://freerainbowtables.mirror.garr.it/rainbow/RTI2/lm/lm_all-space%231-7_0/lm_all-space%231-7_0.md5sums ./file.ext -vv
2023/11/29 15:57:18 DEBUG : rclone: Version "v1.65.0" starting with parameters ["rclone" "copyurl" "https://freerainbowtables.mirror.garr.it/rainbow/RTI2/lm/lm_all-space%231-7_0/lm_all-space%231-7_0.md5sums" "./file.ext" "-vv"]
2023/11/29 15:57:18 DEBUG : Creating backend with remote "./"
2023/11/29 15:57:18 DEBUG : Using config file from "c:\\data\\rclone\\rclone.conf"
2023/11/29 15:57:18 DEBUG : fs cache: renaming cache item "./" to be canonical "//?/c:/data/rclone"
2023/11/29 15:57:19 INFO  :
Transferred:        1.816 KiB / 1.816 KiB, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         0.4s

Pascal666 · December 1, 2023, 2:22am

I installed rclone on my Windows box for testing. I got the commands provided by asdffdsa to work by adding "--tpslimit 1" to them.

I added "tpslimit = 1" to rclone.conf on the server, but that does not appear to have any effect.

How do I use tpslimit with Rclone WebUI?

ncw · December 1, 2023, 9:59am

I don't think the server likes the head requests rclone is doing - they get rate limited very quickly.

Try adding no_head = true to your config.

This server has very strict rate limiting so --tpslimit 1 is a good idea though it will work without when you set no_head.

$ rclone lsf  lm:'lm_all-space#1-7_0/'
lm_all-space#1-7_0.md5sums
lm_all-space#1-7_0_10000x51209963_distrrtgen[p][i]_19.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_00.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_01.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_02.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_03.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_04.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_05.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_06.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_07.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_08.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_09.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_10.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_11.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_12.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_13.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_14.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_15.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_16.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_17.rti2
lm_all-space#1-7_0_10000x67108864_distrrtgen[p][i]_18.rti2

$ rclone copy -Pv  lm:'lm_all-space#1-7_0/lm_all-space#1-7_0_10000x51209963_distrrtgen[p][i]_19.rti2' /tmp/
2023/12/01 09:56:59 INFO  : lm_all-space#1-7_0_10000x51209963_distrrtgen[p][i]_19.rti2: Copied (new)
2023/12/01 09:56:59 INFO  : 
Transferred:      310.008 MiB / 310.008 MiB, 100%, 5.838 MiB/s, ETA 0s
Transferred:            1 / 1, 100%
Elapsed time:        49.2s
$

asdffdsa · December 1, 2023, 2:16pm

that is a global flag, cannot go into the config file, add it as a flag on the command line
for example, rclone ls remote: --tpslimit=1

Pascal666 · December 1, 2023, 6:57pm

Nice! That does indeed fix browsing the directories, so I can now drag & drop individual files via the Explorer tab of Rclone WebUI. Unfortunately, drag & drop entire directories still does not work and I don't see a way to select multiple individual files to transfer.

I do not have access to the command line on the server, only Rclone WebUI. Is there a way to set command line options (or otherwise set tpslimit) via Rclone WebUI? That option is shown in the Backend tab (set to 0), but I can't figure out how to edit it.

asdffdsa · December 1, 2023, 7:07pm

click on the editbox and type the value.

in this example, i typed 2

Pascal666 · December 1, 2023, 7:52pm

Thank you. Yes, I tried that. I don't see any kind of "save" button, and when I switch tabs and then go back to Backend it is still set to 0. Does that actually save the new value for you?

system · December 31, 2023, 7:52pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.