Internet archive as storage (sftp)

Help and Support
1

STOP and READ USE THIS TEMPLATE NO EXCEPTIONS - By not using this, you waste your time, our time and really hate puppies. Please remove these two lines and that will confirm you have read them.

What is the problem you are having with rclone?

I want to use my following archive.org page as a regular sftp server:

i want to be able to upload, download, delete and edit files just like on a normal sftp server.

Run the command 'rclone version' and share the full output of the command.

C:\Users\johnpork>"C:\Users\johnpork\Documents\rclone.exe" version
rclone v1.71.0

  • os/version: Microsoft Windows 10 Enterprise N LTSC 2021 21H2 21H2 (64 bit)
  • os/kernel: 10.0.19044.5737 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.25.0
  • go/linking: static
  • go/tags: cmount

C:\Users\johnpork>

Which cloud storage system are you using? (eg Google Drive)

internet archive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

"C:\Users\johnpork\Documents\rclone.exe" serve sftp ets2mods:rclonemodsyow --addr :2200 --user vulcanocraft --pass password --vfs-cache-mode full --vfs-cache-max-size 200M

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

C:\Users\johnpork>"C:\Users\johnpork\Documents\rclone.exe" config redacted
[ets2mods]
type = internetarchive
access_key_id = XXX
secret_access_key = XXX

### Double check the config for sensitive info before posting publicly

C:\Users\johnpork>

A log from the command that you were trying to run with the -vv flag

C:\Users\johnpork>"C:\Users\johnpork\Documents\rclone.exe" serve sftp ets2mods:rclonemodsyow --addr :2200 --user vulcanocraft --pass password --vfs-cache-mode full --vfs-cache-max-size 200M -vv
2025/09/29 10:55:39 DEBUG : rclone: Version "v1.71.0" starting with parameters ["C:\\Users\\johnpork\\Documents\\rclone.exe" "serve" "sftp" "ets2mods:rclonemodsyow" "--addr" ":2200" "--user" "vulcanocraft" "--pass" "password" "--vfs-cache-mode" "full" "--vfs-cache-max-size" "200M" "-vv"]
2025/09/29 10:55:39 DEBUG : Creating backend with remote "ets2mods:rclonemodsyow"
2025/09/29 10:55:39 DEBUG : Using config file from "C:\\Users\\johnpork\\AppData\\Roaming\\rclone\\rclone.conf"
2025/09/29 10:55:40 INFO  : Internet Archive item rclonemodsyow: poll-interval is not supported by this remote
2025/09/29 10:55:40 DEBUG : Internet Archive item rclonemodsyow: vfs cache: root is "C:\\Users\\johnpork\\AppData\\Local\\rclone"
2025/09/29 10:55:40 DEBUG : Internet Archive item rclonemodsyow: vfs cache: data root is "\\\\?\\C:\\Users\\johnpork\\AppData\\Local\\rclone\\vfs\\ets2mods\\rclonemodsyow"
2025/09/29 10:55:40 DEBUG : Internet Archive item rclonemodsyow: vfs cache: metadata root is "\\\\?\\C:\\Users\\johnpork\\AppData\\Local\\rclone\\vfsMeta\\ets2mods\\rclonemodsyow"
2025/09/29 10:55:40 DEBUG : Creating backend with remote ":local,encoding='Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot',links=false:C:/Users/johnpork/AppData/Local/rclone/vfs/ets2mods/rclonemodsyow"
2025/09/29 10:55:40 DEBUG : :local: detected overridden config - adding "{R5KD5}" suffix to name
2025/09/29 10:55:40 DEBUG : fs cache: renaming cache item ":local,encoding='Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot',links=false:C:/Users/johnpork/AppData/Local/rclone/vfs/ets2mods/rclonemodsyow" to be canonical ":local{R5KD5}://?/C:/Users/johnpork/AppData/Local/rclone/vfs/ets2mods/rclonemodsyow"
2025/09/29 10:55:40 DEBUG : Creating backend with remote ":local,encoding='Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot',links=false:C:/Users/johnpork/AppData/Local/rclone/vfsMeta/ets2mods/rclonemodsyow"
2025/09/29 10:55:40 DEBUG : :local: detected overridden config - adding "{R5KD5}" suffix to name
2025/09/29 10:55:40 DEBUG : fs cache: renaming cache item ":local,encoding='Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Ctl,RightSpace,RightPeriod,InvalidUtf8,Dot',links=false:C:/Users/johnpork/AppData/Local/rclone/vfsMeta/ets2mods/rclonemodsyow" to be canonical ":local{R5KD5}://?/C:/Users/johnpork/AppData/Local/rclone/vfsMeta/ets2mods/rclonemodsyow"
2025/09/29 10:55:40 DEBUG : vulcanomods_1.55.zip: vfs cache: truncate to size=8975558246 (not needed as size correct)
2025/09/29 10:55:40 DEBUG : vulcanomods_1.55.zip: vfs cache: setting modification time to 2025-09-28 16:32:02.4385577 +0200 CEST
2025/09/29 10:55:40 INFO  : vulcanomods_1.55.zip: vfs cache: queuing for upload in 5s
2025/09/29 10:55:40 DEBUG : Added virtual directory entry vAddFile: "vulcanomods_1.55.zip"
2025/09/29 10:55:40 DEBUG : Internet Archive item rclonemodsyow: vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item vulcanomods_1.55.zip not removed, freed 0 bytes
2025/09/29 10:55:40 NOTICE: Loaded 0 authorized keys from "C:\\Users\\johnpork\\.ssh\\authorized_keys"
2025/09/29 10:55:40 INFO  : Internet Archive item rclonemodsyow: vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 1, uploading 0, total size 8.359Gi (was 8.359Gi)
2025/09/29 10:55:40 DEBUG : Loaded private key from "C:\\Users\\johnpork\\AppData\\Local\\rclone\\serve-sftp\\id_rsa"
2025/09/29 10:55:40 DEBUG : Loaded private key from "C:\\Users\\johnpork\\AppData\\Local\\rclone\\serve-sftp\\id_ecdsa"
2025/09/29 10:55:40 DEBUG : Loaded private key from "C:\\Users\\johnpork\\AppData\\Local\\rclone\\serve-sftp\\id_ed25519"
2025/09/29 10:55:40 NOTICE: SFTP server listening on [::]:2200
2025/09/29 10:55:45 DEBUG : vulcanomods_1.55.zip: vfs cache: starting upload
2025/09/29 10:55:45 DEBUG : vulcanomods_1.55.zip: Setting IA item derive: true
2025/09/29 10:56:18 DEBUG : serve sftp 127.130.175.201:50724->127.0.0.1:2200: ssh auth "none" from "SSH-2.0-rclone/v1.71.0": ssh: no auth passed yet
2025/09/29 10:56:18 DEBUG : serve sftp 127.130.175.201:50724->127.0.0.1:2200: Password login attempt for vulcanocraft
2025/09/29 10:56:18 DEBUG : serve sftp 127.130.175.201:50724->127.0.0.1:2200: ssh auth "password" from "SSH-2.0-rclone/v1.71.0": password rejected for "vulcanocraft"
2025/09/29 10:56:18 ERROR : serve sftp 127.130.175.201:50724->127.0.0.1:2200: SSH login failed: [ssh: no auth passed yet, password rejected for "vulcanocraft"]
2025/09/29 10:56:20 DEBUG : serve sftp 127.130.175.201:50730->127.0.0.1:2200: ssh auth "none" from "SSH-2.0-rclone/v1.71.0": ssh: no auth passed yet
2025/09/29 10:56:20 DEBUG : serve sftp 127.130.175.201:50730->127.0.0.1:2200: Password login attempt for vulcanocraft
2025/09/29 10:56:20 DEBUG : serve sftp 127.130.175.201:50730->127.0.0.1:2200: ssh auth "password" from "SSH-2.0-rclone/v1.71.0": password rejected for "vulcanocraft"
2025/09/29 10:56:20 ERROR : serve sftp 127.130.175.201:50730->127.0.0.1:2200: SSH login failed: [ssh: no auth passed yet, password rejected for "vulcanocraft"]
2025/09/29 10:56:22 DEBUG : serve sftp 127.130.175.201:50736->127.0.0.1:2200: ssh auth "none" from "SSH-2.0-rclone/v1.71.0": ssh: no auth passed yet
2025/09/29 10:56:22 DEBUG : serve sftp 127.130.175.201:50736->127.0.0.1:2200: Password login attempt for vulcanocraft
2025/09/29 10:56:22 DEBUG : serve sftp 127.130.175.201:50736->127.0.0.1:2200: ssh auth "password" from "SSH-2.0-rclone/v1.71.0": password rejected for "vulcanocraft"
2025/09/29 10:56:22 ERROR : serve sftp 127.130.175.201:50736->127.0.0.1:2200: SSH login failed: [ssh: no auth passed yet, password rejected for "vulcanocraft"]
2025/09/29 10:56:24 DEBUG : serve sftp 127.130.175.201:50740->127.0.0.1:2200: ssh auth "none" from "SSH-2.0-rclone/v1.71.0": ssh: no auth passed yet
2025/09/29 10:56:24 DEBUG : serve sftp 127.130.175.201:50740->127.0.0.1:2200: Password login attempt for vulcanocraft
2025/09/29 10:56:24 DEBUG : serve sftp 127.130.175.201:50740->127.0.0.1:2200: ssh auth "password" from "SSH-2.0-rclone/v1.71.0": password rejected for "vulcanocraft"
2025/09/29 10:56:24 ERROR : serve sftp 127.130.175.201:50740->127.0.0.1:2200: SSH login failed: [ssh: no auth passed yet, password rejected for "vulcanocraft"]
2025/09/29 10:56:26 DEBUG : serve sftp 127.130.175.201:50748->127.0.0.1:2200: ssh auth "none" from "SSH-2.0-rclone/v1.71.0": ssh: no auth passed yet
2025/09/29 10:56:26 DEBUG : serve sftp 127.130.175.201:50748->127.0.0.1:2200: Password login attempt for vulcanocraft
2025/09/29 10:56:26 DEBUG : serve sftp 127.130.175.201:50748->127.0.0.1:2200: ssh auth "password" from "SSH-2.0-rclone/v1.71.0": password rejected for "vulcanocraft"
2025/09/29 10:56:26 ERROR : serve sftp 127.130.175.201:50748->127.0.0.1:2200: SSH login failed: [ssh: no auth passed yet, password rejected for "vulcanocraft"]
2025/09/29 10:56:28 DEBUG : serve sftp 127.130.175.201:50754->127.0.0.1:2200: ssh auth "none" from "SSH-2.0-rclone/v1.71.0": ssh: no auth passed yet
2025/09/29 10:56:28 DEBUG : serve sftp 127.130.175.201:50754->127.0.0.1:2200: Password login attempt for vulcanocraft
2025/09/29 10:56:28 DEBUG : serve sftp 127.130.175.201:50754->127.0.0.1:2200: ssh auth "password" from "SSH-2.0-rclone/v1.71.0": password rejected for "vulcanocraft"
2025/09/29 10:56:28 ERROR : serve sftp 127.130.175.201:50754->127.0.0.1:2200: SSH login failed: [ssh: no auth passed yet, password rejected for "vulcanocraft"]
2025/09/29 10:56:30 DEBUG : serve sftp 127.130.175.201:50760->127.0.0.1:2200: ssh auth "none" from "SSH-2.0-rclone/v1.71.0": ssh: no auth passed yet
2025/09/29 10:56:30 DEBUG : serve sftp 127.130.175.201:50760->127.0.0.1:2200: Password login attempt for vulcanocraft
2025/09/29 10:56:30 DEBUG : serve sftp 127.130.175.201:50760->127.0.0.1:2200: ssh auth "password" from "SSH-2.0-rclone/v1.71.0": password rejected for "vulcanocraft"
2025/09/29 10:56:30 ERROR : serve sftp 127.130.175.201:50760->127.0.0.1:2200: SSH login failed: [ssh: no auth passed yet, password rejected for "vulcanocraft"]
2025/09/29 10:56:32 DEBUG : serve sftp 127.130.175.201:50766->127.0.0.1:2200: ssh auth "none" from "SSH-2.0-rclone/v1.71.0": ssh: no auth passed yet
2025/09/29 10:56:32 DEBUG : serve sftp 127.130.175.201:50766->127.0.0.1:2200: Password login attempt for vulcanocraft
2025/09/29 10:56:32 DEBUG : serve sftp 127.130.175.201:50766->127.0.0.1:2200: ssh auth "password" from "SSH-2.0-rclone/v1.71.0": password rejected for "vulcanocraft"
2025/09/29 10:56:32 ERROR : serve sftp 127.130.175.201:50766->127.0.0.1:2200: SSH login failed: [ssh: no auth passed yet, password rejected for "vulcanocraft"]
2025/09/29 10:56:34 DEBUG : serve sftp 127.130.175.201:50773->127.0.0.1:2200: ssh auth "none" from "SSH-2.0-rclone/v1.71.0": ssh: no auth passed yet
2025/09/29 10:56:34 DEBUG : serve sftp 127.130.175.201:50773->127.0.0.1:2200: Password login attempt for vulcanocraft
2025/09/29 10:56:34 DEBUG : serve sftp 127.130.175.201:50773->127.0.0.1:2200: ssh auth "password" from "SSH-2.0-rclone/v1.71.0": password rejected for "vulcanocraft"
2025/09/29 10:56:34 ERROR : serve sftp 127.130.175.201:50773->127.0.0.1:2200: SSH login failed: [ssh: no auth passed yet, password rejected for "vulcanocraft"]
2025/09/29 10:56:36 DEBUG : serve sftp 127.130.175.201:50779->127.0.0.1:2200: ssh auth "none" from "SSH-2.0-rclone/v1.71.0": ssh: no auth passed yet
2025/09/29 10:56:36 DEBUG : serve sftp 127.130.175.201:50779->127.0.0.1:2200: Password login attempt for vulcanocraft
2025/09/29 10:56:36 DEBUG : serve sftp 127.130.175.201:50779->127.0.0.1:2200: ssh auth "password" from "SSH-2.0-rclone/v1.71.0": password rejected for "vulcanocraft"
2025/09/29 10:56:36 ERROR : serve sftp 127.130.175.201:50779->127.0.0.1:2200: SSH login failed: [ssh: no auth passed yet, password rejected for "vulcanocraft"]
2025/09/29 10:56:40 INFO  :
Transferred:        9.965 MiB / 8.359 GiB, 0%, 131.591 KiB/s, ETA 18h28m51s
Checks:                 0 / 0, -, Listed 2
Transferred:            0 / 1, 0%
Elapsed time:        59.7s
Transferring:
 *                          vulcanomods_1.55.zip:  0% /8.359Gi, 138.364Ki/s, 17h34m34s

2025/09/29 10:56:40 DEBUG : Internet Archive item rclonemodsyow: vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item vulcanomods_1.55.zip not removed, freed 0 bytes
2025/09/29 10:56:40 INFO  : Internet Archive item rclonemodsyow: vfs cache: cleaned: objects 1 (was 1) in use 1, to upload 0, uploading 1, total size 8.359Gi (was 8.359Gi)
2025/09/29 10:56:40 DEBUG : serve sftp 127.130.175.201:50794->127.0.0.1:2200: ssh auth "none" from "SSH-2.0-FileZilla_3.69.3": ssh: no auth passed yet
2025/09/29 10:56:40 DEBUG : serve sftp 127.130.175.201:50794->127.0.0.1:2200: Password login attempt for vulcanocraft
2025/09/29 10:56:40 DEBUG : serve sftp 127.130.175.201:50794->127.0.0.1:2200: ssh auth "password" from "SSH-2.0-FileZilla_3.69.3": OK
2025/09/29 10:56:40 INFO  : serve sftp 127.130.175.201:50794->127.0.0.1:2200: SSH login from vulcanocraft using SSH-2.0-FileZilla_3.69.3
2025/09/29 10:56:40 DEBUG : serve sftp 127.130.175.201:50794->127.0.0.1:2200: Incoming channel: session
2025/09/29 10:56:40 DEBUG : serve sftp 127.130.175.201:50794->127.0.0.1:2200: Channel accepted
2025/09/29 10:56:40 DEBUG : serve sftp 127.130.175.201:50794->127.0.0.1:2200: Request: simple@putty.projects.tartarus.org
2025/09/29 10:56:40 DEBUG : serve sftp 127.130.175.201:50794->127.0.0.1:2200:  - accepted: false
2025/09/29 10:56:40 DEBUG : serve sftp 127.130.175.201:50794->127.0.0.1:2200: Request: subsystem
2025/09/29 10:56:40 DEBUG : serve sftp 127.130.175.201:50794->127.0.0.1:2200: Subsystem: sftp
2025/09/29 10:56:40 DEBUG : serve sftp 127.130.175.201:50794->127.0.0.1:2200:  - accepted: true
2025/09/29 10:56:40 DEBUG : serve sftp 127.130.175.201:50794->127.0.0.1:2200: Starting SFTP server
2025/09/29 10:56:41 DEBUG : rclonemodsyow_archive.torrent: Set virtual modtime to 2025-09-28 15:49:40 +0200 CEST
2025/09/29 10:56:41 DEBUG : rclonemodsyow_files.xml: Set virtual modtime to 1970-01-01 01:00:00 +0100 CET
2025/09/29 10:56:41 DEBUG : rclonemodsyow_meta.sqlite: Set virtual modtime to 2025-09-28 15:49:39 +0200 CEST
2025/09/29 10:56:41 DEBUG : rclonemodsyow_meta.xml: Set virtual modtime to 2025-09-28 15:43:14 +0200 CEST
2025/09/29 10:56:41 DEBUG : vulcanomods_1.55.zip: Set virtual modtime to 2025-09-28 16:32:02.4385577 +0200 CEST
2025/09/29 10:56:41 DEBUG : rclonemodsyow_archive.torrent: Returning virtual modtime 2025-09-28 15:49:40 +0200 CEST
2025/09/29 10:56:41 DEBUG : rclonemodsyow_files.xml: Returning virtual modtime 1970-01-01 01:00:00 +0100 CET
2025/09/29 10:56:41 DEBUG : rclonemodsyow_meta.sqlite: Returning virtual modtime 2025-09-28 15:49:39 +0200 CEST
2025/09/29 10:56:41 DEBUG : rclonemodsyow_meta.xml: Returning virtual modtime 2025-09-28 15:43:14 +0200 CEST

when i open the sftp server in filezilla, i can just see the file i uploaded:

image
image1918×1078 283 KB

but on archive.org i just don’t see my file appear:

image
image1918×1078 75.7 KB

2

You are serving your sftp on port 2200 .

Your picture of FileZilla shows you connecting to port 23879.

Are you sure you are not mixing something here?

3

i use a tunnel for port forwarding so that’s why

image
image1093×619 49.8 KB

4

bump @kapitainsky

5

do simple command work, such as rclone copy and rclone delete ?

6

yes, it works, the changes are viewable at the sftp server itself. but the changes aren’t comming to internet archive

7

sftp changes are visible immediately as files reside in cache (--vfs-cache-mode full). It takes by default 5 seconds to start their upload to underlying remote + whatever it takes to transfer all data. See docs for VFS cache details.

Try with some small file first. It should be much faster.

Your log shows:

So you would have to wait 19 hours with this speed before this file is uploaded (and visible) in internet archive.

8

wait, is the upload speed so slow? my internet isn’t that slow.

9

try without sftp. Maybe internet archive is slow for large files regardless. You have to try to identify where the bottleneck is.

10

i guess it’s internet archive, their web uploader is very slow, uploading via their python api is slightly faster but still slow

11

I’ve heard that it is only painfully slow outside of US. In case you are not in the ol' US of A try VPN terminating you there.

12

ok, great tip. yeah i am from belgium so maybe that’s the issue. can you maybe recommend me a free USA VPN?

13

Only “free“ and working reliably is a friend in US who will setup VPN server for you. Otherwise I have never heard about free good solutions. Unless you want to spend time every few days looking for replacement. Free VPNs are truly ephemeral thing:)

As a proof of concept you can try some commercial VPN first. Most of them allow you to cancel for some limited time period.

14

i use mullvad vpn. open source, $5.00/month, very powerful, especially when using firefox + multi-account containers extension.
and of course, mullvad browser.

1 Like
15

And their servers can be used as tailscale exit nodes. This is real cool. Have to play with it a bit more. Maybe time to change VPN provider. I’ve been using ExpressVPN, with 2 years contract it was $4.5 per month.

16

yeah, i was going to mention that but i have not tried it yet.

with the multi-account containers, an official extension from mozilla:

  • when connecting using wireguard, have access to many dozens of proxy server.
    this allows per website to have private cookie container and a custom proxy server.