Hello,
I'm trying to download a archive.org repository, but since its slow and I don't want to overload its servers, I'm trying to transfer one file at a time.
I believe rclone is not correctly parsing the arguments for the following reasons:
1 - I have transfers=1 and multi-thread-streams=0, but it will still start downloading more than 1 file
2 - I have local-no-sparse but it still uses sparse files
3 - I have multi-thread-streams disabled but it still uses multi-threads
4 - Also I'm using max-depth=1 but it still tries to download one additional level (archive.org has a link for each file which lists its contents, but it effectively is one additional level - see error "error reading source directory: error listing "Gradius II - X68k FM+MT-32 OST (Hack) v1 protogem.7z/": directory not found" below)
Is it because of these combinations of arguments? Or is it because of the :http: argument?
Thanks,
Mike
Operating system: Windows 10
rclone version v1.52.0-032-g7458d37d-beta (but this also happens with the non-beta)
C:\rclone>rclone -vvv sync --http-url "https://archive.org/download/everdrivepack/Project%20Peacock/" :http: "X:\mirrors\Everdrive\Project Peacock\" --checkers 1 --transfers=1 --progress --delete-during --max-depth 1 --local-no-sparse --multi-thread-streams 0 --low-level-retries 32 --retries 10
2020/06/09 09:45:55 DEBUG : rclone: Version "v1.52.0-032-g7458d37d-beta" starting with parameters ["rclone" "-vvv" "sync" "--http-url" "https://archive.org/download/everdrivepack/Project%20Peacock/" ":http:" "X:\\mirrors\\Everdrive\\Project Peacock\" --checkers 1 --transfers=1 --progress --delete-during --max-depth 1 --local-no-sparse --multi-thread-streams 0 --low-level-retries 32 --retries 10 "]
2020/06/09 09:45:55 DEBUG : Using config file from "C:\\Users\\Mike\\.config\\rclone\\rclone.conf"
2020/06/09 09:45:55 DEBUG : fs cache: renaming cache item "X:\\mirrors\\Everdrive\\Project Peacock\" --checkers 1 --transfers=1 --progress --delete-during --max-depth 1 --local-no-sparse --multi-thread-streams 0 --low-level-retries 32 --retries 10 " to be canonical "//?/X:/mirrors/Everdrive/Project Peacock\" --checkers 1 --transfers=1 --progress --delete-during --max-depth 1 --local-no-sparse --multi-thread-streams 0 --low-level-retries 32 --retries 10 "
2020/06/09 09:46:55 INFO :
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Elapsed time: 0.0s
2020/06/09 09:47:17 DEBUG : Ai Chou Aniki (Japan) (SADS).7z: Sizes differ (src 485161861 vs dst 243580494)
2020/06/09 09:47:17 DEBUG : Avenger (Japan).7z: Sizes differ (src 265694666 vs dst 133774927)
2020/06/09 09:47:17 DEBUG : CD Denjin - Rockabilly Tengoku (Japan).7z: Sizes differ (src 448394290 vs dst 224771662)
2020/06/09 09:47:17 DEBUG : Chou Aniki (Japan) (FAAT, FACT).7z: Sizes differ (src 228641267 vs dst 126976)
2020/06/09 09:47:17 INFO : Writing sparse files: use --local-no-sparse or --multi-thread-streams 0 to disable
2020/06/09 09:47:17 DEBUG : Daisenpuu Custom (Japan).7z: Starting multi-thread copy with 2 parts of size 169.500M
2020/06/09 09:47:17 DEBUG : Daisenpuu Custom (Japan).7z: multi-thread copy: stream 2/2 (177733632-355349463) size 169.388M starting
2020/06/09 09:47:17 DEBUG : Chou Jikuu Yousai Macross 2036 (Japan).7z: Starting multi-thread copy with 2 parts of size 224.750M
2020/06/09 09:47:17 DEBUG : Chou Jikuu Yousai Macross 2036 (Japan).7z: multi-thread copy: stream 2/2 (235667456-471322533) size 224.738M starting
2020/06/09 09:47:17 DEBUG : Chou Jikuu Yousai Macross 2036 (Japan).7z: multi-thread copy: stream 1/2 (0-235667456) size 224.750M starting
2020/06/09 09:47:17 DEBUG : Daisenpuu Custom (Japan).7z: multi-thread copy: stream 1/2 (0-177733632) size 169.500M starting
2020/06/09 09:47:33 ERROR : Gradius II - X68k FM+MT-32 OST (Hack) v1 protogem.7z: error reading source directory: error listing "Gradius II - X68k FM+MT-32 OST (Hack) v1 protogem.7z/": directory not found
2020/06/09 09:47:55 INFO :
Transferred: 13.116M / 28.062 GBytes, 0%, 351.872 kBytes/s, ETA 23h13m7s
Errors: 1 (retrying may help)
Checks: 4 / 4, 100%
Transferred: 0 / 85, 0%
Elapsed time: 38.1s
Transferring:
* Down Load 2 (Japan).7z: 1% /225.871M, 110.465k/s, 34m26s
* Daisenpuu Custom (Japan).7z: 1% /338.888M, 173.437k/s, 32m51s
* Chou Jikuu Yousai Macross 2036 (Japan).7z: 0% /449.488M, 135.523k/s, 56m4s
* Cycho Rider (from Seiy…th Fantasy Stories).7z: 2% /39.806M, 32.343k/s, 20m28s
2020/06/09 09:47:57 ERROR : @Shooter Elements (CD)/Xak III - The Eternal Recurrence - English (Japan) T+Eng v1 X3 De-Japanization 2004.7z: error reading source directory: error listing "@Shooter Elements (CD)/Xak III - The Eternal Recurrence - English (Japan) T+Eng v1 X3 De-Japanization 2004.7z/": directory not found
2020/06/09 09:48:08 DEBUG : Local file system at //?/X:/mirrors/Everdrive/Project Peacock" --checkers 1 --transfers=1 --progress --delete-during --max-depth 1 --local-no-sparse --multi-thread-streams 0 --low-level-retries 32 --retries 10 : Waiting for checks to finish
2020/06/09 09:48:08 DEBUG : Local file system at //?/X:/mirrors/Everdrive/Project Peacock" --checkers 1 --transfers=1 --progress --delete-during --max-depth 1 --local-no-sparse --multi-thread-streams 0 --low-level-retries 32 --retries 10 : Waiting for transfers to finish
2020/06/09 09:48:55 INFO :
Transferred: 34.178M / 28.062 GBytes, 0%, 356.510 kBytes/s, ETA 22h53m59s
Errors: 2 (retrying may help)
Checks: 4 / 4, 100%
Transferred: 0 / 85, 0%
Elapsed time: 1m38.1s
Transferring:
* Down Load 2 (Japan).7z: 3% /225.871M, 76.293k/s, 48m44s
* Daisenpuu Custom (Japan).7z: 4% /338.888M, 148.474k/s, 37m16s
* Chou Jikuu Yousai Macross 2036 (Japan).7z: 1% /449.488M, 93.808k/s, 1h20m12s
* Cycho Rider (from Seiy…th Fantasy Stories).7z: 7% /39.806M, 26.579k/s, 23m38s
2020/06/09 09:49:55 INFO :
Transferred: 62.647M / 28.062 GBytes, 0%, 405.569 kBytes/s, ETA 20h6m35s
Errors: 2 (retrying may help)
Checks: 4 / 4, 100%
Transferred: 0 / 85, 0%
Elapsed time: 2m38.1s
Transferring:
* Down Load 2 (Japan).7z: 6% /225.871M, 121.659k/s, 29m43s
* Daisenpuu Custom (Japan).7z: 7% /338.888M, 128.984k/s, 41m29s
* Chou Jikuu Yousai Macross 2036 (Japan).7z: 3% /449.488M, 91.364k/s, 1h21m16s
* Cycho Rider (from Seiy…th Fantasy Stories).7z: 22% /39.806M, 85.730k/s, 6m8s
2020/06/09 09:50:55 INFO :
Transferred: 101.069M / 28.062 GBytes, 0%, 474.378 kBytes/s, ETA 17h10m11s
Errors: 2 (retrying may help)
Checks: 4 / 4, 100%
Transferred: 0 / 85, 0%
Elapsed time: 3m38.1s
Transferring:
* Down Load 2 (Japan).7z: 8% /225.871M, 119.659k/s, 29m21s
* Daisenpuu Custom (Japan).7z: 11% /338.888M, 207.949k/s, 24m35s
* Chou Jikuu Yousai Macross 2036 (Japan).7z: 6% /449.488M, 214.881k/s, 33m29s
* Cycho Rider (from Seiy…th Fantasy Stories).7z: 35% /39.806M, 128.802k/s, 3m25s
2020/06/09 09:51:55 INFO :
Transferred: 122.928M / 28.062 GBytes, 0%, 452.525 kBytes/s, ETA 17h59m6s
Errors: 2 (retrying may help)
Checks: 4 / 4, 100%
Transferred: 0 / 85, 0%
Elapsed time: 4m38.1s
Transferring:
* Down Load 2 (Japan).7z: 9% /225.871M, 14.431k/s, 4h1m6s
* Daisenpuu Custom (Japan).7z: 14% /338.888M, 94.190k/s, 52m26s
* Chou Jikuu Yousai Macross 2036 (Japan).7z: 7% /449.488M, 125.852k/s, 56m16s
* Cycho Rider (from Seiy…th Fantasy Stories).7z: 42% /39.806M, 44.656k/s, 8m43s