Input/output error reading .ZIP file mounted via rclone from Google Drive

Dave92f1 · January 2, 2024, 10:30pm

What is the problem you are having with rclone?

I'm unable to read some .ZIP files from Google Drive, having mounted the drive via rclone. I get "Input/output error".

I can access other files fine (read and write). Just a few .ZIP files have the problem.

Example:

dave@MONOLITH:~/Desktop/Linux_setup (2023)/todo/Mini keyboard/Mini Keyboard Sikai$ ls -al Macropad.zip 
-rw-rw-r-- 1 dave dave 28062343 Dec 29 20:07 Macropad.zip
dave@MONOLITH:~/Desktop/Linux_setup (2023)/todo/Mini keyboard/Mini Keyboard Sikai$ cat Macropad.zip 
cat: Macropad.zip: Input/output error

I'm able to read the files without problem via the Google Drive web interface. When I do it that way, GDrive complains that the .ZIP file is "too big to scan" (for viruses) and I have to click "Download anyway" in the web GUI.

I suspect this is the source of the problem - rclone somehow isn't telling GDrive to "download anyway".

Run the command 'rclone version' and share the full output of the command.

rclone v1.65.0
- os/version: debian bookworm/sid (64 bit)
- os/kernel: 6.2.0-37-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.4
- go/linking: static
- go/tags: none

FWIW, I'm on KDE Neon (downstream from Ubuntu 22.04 LTS):

Operating System: KDE neon 5.27
KDE Plasma Version: 5.27.10
KDE Frameworks Version: 5.113.0
Qt Version: 5.15.11
Kernel Version: 6.2.0-37-generic (64-bit)
Graphics Platform: X11
Processors: 32 × 13th Gen Intel® Core™ i9-13900K
Memory: 62.6 GiB of RAM
Graphics Processor: NVIDIA GeForce RTX 2060 SUPER/PCIe/SSE2
Manufacturer: Micro-Star International Co., Ltd.
Product Name: MS-7D30
System Version: 1.0

Which cloud storage system are you using? (eg Google Drive)

Google Drive.

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

cat Macropad.zip

cp also fails. I can't even right-click the file in the Dolphin file browser - it locks up when I try (I suspect something is waiting on Google).

GDrive is mounted at /home/dave/mnt/HookeMyDrive

Full path to file is:

"/home/dave/mnt/HookeMyDrive/dave_data/System Files & Notes/Linux/Linux_setup (2023)/todo/Mini keyboard/Mini Keyboard Sikai/Macropad.zip"

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

dave@MONOLITH:~$ rclone config redacted
[By Model - All quotes]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[By Model - Results]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[By Model - Studies]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[Customers]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[Employee Handbook and Policies and Phone numbers]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[Equipment manuals and info]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[HR incl. consultants CONFIDENTIAL]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[HookeMyDrive]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = 

[IACUC]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[IACUC protocols]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[IT]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[IT Confidential]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[Kit Lab]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[LabData]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[Legal]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[Model background and sample data]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[Office]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[Products]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[SOPs]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[Safety Data Sheets]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[Templates]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[Web development]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[Web page]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = XXX
root_folder_id = 

[gmailMyDrive]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = 
### Double check the config for sensitive info before posting publicly

A log from the command that you were trying to run with the `-vv` flag

dave@MONOLITH:~/mnt/HookeMyDrive/dave_data/System Files & Notes/Linux/Linux_setup (2023)/todo/Mini keyboard/Mini Keyboard Sikai$ cat -vv Macropad.zip
cat: Macropad.zip: Input/output error

Here's the mount command I've been using:

rclone mount "${remote_name}": "${mount_point}" --daemon \
        --vfs-cache-mode full \
        --vfs-read-ahead 128M \
        --buffer-size 64M \
        --drive-chunk-size 128M \
        --cache-dir "/home/dave/.cache/rclone/${remote_name}"

Here's the log output (see last line in particular):

2024/01/02 23:02:11 DEBUG : &{dave_data/System Files & Notes/Linux/Linux_setup (2023)/todo/Mini keyboard/Mini Keyboard Sikai/https_⁄⁄www.energyfort.cc⁄blog⁄mini-keyboard-firmware-instruction-and-download.desktop (rw)}: >Release: err=<nil>
2024/01/02 23:02:11 DEBUG : dave_data/System Files & Notes/Linux/Linux_setup (2023)/todo/Mini keyboard/Mini Keyboard Sikai/: Lookup: name=".bzr"
2024/01/02 23:02:11 DEBUG : dave_data/System Files & Notes/Linux/Linux_setup (2023)/todo/Mini keyboard/Mini Keyboard Sikai/: >Lookup: node=<nil>, err=no such file or directory
2024/01/02 23:02:13 DEBUG : /: Lookup: name="dave_data"
2024/01/02 23:02:13 DEBUG : /: >Lookup: node=dave_data/, err=<nil>
2024/01/02 23:02:13 DEBUG : dave_data/: Attr: 
2024/01/02 23:02:13 DEBUG : dave_data/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2024/01/02 23:02:13 DEBUG : dave_data/: Lookup: name="System Files & Notes"
2024/01/02 23:02:13 DEBUG : dave_data/: >Lookup: node=dave_data/System Files & Notes/, err=<nil>
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/: Attr: 
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/: Lookup: name="Linux"
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/: >Lookup: node=dave_data/System Files & Notes/Linux/, err=<nil>
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/Linux/: Attr: 
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/Linux/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/Linux/: Lookup: name="Linux_setup (2023)"
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/Linux/: >Lookup: node=dave_data/System Files & Notes/Linux/Linux_setup (2023)/, err=<nil>
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/Linux/Linux_setup (2023)/: Attr: 
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/Linux/Linux_setup (2023)/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2024/01/02 23:02:15 INFO  : dave_data/System Files & Notes/Linux/Linux_setup (2023)/todo/Mini keyboard/Mini Keyboard Sikai/Macropad.zip: vfs cache: downloader: error count now 3: vfs reader: failed to write to cache file: open file failed: use the --drive-acknowledge-abuse flag to download this file: googleapi: Error 403: This file has been identified as malware or spam and cannot be downloaded., cannotDownloadAbusiveFile
2024/01/02 23:02:15 ERROR : dave_data/System Files & Notes/Linux/Linux_setup (2023)/todo/Mini keyboard/Mini Keyboard Sikai/Macropad.zip: vfs cache: failed to download: vfs reader: failed to write to cache file: open file failed: use the --drive-acknowledge-abuse flag to download this file: googleapi: Error 403: This file has been identified as malware or spam and cannot be downloaded., cannotDownloadAbusiveFile

asdffdsa · January 2, 2024, 10:49pm

hi,
the only things missing are
--- the exact mount command
--- the rclone debug log, that shows the error.

given that rclone is reading/downloading, not writing/uploading, the default --vfs-cache-mode=off should work.
sometimes, might need to use a cache mode such as --vfs-cache-mode=full

Dave92f1 · January 3, 2024, 3:39am

Thanks for the reply. Here's the mount command I've been using:

rclone mount "${remote_name}": "${mount_point}" --daemon \
        --vfs-cache-mode full \
        --vfs-read-ahead 128M \
        --buffer-size 64M \
        --drive-chunk-size 128M \
        --cache-dir "/home/dave/.cache/rclone/${remote_name}"

I just added this to the end of the mount command:

        --cache-dir "/home/dave/.cache/rclone/${remote_name} \
        --vv --log-file /home/dave/Dettkop/rclone.log"

Here's the log output (see last line in particular):


2024/01/02 23:02:11 DEBUG : &{dave_data/System Files & Notes/Linux/Linux_setup (2023)/todo/Mini keyboard/Mini Keyboard Sikai/https_⁄⁄www.energyfort.cc⁄blog⁄mini-keyboard-firmware-instruction-and-download.desktop (rw)}: >Release: err=<nil>
2024/01/02 23:02:11 DEBUG : dave_data/System Files & Notes/Linux/Linux_setup (2023)/todo/Mini keyboard/Mini Keyboard Sikai/: Lookup: name=".bzr"
2024/01/02 23:02:11 DEBUG : dave_data/System Files & Notes/Linux/Linux_setup (2023)/todo/Mini keyboard/Mini Keyboard Sikai/: >Lookup: node=<nil>, err=no such file or directory
2024/01/02 23:02:13 DEBUG : /: Lookup: name="dave_data"
2024/01/02 23:02:13 DEBUG : /: >Lookup: node=dave_data/, err=<nil>
2024/01/02 23:02:13 DEBUG : dave_data/: Attr: 
2024/01/02 23:02:13 DEBUG : dave_data/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2024/01/02 23:02:13 DEBUG : dave_data/: Lookup: name="System Files & Notes"
2024/01/02 23:02:13 DEBUG : dave_data/: >Lookup: node=dave_data/System Files & Notes/, err=<nil>
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/: Attr: 
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/: Lookup: name="Linux"
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/: >Lookup: node=dave_data/System Files & Notes/Linux/, err=<nil>
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/Linux/: Attr: 
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/Linux/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/Linux/: Lookup: name="Linux_setup (2023)"
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/Linux/: >Lookup: node=dave_data/System Files & Notes/Linux/Linux_setup (2023)/, err=<nil>
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/Linux/Linux_setup (2023)/: Attr: 
2024/01/02 23:02:13 DEBUG : dave_data/System Files & Notes/Linux/Linux_setup (2023)/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2024/01/02 23:02:15 INFO  : dave_data/System Files & Notes/Linux/Linux_setup (2023)/todo/Mini keyboard/Mini Keyboard Sikai/Macropad.zip: vfs cache: downloader: error count now 3: vfs reader: failed to write to cache file: open file failed: use the --drive-acknowledge-abuse flag to download this file: googleapi: Error 403: This file has been identified as malware or spam and cannot be downloaded., cannotDownloadAbusiveFile
2024/01/02 23:02:15 ERROR : dave_data/System Files & Notes/Linux/Linux_setup (2023)/todo/Mini keyboard/Mini Keyboard Sikai/Macropad.zip: vfs cache: failed to download: vfs reader: failed to write to cache file: open file failed: use the --drive-acknowledge-abuse flag to download this file: googleapi: Error 403: This file has been identified as malware or spam and cannot be downloaded., cannotDownloadAbusiveFile

kapitainsky · January 3, 2024, 7:34am

Based on this error we have to confirm to Google than we do not mind downloading potentially unsafe files (which is in line with your observation when using web GUI).

Add --drive-acknowledge-abuse flag to your mount command:

rclone mount "${remote_name}": "${mount_point}" --daemon \
        --vfs-cache-mode full \
        --vfs-read-ahead 128M \
        --buffer-size 64M \
        --drive-chunk-size 128M \
        --cache-dir "/home/dave/.cache/rclone/${remote_name}" \
        --drive-acknowledge-abuse

Dave92f1 · January 3, 2024, 9:22pm

That fixed it, thank you!

system · January 6, 2024, 9:22pm

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.