What is the problem you are having with rclone?
When I attempt to list all the files in one of my Google Drives (haven't tested others) a decent amount of the files end up missing, ~5%. The files that are not returned seems consistent. This happens with and without fast-list option.
My drive is a bunch of files stored by their MD5s. A file with MD5 'ffff124484cec49abc87e355d8de14c7' would be found at drive_name:MD5/ff/ff/ffff124484cec49abc87e355d8de14c7
There should be 962 files returned from the MD5/ff folder, but it only lists 955. The missing files are visible in Google Drive web interface, Google Drive Filestream, and using the Google Python Library. I added '--dump responses' to the command and only see '200 OK' responses. If I add the second level of folders to the request (:MD5/ff/ff vs :MD5/ff) missing files appear.
What is your rclone version (output from
rclone v1.52.3 - os/arch: linux/amd64 - go version: go1.13.15
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Linux (RHEL 8) 64 bit
Which cloud storage system are you using? (eg Google Drive)
The command you were trying to run (eg
rclone copy /tmp remote:tmp)
rclone ls gd_backup_zdata_home:MD5/ff
The rclone config contents with secrets removed.
[gd_backup_zdata_home] type = drive client_id = 1...2.apps.googleusercontent.com client_secret = n...V scope = drive root_folder_id = 0...A service_account_file = /opt/.../zfsbackup.json
A log from the command with the
2020/09/01 16:30:25 DEBUG : rclone: Version "v1.52.3" starting with parameters ["/opt/.../envs/zfs_backup/bin/rclone" "ls" "--log-file" "log.txt" "-vv" "gd_backup_zdata_home:MD5/ff"] 2020/09/01 16:30:25 DEBUG : Using config file from "/opt/.../etc/zfs_backup/rclone.conf" 2020/09/01 16:30:32 DEBUG : 21 go routines active
Additional example output
rclone ls gd_backup_zdata_home:MD5/ff/ff | grep 'ffff' 1608 ffff124484cec49abc87e355d8de14c7 321 ffffde984a43be4653c91a803a0e9be5 rclone ls gd_backup_zdata_home:MD5/ff | grep '\/ffff' 321 ff/ffffde984a43be4653c91a803a0e9be5
Thanks and let me know if you need additional details of for me to testing anything.