What is the problem you are having with rclone?
Rclone is being used to present an Openstack Swift object store as a file system for CIFS use. Right now CIFS isn't the problem since Rclone (after working for years) has stopped showing entire directories' contents. Some of these directories have 100,000+ entries in them which wasn't a problem before.
What is your rclone version (output from rclone version)
rclone v1.55.1
- os/type: linux
- os/arch: amd64
- go/version: go1.16.3
- go/linking: static
- go/tags: none
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Ubuntu 18.04LTS / 64 bit
Which cloud storage system are you using? (eg Google Drive)
Openstack Swift
The command you were trying to run (eg rclone copy /tmp remote:tmp)
ls or ls -al
The rclone config contents with secrets removed.
[remote]
type = swift
user = storage_XXXX
key = XXXX
auth = http://XXX:5000/v2.0
tenant = XXX
auth_version = 2
I simplified the launch command:
/usr/bin/rclone --config=/etc/ABCconf/ABCstore.config mount remote:ABCVIDEOSTORE /root/ABCVIDEOSTORE/ABCVIDEOSTORE \
--transfers=8 \
--log-level DEBUG --log-file /var/log/rclone/ABCVIDEOSTORE.log &
A log from the command with the -vv flag
The relevant log excerpt is short:
2021/05/30 14:43:46 DEBUG : : >Statfs: stat={Blocks:312985161511 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2021/05/30 14:43:47 DEBUG : ABCDvideos/ABCDLibrary/: Attr:
2021/05/30 14:43:47 DEBUG : ABCDvideos/ABCDLibrary/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2021/05/30 14:43:47 DEBUG : ABCDvideos/ABCDLibrary/: ReadDirAll:
2021/05/30 14:43:47 DEBUG : ABCDvideos/ABCDLibrary/: >ReadDirAll: item=2, err=<nil>
2021/05/30 14:43:47 DEBUG : /: Attr:
2021/05/30 14:43:47 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2021/05/30 14:43:47 DEBUG : : Statfs:
2021/05/30 14:43:47 DEBUG : : >Statfs: stat={Blocks:312985161511 Bfree:274877906944 Bavail:274877906944 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
I ran the ls command and got nothing returned.
# ls -al
total 0
drwxr-xr-x 1 root root 0 May 30 14:43 .
drwxr-xr-x 1 root root 0 May 30 14:42 ..
#
https://pastebin.com/B2bLNMpi
shows a bunch of directory look ups that do exist on the object store, but are failing.
The total volume size reported by df -kh seems correct, but the total file size reported by du -kh is vastly understated.
The following excerpt shows the relevant directory being rescanned at the end of 5 minutes, but no errors, and no contents result.
2021/05/30 14:56:02 DEBUG : /: Lookup: name="ABCDvideos"
2021/05/30 14:56:02 DEBUG : : Re-reading directory (5m0.483601361s old)
2021/05/30 14:56:02 DEBUG : /: >Lookup: node=ABCDvideos/, err=
2021/05/30 14:56:02 DEBUG : ABCDvideos/: Attr:
2021/05/30 14:56:02 DEBUG : ABCDvideos/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2021/05/30 14:56:02 DEBUG : ABCDvideos/: Lookup: name="ABCDLibrary"
2021/05/30 14:56:02 DEBUG : ABCDvideos: Re-reading directory (5m0.457610857s old)
2021/05/30 14:56:02 DEBUG : ABCDvideos/: >Lookup: node=ABCDvideos/ABCDLibrary/, err=
2021/05/30 14:56:02 DEBUG : ABCDvideos/ABCDLibrary/: Attr:
2021/05/30 14:56:02 DEBUG : ABCDvideos/ABCDLibrary/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2021/05/30 14:56:02 DEBUG : ABCDvideos/ABCDLibrary/: Lookup: name="8a164c4c4ba14570862759f627d61ca7"
2021/05/30 14:56:02 DEBUG : ABCDvideos/ABCDLibrary: Re-reading directory (5m0.524703817s old)
2021/05/30 14:56:02 DEBUG : /: Attr:
I have tried a wide variety of caching/no caching and these settings and the rclone version were all working stably until a few days ago.
If rclone were having problems with the object store, I'm assuming it would log something. I did see some kernel timeouts, but rebooted and they have not reoccurred.
Thanks in advance!