I am very excited to use the new metadata features, but when attempting to sync data w/ metadata from my local, centos7 box to aws s3, i get the below error. I saw this post from a few weeks ago where the poster seemed to be doing exactly what I want to do and got a lot farther than i did, and I can't figure out why my attempt would be any different than his. I went as far as trying @ncwfix-xattr-notsup branch to see if that helps but i get the same error. basically no matter what I do:
failed to read metadata from source object: function not implemented
the dir itself I'm testing locally:
[zack.ramjan@submit admin]$ ls -laR testmd
testmd:
total 18
drwxrwx---+ 2 zack.ramjan hpcusers 4096 Sep 26 12:23 .
drwxrwx---+ 62 zack.ramjan it infrastructure team 8192 Sep 26 13:46 ..
-rw-rwx---+ 1 zack.ramjan hpcadmins 6 Sep 26 12:22 test.withacls
Run the command 'rclone version' and share the full output of the command.
[zack.ramjan@submit admin]$ ~/software/rclone/rclone-v1.59.2-linux-amd64/rclone sync -vvvv --metadata testmd/test.withacls aws:motuz-hot/testmd/
2022/09/26 13:48:22 DEBUG : rclone: Version "v1.59.2" starting with parameters ["/home/zack.ramjan/software/rclone/rclone-v1.59.2-linux-amd64/rclone" "sync" "-vvvv" "--metadata" "testmd/test.withacls" "aws:motuz-hot/testmd/"]
2022/09/26 13:48:22 DEBUG : Creating backend with remote "testmd/test.withacls"
2022/09/26 13:48:22 DEBUG : Using config file from "/home/zack.ramjan/.config/rclone/rclone.conf"
2022/09/26 13:48:22 DEBUG : fs cache: adding new entry for parent of "testmd/test.withacls", "/varidata/research/admin/testmd"
2022/09/26 13:48:22 DEBUG : Creating backend with remote "aws:motuz-hot/testmd/"
2022/09/26 13:48:22 DEBUG : fs cache: renaming cache item "aws:motuz-hot/testmd/" to be canonical "aws:motuz-hot/testmd"
2022/09/26 13:48:22 DEBUG : test.withacls: Need to transfer - File not found at Destination
2022/09/26 13:48:22 ERROR : test.withacls: Failed to copy: failed to read metadata from source object: function not implemented
2022/09/26 13:48:22 ERROR : Attempt 1/3 failed with 1 errors and: failed to read metadata from source object: function not implemented
2022/09/26 13:48:22 DEBUG : test.withacls: Need to transfer - File not found at Destination
2022/09/26 13:48:22 ERROR : test.withacls: Failed to copy: failed to read metadata from source object: function not implemented
2022/09/26 13:48:22 ERROR : Attempt 2/3 failed with 1 errors and: failed to read metadata from source object: function not implemented
2022/09/26 13:48:22 DEBUG : test.withacls: Need to transfer - File not found at Destination
2022/09/26 13:48:22 ERROR : test.withacls: Failed to copy: failed to read metadata from source object: function not implemented
2022/09/26 13:48:22 ERROR : Attempt 3/3 failed with 1 errors and: failed to read metadata from source object: function not implemented
2022/09/26 13:48:22 INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.3s
2022/09/26 13:48:22 DEBUG : 5 go routines active
2022/09/26 13:48:22 Failed to sync: failed to read metadata from source object: function not implemented```
The filesystem is mounted on both centos7 and centos8, so when I read the file2.txt created on centos7 using centos8 i see the metadata, so its getting set via rclone, just not able to read.
I've attached the strace output from both the working (rhel8) and not working (centos7) systems. It appears that the process sends itself a SIGURG (line 718 in centos7 trace)
VERSIONS
statx() was added to Linux in kernel 4.11; library support was added in
glibc 2.28.
What are the kernel versions for centos7/centos8?
4.11 was released in April 2017 so relatively recently. I think rclone should probably have a fallback if statx isn't implemented.
Can you try this which implements a fallback if statx() returns ENOTSUP. This means we can't read the birth time of the file but that shouldn't be too much of an issue.
That must be it. RHEL 7 uses kernel 3.10xxx, so way below the req'd 4.11 of statx. 3.10 is indeed very old. But on the other hand its branch is still actively being supported and maintained by Redhat/IBM for at least another couple of years, and still fairly popular. So I think your workaround will be of benefit to at a meaningful subset of the community, so thanks for making the fix. I will test shortly and let you know the results.
So in testing, it did not seem to change the output:
[zack.ramjan@login02 test2]$ /tmp/rclone-v1.60.0-beta.6469.8b52544eb.fix-local-metadata-linux-amd64/rclone lsf -F pM test
2022/10/06 10:15:18 ERROR : test: Failed to read metadata: function not implemented
test;{}
[zack.ramjan@login02 test2]$ /tmp/rclone-v1.60.0-beta.6475.1c8639a79.fix-local-metadata-linux-amd64/rclone lsf -F pM test
test;{"atime":"2022-10-06T13:10:02.520922349-04:00","gid":"1351814256","mode":"100644","mtime":"2022-09-26T15:53:12.849183-04:00","uid":"1351813916"}
seems to be working now on centos7 RHEL7, great!
I am gearing up to move multiple petabytes and hundreds of millions of files of research data so have been doing various testing. I encountered another metadata issue which I just created a new post for. Thanks for all your hard work!