Git repo on GDrive remote

I have a remote that I use to keep backups of some git repo for compiling D toolchains. Normally I will do the git pull and compilation locally then rclone sync the results for backup. Recently I tried to do git pull directly on the mounted remote just as an experiment. It seems to work but fails at the step where it needs to update some refs. Should this even work with rclone’s vfs or would I need to use overlayfs/unionfs/mergefs or one of their kins?

Here’s my mount command

REMOTE=/home/ec2-user;RCLONE_CONFIG_PASS=$(ssh salts openssl enc -d -aes-256-cbc -in "$REMOTE/<cryptedsalt>" -kfile \<\(openssl rsautl -decrypt -inkey $REMOTE/<privatekey> -in "$REMOTE/<cryptedrcloneinfos>"\) | awk '($0 ~ /^Password/){print substr($0,match($0,":")+2)}') rclone mount crypted_GDrive_remote:/ ~/eDevelopments -c -vv --vfs-read-chunk-size 4k --vfs-read-chunk-size-limit 2M --vfs-cache-mode writes --write-back-cache --dir-cache-time 48h --poll-interval 5m --buffer-size 2M --fast-list --tpslimit 8 --tpslimit-burst 2 --links --ask-password=false
[ocelot@yellowtrain dmd-test]$ git pull --verbose
error: cannot update the ref 'refs/remotes/origin/master': unable to append to '.git/logs/refs/remotes/origin/master': Bad file descriptor
 ! b8c8d956f..855dbe092  master                               -> origin/master  (unable to update local ref)
 = [up to date]          dcache                               -> origin/dcache
 = [up to date]          dmd-1.x                              -> origin/dmd-1.x
 = [up to date]          dmd-cxx                              -> origin/dmd-cxx
 = [up to date]          markdown                             -> origin/markdown
 = [up to date]          newCTFE                              -> origin/newCTFE
 = [up to date]          revert-9282-globtemplate             -> origin/revert-9282-globtemplate
 = [up to date]          revert-9484-move-diagnostic-reporter -> origin/revert-9484-move-diagnostic-reporter
error: cannot update the ref 'refs/remotes/origin/stable': unable to append to '.git/logs/refs/remotes/origin/stable': Bad file descriptor
 ! 881381ea9..becd870db  stable                               -> origin/stable  (unable to update local ref)

dmd-test/ is a directory on the remote that has a copy of a git repo

I don’t see why it wouldn’t with vfs. Vfs writes or full mode should work.

Nope, neither worked.

What was the error? Might be worth producing a debug output and potentially opening an issue. Vfs full should support most file system operations.

-vv --debug-fuse

You could also use the cache backend.

Well on the git end it is error: cannot update the ref 'refs/remotes/origin/master': unable to append to '.git/logs/refs/remotes/origin/master': Bad file descriptor

On the mount end, the log says either DEBUG : fuse: -> [ID=0xf36] Read error=bad file descriptor or DEBUG : fuse: -> [ID=0xf86] Lookup error=ENOENT

Here’s the full log

I would open an issue. I don’t believe you should be getting that error. The cache backend might be more appropriate for your needs anyway though.

This is the problem

2019/05/06 21:56:16 DEBUG : gitDToolChain/dmd-test/.git/logs/refs/remotes/origin/master: Open: flags=O_WRONLY|O_APPEND
2019/05/06 21:56:16 DEBUG : gitDToolChain/dmd-test/.git/logs/refs/remotes/origin/master: >Open: fd=gitDToolChain/dmd-test/.git/logs/refs/remotes/origin/master (rw), err=<nil>
2019/05/06 21:56:16 DEBUG : gitDToolChain/dmd-test/.git/logs/refs/remotes/origin/master: >Open: fh=&{gitDToolChain/dmd-test/.git/logs/refs/remotes/origin/master (rw)}, err=<nil>
2019/05/06 21:56:16 DEBUG : fuse: -> [ID=0xf34] Open 0x5 fl=0
2019/05/06 21:56:16 DEBUG : fuse: <- Read [ID=0xf36 Node=0x38 Uid=1000 Gid=1000 Pid=13861] 0x5 4096 @0x0 dir=false fl=0 lock=0 ffl=OpenWriteOnly+OpenAppend
2019/05/06 21:56:16 DEBUG : &{gitDToolChain/dmd-test/.git/logs/refs/remotes/origin/master (rw)}: Read: len=4096, offset=0
2019/05/06 21:56:16 DEBUG : &{gitDToolChain/dmd-test/.git/logs/refs/remotes/origin/master (rw)}: >Read: read=0, err=bad file descriptor
2019/05/06 21:56:16 DEBUG : fuse: -> [ID=0xf36] Read error=bad file descriptor

The file is opened write only and append, but then read from. Rclone returns EBADF for that which according to the man page for write(2) is the correct thing to return

   EBADF  fd is not a valid file descriptor or is not open for reading.

Why git does that, I don’t know…

It might be worth stracing git to see if that is what it thinks it is doing or whether something has got lost in translation in the kernel/FUSE layers

Hm, yeah I don’t get it. Git is weird… This is a strace of a git clone instead of git pull as I’ve already updated the dmd repo

[ocelot@yellowtrain gitDToolChain]$ strace -yy -o ~/gitstrace git clone --verbose ~/eDevelopments/gitDToolChain/dmd-test
Cloning into '/home/ocelot/eDevelopments/gitDToolChain/dmd-test'...
fatal: Unable to read current working directory: No such file or directory

There’s clearly a dmd-test directory on that remote. I created it before issuing the git clone command

So it seems that git expects a .git directory but doesn’t make one if it doesn’t exits like it would normally if cloning locally…Hm non-existent directory being open readonly is really fishy…

I’m not sure what is going on there! I tried the clone command to a local disk which worked fine, and then to google drive which took ages but appeared to be working…

I think there are corner cases in rclone mount which still need to be wrinkled out. For example this issue: and I guess git is hitting those, being quite an intensive file system user. If we can figure out exactly what is happening then I’m sure I can fix it!

In fact git operations are probably a good test - maybe I could make a test suite for rclone mount using them…

I note also this project which can make a git annex using rclone:

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.