Okay I got the files I wanted by downloading them via the command I used before, but I switched the source to the unencrypted remote gdrive:encrypted. The files I downloaded were of course encrypted, so I had to copy the config of the cryp: remote and change the path to the local path where I downloaded the files to, after which I could decrypt the encrypted files via that new remote by simply moving them to any other location on my system.