What is the problem you are having with rclone?
I have lots of error that similar to this
2019/10/14 01:04:25 ERROR : k1b727b8va,SQ8QODWABQ01L/1N7g4orK7ywi3ehlDvuAk1oA2hwxmHyPn76/30bV4VBklyXF8E9yWsRSaeKo: Couldn't delete: remove /mnt/user/Cloud/ijm/.move/k1b727b8va,SQ8QODWABQ01L/1N7g4orK7ywi3ehlDvuAk1oA2hwxmHyPn76/30bV4VBklyXF8E9yWsRSaeKo: read-only file system
More error logs https://pastebin.com/fVwV9ZB9
What is your rclone version (output from
- os/arch: linux/amd64
- go version: go1.12.10
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Which cloud storage system are you using? (eg Google Drive)
The command you were trying to run (eg
rclone copy /tmp remote:tmp)
rclone move --drive-use-trash --transfers 2 --bwlimit "2.0M" --drive-chunk-size 256k -v /mnt/user/Cloud/ijm/.move/ ijm:encfs
A log from the command with the
-vv flag (eg output from
rclone -vv copy /tmp remote:tmp)
More error logs https://pastebin.com/fVwV9ZB9
Please help. THanks.
The problem seems fairly clear from logs - that this location:
can not be written to (and thus files can not be deleted either at the end of the move).
This may be a general permission problem for this location, or it may be related to lacking permissions on the user-account that rclone runs under.
The obvious next step is to check what the permission settings for that folder-structure looks like. Feel free to post that here. I am pretty bad at Linux permissions, but we have plenty of Linux veterans around who can help if I fail to identify the problem.
Additionally you may want to say what type of user-account rclone runs under. It is your normal user for example? This is helpful to know which category of permissions is relevant here when we look at the files permission settings.
Since you seem to be copying from a mounted location - these flags mount-flags may also be relevant to the issue:
--read-only (obviously this would block write-access if used on the mount)
--allow-other (unless this is used, files shoudl not be generally accessible to other users than the one who started the mount. This should apply to reads as well as writes though, so I doubt this is the case). Most likely rclone mount and rclone move both use the same user-account here anyway? (presumably, but not necessarily).
This is the permission for one of the file that I tracked with the error.
What should I change it into?
Not sure if this helps, but this is the mount command(without revealing the password).
-S -o rw -o allow_other -o uid=99 -o gid=100
The problem I have is only some of the files have this error. a lot of other files doesnt have this error.
I tried to manually delete the file. This is what I get.
So I guess, the problem is not with rclone. Anyway, do you guys have any clue on how to resolve this?
Hmm, I can't spot any obvious problem - but Linux permissions and FUSE options are a little bit out of my normal area of expertise so I think I need a second opinion from someone who knows his stuff on this
@Animosity022 Could you give this a once-over and see if you identify problems?
@bukan.ijam Am i reading this right that you are using encfs for your encryption instead of rclone's native system? (any particular reason for that?) This may have something to do with this. I don't know why it would cause issues, but since I am very unfamiliar with encfs I can not rule it out either...
You aren't using any sort of settings on your encfs format that might relate to this maybe? ...
The reason I use encfs is, at that point in time rclone encrypt function wasnt available.
So this is a quite old setup then? New problems on a system that has been working? Did you make any changes recently?
It would be nice as a quick test if you could just shortcut the encryption and see if the problem goes away then. If it does then we fairly assume that this is related and focus our troubleshooting efforts in that direction. (or rule it out).
The problem has been since the beginning. I just ignore it, until today, when my storage was almost depleted. No new changes.
Another way to 'fix' this problem is by finding duplicate files and delete them. find files in identical folder structure and check each files inside both directory and if it has the same md5, delete them.
Tried google, but i guess my search keyword is not quite specific. Any tips guys?
What are you trying to search for?
As I said I would give it a quick shot to remove the ecryption part of your setup and just test what happens if you try to write and then delete a file. The result of that will tell us a lot. If the problem goes away then we have learned that the problem is likely related to encfs in some fashion and we can work with that. If the problem persists then we can rule out encfs and be reasonably sure that this a problem in either basic permissions or FUSE.
Either way we at least reduce the problem size in half
I suspect when Animosity pops his head in he will probably have a good idea of what the problem is.
Sorry. I didnt quite catch what you said.
Here are the results if i remove the original file. I also double check the file in the encrypted folder. gone.
I think I can confirm that the problem is caused by encfs.
It sounds to me like the problem is related to the mount-system that encfs uses. This may be set to some sort of protected-mode or read-only, or it may just not allow you to delete files on the encrypted end while the mount is active (I don't know as I am not familiar with encfs as I said, but I did read up on the basics of what the system does, which seems not to dis-similar to an rclone crypt remote.)
"fixing" encfs is not something I can help with unfortunately. Animosity may know how, or else we can hope that someone else shows up who has knowledge of it. There are a lot of Linux users here so there's a good chance we can get good help with this. I would suggest you edit the title to include "encfs" so we can attract the attention of someone who knows that system.
Alternatively we can just set up a second mount for you that uses rclone crypt and saves to a separate folder on your remote, then you can slowly migrate over to that system as you go - but still be able to fetch all your existing data through the old setup. This part I can absolutely help you with, and it is not difficult and should slot into your existing setup without much problem.
If you want help on that it would help greatly if you could share your rclone.conf file (but please take care to redact sensitive data, such as any client secrets or crypt keys that might exist). Then I can just add the necessary changes for you if you are uncertain about how to add rclone crypt.
If you try to remove something and it's read-only, you have a mount somewhere there is read only.
You'd have to share your steps and how you are mounting things.
The rclone error isn't from rclone, but from the mount underneath it.
Guys, thanks for all the help. Finally found out the root cause.
Im using the old command(--reverse) while newer encfs uses a new command (--reversewrite)
Normally EncFS provides a plaintext view of data on demand: it stores enciphered data and displays plaintext data. With --reverse it takes as source plaintext data and produces enciphered data on-demand. This can be useful for creating remote encrypted backups, where you do not wish to keep the local files unencrypted.
For example, the following would create an encrypted view in /tmp/crypt-view.
encfs --reverse /home/me /tmp/crypt-view
You could then copy the /tmp/crypt-view directory in order to have a copy of the encrypted data. You must also keep a copy of the file /home/me/.encfs6.xml which contains the filesystem information. Together, the two can be used to reproduce the unencrypted data:
ENCFS6_CONFIG=/home/me/.encfs6.xml encfs /tmp/crypt-view /tmp/plain-view
Now /tmp/plain-view contains the same data as /home/me
Note that --reverse mode only works with limited configuration options, so many settings may be disabled when used. Incompatible options as for now : Filename Initialization Vector Chaining and External IV Chaining.
Same as --reverse but will allow writes, if possible (configuration must have UniqueIV disabled). Incompatible option : Per-File Initialization Vectors.
Thanks @thestigma and @Animosity022 .
Ah, there we go
Glad you got it fixed
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.