notyou
July 12, 2023, 4:14pm
1
Today, Proton Drive released a Windows desktop client. There's also an opensourced Android client . Given that official opensource clients exist that talk to an API, how feasible would it be to add Proton Drive support to rclone?
If you implement it, find somebody to do this for you or sponsor it.
There are 70 new backend requests pending:) some many years old. Proton Drive is among them already:
opened 08:27PM - 30 Mar 22 UTC
closed 12:08PM - 31 Mar 22 UTC
new backend
Many use ProtonMail, I've been a subscriber for years.
They have opening up P… roton Drive - it's in Beta - but it would be great for rclone to get on the bandwagon as a client application.
But there is good news:)
Somebody is actually working on it:
rclone:master ← henrybear327:backend/ProtonDrive
opened 10:25PM - 28 Jun 23 UTC
#### What is the purpose of this change?
Add support for a new backend [Proto… n Drive](https://proton.me/drive)
#### Was the change discussed in an issue or in the forum before?
#6072
#### Checklist
- [x] I have read the [contribution guidelines](https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#submitting-a-new-feature-or-bug-fix).
- [x] I have added tests for all changes in this PR if appropriate.
- [x] I have added documentation for the changes if appropriate.
- [ ] All commit messages are in [house style](https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#commit-messages).
- [ ] I'm done, this Pull Request is ready for review :-)
#### Tests performed (passed all tests for now)
- At `root`: `make && test_all -backends protondrive`
- At `backend/protondrive`: `go test -race -v ./...`
- At `root`: `make && rclone purge -v TestProtonDrive:rclone-info; rclone test info --all -vv --write-json remote.json TestProtonDrive:rclone-info && go run cmd/test/info/internal/build_csv/main.go -o remote.csv remote.json`
- Linter
- At `backend/protondrive`, run `golangci-lint run`
#### Notes
- Willing to keep on maintaining and improving this backend
- Wrote the [Proton-API-Bridge](https://github.com/henrybear327/Proton-API-Bridge) library to free the rclone codebase from dealing with the encryption and decryption details
- All the rclone unit and integration tests are run and passed (as specified above)
#### Known issue
- After adding the `Proton-API-Bridge`, as we need this for handling the specifics of the encryption, it introduces a lot of new entries in the `go.mod` and `go.sum`
- AppString, which is currently handled by the bridging library, is hardcoded
- Access token isn’t cached, so if the account has 2FA enabled, the config needs to be changed every time as the 2FA code is read from the config file
- Not working
- `rclone purge -v TestProtonDrive:`: not implemented as we can't delete root folder and recreate one
- ~~`rclone ls TestProtonDrive:`: error msg `Failed to ls with 2 errors: last error was: gopenpgp: error in reading message: openpgp: incorrect key`~~
#### The improvement work for the future
I foresee [the work](https://github.com/henrybear327/Proton-API-Bridge#v2) to be done mostly from the bridging library, as there won't be big changes in the rclone backend code, and would be mostly centered around performance, edge cases, and some error-handling improvements in the bridging library, e.g.:
- Caching the `proton.Link` object, should be able to reduce quite some API calls when doing List()
- Downloading and uploading large files: currently, the entire file will be loaded into the memory and then the encryption/decryption operation for upload/download will then be performed
- Optimize the MimeType detection
- Check the client code and completes better signature verification and edge case handling, e.g. failed upload will leave the draft revision and blocks behind, validation/limitations seen in the open source clients
- Thumbnail support
2 Likes
isuru
August 31, 2023, 11:06am
4
this is so lovely! Can we pay a small amount of money to support this implementation?
It has been already merged and is aimed to be released in v1.64. You can try it now by running the latest rclone beta version
rclone:master ← henrybear327:backend/ProtonDrive
opened 10:25PM - 28 Jun 23 UTC
#### What is the purpose of this change?
Add support for a new backend [Proto… n Drive](https://proton.me/drive)
#### Was the change discussed in an issue or in the forum before?
#6072
#### Checklist
- [x] I have read the [contribution guidelines](https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#submitting-a-new-feature-or-bug-fix).
- [x] I have added tests for all changes in this PR if appropriate.
- [x] I have added documentation for the changes if appropriate.
- [x] All commit messages are in [house style](https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#commit-messages).
- [x] I'm done, this Pull Request is ready for review :-)
#### Tests performed (passed all tests for now)
- At `root`: `make && test_all -backends protondrive`
- At `backend/protondrive`: `go test -race -v ./...`
- At `root`: `make && rclone purge -v TestProtonDrive:rclone-info; rclone test info --all -vv --write-json remote.json TestProtonDrive:rclone-info && go run cmd/test/info/internal/build_csv/main.go -o remote.csv remote.json`
- Linter
- At `backend/protondrive`, run `golangci-lint run`
#### Notes
- Willing to keep on maintaining and improving this backend
- Wrote the [Proton-API-Bridge](https://github.com/henrybear327/Proton-API-Bridge) library to free the rclone codebase from dealing with the encryption and decryption details
- All the rclone unit and integration tests are run and passed (as specified above)
#### Known issue
- After adding the `Proton-API-Bridge`, as we need this for handling the specifics of the encryption, it introduces a lot of new entries in the `go.mod` and `go.sum`
- AppString, which is currently handled by the bridging library, is hardcoded
- Access token isn’t cached, so if the account has 2FA enabled, the config needs to be changed every time as the 2FA code is read from the config file
- Not working
- `rclone purge -v TestProtonDrive:`: not implemented as we can't delete root folder and recreate one
- ~~`rclone ls TestProtonDrive:`: error msg `Failed to ls with 2 errors: last error was: gopenpgp: error in reading message: openpgp: incorrect key`~~
#### The improvement work for the future
I foresee [the work](https://github.com/henrybear327/Proton-API-Bridge#v2) to be done mostly from the bridging library, as there won't be big changes in the rclone backend code, and would be mostly centered around performance, edge cases, and some error-handling improvements in the bridging library, e.g.:
- Caching the `proton.Link` object, should be able to reduce quite some API calls when doing List()
- Downloading and uploading large files: currently, the entire file will be loaded into the memory and then the encryption/decryption operation for upload/download will then be performed
- Optimize the MimeType detection
- Check the client code and completes better signature verification and edge case handling, e.g. failed upload will leave the draft revision and blocks behind, validation/limitations seen in the open source clients
- Thumbnail support
You can always do this to help this project going:)
gmnts
August 31, 2023, 2:02pm
7
Hi,
not sure if this is the right place for this.
I downloaded the current beta and tried the new proton backend.
Most files sync fine from the local HDD but I get a lot of errors, mainly these 2:
31 15:57:27.129406 ERROR RESTY 422: Datei oder Ordner mit diesem Namen existiert bereits. (Code=2500, Status=422), Attempt 1
2023/08/31 15:57:29 ERROR : _BB/US02/IMG_20230308_2_5.jpg: Failed to copy: a draft exist - usually this means a file is being uploaded at another client, or, there was a failed upload attempt
the destination was empty and I waited until rclone finshed. The summary was:
2023/08/31 15:58:02 ERROR : Attempt 3/3 failed with 36 errors and: a draft exist - usually this means a file is being uploaded at another client, or, there was a failed upload attempt
2023/08/31 15:58:02 Failed to copy with 36 errors: last error was: a draft exist - usually this means a file is being uploaded at another client, or, there was a failed upload attempt
The files where these errors were thrown, were in deed NOT synced.
I'm not sure what is going on here.
It is still work in progress - read all proton PRs - Pull requests · rclone/rclone · GitHub - maybe answer or clues are there.
You could also try to contact the author on github:
Beta testing can be actually valuable help in fixing all bugs before release.
ncw
August 31, 2023, 3:34pm
9
The protondrive backend author has been busy but we hope to get it nailed down for the release.
I'll send them this link!
asdffdsa
August 31, 2023, 4:23pm
10
gmnts:
I get a lot of errors,
i am not getting errors when syncing from local to pdrive.rclone version-vv --dump=headers
Hey @gmnts ,
Can you please try the configuration mentioned here (https://github.com/rclone/rclone/issues/6072#issuecomment-1691347623 ) and let me know if it works? Thank you!
I will be making a change in the error message so it's easier to understand!
gmnts
September 1, 2023, 9:05am
12
@asdffdsa @henrybear327 thanks for your replies
I have appended my findings to the GH Issue, as I think that's a better place for this. Tell me if I should rather post here or open a new topic.
1 Like