Edit: Fixed it by setting up my own credentials and fixed some memory error.
Hello,
I have trouble with some weird characters which kills rclone completely.
The log: (a spoiler tag would be great here…)
2018/07/13 22:03:35 DEBUG : rclone: Version "v1.42" starting with parameters ["moveto" "/home/downloads/not_processed/" "remote:/some/folders" "drive-chunk-size" "4096M" "--transfers" "1" "-vv"]
ton of uploads after that - nothing unusual - until:
2018/07/13 22:05:36 INFO :
Transferred: 2.409 GBytes (20.398 MBytes/s)
Errors: 0
Checks: 22
Transferred: 22
Elapsed time: 2m0.9s
Transferring:
* ...a.gerson.and.lucy.heart.enthusiastic.mp4: 0% /1.391G, 0/s, -
2018/07/13 22:05:38 DEBUG :
Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=202264815644, userRateLimitExceeded)
xxx/vivthomas.18.07.13.gina.gerson.and.lucy.heart.enthusiastic.mp4: Sending chunk 0 length 1493400521
2018/07/13 22:06:06 INFO : xxx/vivthomas.18.07.13.gina.gerson.and.lucy.heart.enthusiastic.mp4: Copied (new)
2018/07/13 22:06:06 INFO : xxx/vivthomas.18.07.13.gina.gerson.and.lucy.heart.enthusiastic.mp4: Deleted
2018/07/13 22:06:08 NOTICE: Local file system at /home/downloads/not_processed: Can't store file with control characters: "tv/Marvels.Luke.Cage.S02E08.1080p.WEB.x264-METCON[rarbg]/P^��y\u007f"
2018/07/13 22:06:09 INFO : other/Hacking Raspberry Pi/Hacking Raspberry Pi.zip: Copied (new)
2018/07/13 22:06:09 INFO : other/Hacking Raspberry Pi/Hacking Raspberry Pi.zip: Deleted
2018/07/13 22:06:10 INFO : software/4K Video Downloader 4.4.8.2317 + Crack [CracksNow]/Visit SoupGet.com.url: Copied (new)
2018/07/13 22:06:10 INFO : software/4K Video Downloader 4.4.8.2317 + Crack [CracksNow]/Visit SoupGet.com.url: Deleted
Killed
I see the ‘Killed’ only via shell output -vv. In the .log is only the usual stuff and just ends one line before killed.
Is there a solution for this problem? I just renamed it manually to a random word, so it gets uploaded. If rclone can not provide a solution then I have to write a short script to sanitize bad characters from the upload folder - But I think I am not the only one here with the bad char problem, am I?
By the way. If I do more transfers than one with rclone, every transfer is capped at exactly 0.7 mb/s. No matter for two or ten transfers at the same time. Really weird. But I can most of the time upload with max. bandwidth (500 Mbit server) with just one transfer, so I had not the struggle to investigate into that further.
Edit: After the renaming of the bad chars and doing more running tests I still get the ‘killed’ thing without any error. Really annoying.
Transferred: 1.598 GBytes (26.950 MBytes/s)
Errors: 0
Checks: 5
Transferred: 5
Elapsed time: 1m0.7s
Transferring:
* ...AC@44.1khz24bit]/01. Missionary Man.flac: 0% /54.213M, 0/s, -
2018/07/13 22:36:21 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=202264815644, userRateLimitExceeded)
2018/07/13 22:37:21 DEBUG : music/Eurythmics - 1986 - Revenge (Remastered) (2018 HDtracks) [FLAC@44.1khz24bit]/01. Missionary Man.flac: Sending chunk 0 length 56845965
2018/07/13 22:37:23 INFO : music/Eurythmics - 1986 - Revenge (Remastered) (2018 HDtracks) [FLAC@44.1khz24bit]/01. Missionary Man.flac: Copied (new)
2018/07/13 22:37:23 INFO : music/Eurythmics - 1986 - Revenge (Remastered) (2018 HDtracks) [FLAC@44.1khz24bit]/01. Missionary Man.flac: Deleted
2018/07/13 22:37:25 DEBUG : other/Inside Solid State Drives (SSDs), Second Edition/Inside Solid State Drives (SSDs), Second Edition.zip: Sending chunk 0 length 27324562
2018/07/13 22:37:27 INFO : other/Inside Solid State Drives (SSDs), Second Edition/Inside Solid State Drives (SSDs), Second Edition.zip: Copied (new)
2018/07/13 22:37:27 INFO : other/Inside Solid State Drives (SSDs), Second Edition/Inside Solid State Drives (SSDs), Second Edition.zip: Deleted
2018/07/13 22:37:28 INFO : games/hi-the.vagrant/IGG-GAMES.COM.url: Copied (new)
2018/07/13 22:37:28 INFO : games/hi-the.vagrant/IGG-GAMES.COM.url: Deleted
2018/07/13 22:37:30 INFO : games/codex-dark.souls.remastered.update.v1.03/CODEX/steam_api64.cdx: Copied (new)
2018/07/13 22:37:30 INFO : games/codex-dark.souls.remastered.update.v1.03/CODEX/steam_api64.cdx: Deleted
Killed
Maybe it has to do with the ‘user exceed configured project quota’ ? I am using the credentials from rclone. But it just stops right after a transfer and thats it. No indication what killed rclone.
I have just uploaded ~ 300 GB today into the google drive, so I am not shadow banned for hitting the daily cap.
Another test:
2018/07/13 22:44:05 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2018/07/13 22:44:05 DEBUG : remote: Saved new token in config file
2018/07/13 22:44:05 DEBUG : pacer: Rate limited, sleeping for 1.886117799s (1 consecutive low level retries)
2018/07/13 22:44:05 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/07/13 22:44:05 DEBUG : pacer: Rate limited, sleeping for 2.369968834s (2 consecutive low level retries)
2018/07/13 22:44:05 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/07/13 22:44:07 DEBUG : pacer: Resetting sleep to minimum 10ms on success
2018/07/13 22:44:10 DEBUG : pacer: Rate limited, sleeping for 1.776575742s (1 consecutive low level retries)
2018/07/13 22:44:10 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=202264815644, userRateLimitExceeded)
2018/07/13 22:44:10 DEBUG : pacer: Resetting sleep to minimum 10ms on success
2018/07/13 22:44:10 DEBUG : pacer: Rate limited, sleeping for 1.891551926s (1 consecutive low level retries)
2018/07/13 22:44:10 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=202264815644, userRateLimitExceeded)
2018/07/13 22:44:10 DEBUG : pacer: Rate limited, sleeping for 2.467247173s (2 consecutive low level retries)
2018/07/13 22:44:10 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=202264815644, userRateLimitExceeded)
2018/07/13 22:44:10 DEBUG : pacer: Resetting sleep to minimum 10ms on success
2018/07/13 22:44:16 DEBUG : tv/Robot Chicken S09E17 (1080p AMZN WEB-DL x265 HEVC 10bit AAC 5.1 Qman) [UTR]/Robot.Chicken.S09E17.1080p.AMZN.WEB-DL.x265.10bit.AAC.5.1-Qman[UTR].mkv: Sending chunk 0 length 233674860
2018/07/13 22:44:21 INFO : tv/Robot Chicken S09E17 (1080p AMZN WEB-DL x265 HEVC 10bit AAC 5.1 Qman) [UTR]/Robot.Chicken.S09E17.1080p.AMZN.WEB-DL.x265.10bit.AAC.5.1-Qman[UTR].mkv: Copied (new)
2018/07/13 22:44:21 INFO : tv/Robot Chicken S09E17 (1080p AMZN WEB-DL x265 HEVC 10bit AAC 5.1 Qman) [UTR]/Robot.Chicken.S09E17.1080p.AMZN.WEB-DL.x265.10bit.AAC.5.1-Qman[UTR].mkv: Deleted
2018/07/13 22:44:26 DEBUG : tv/Shades of Blue (2016) S03E04 (1080p AMZN WEB-DL x265 HEVC 10bit AAC 5.1 Vyndros)/Shades.of.Blue.S03E04.A.Walking.Shadow.1080p.10bit.AMZN.WEB-DL.AAC5.1.HEVC-Vyndros.mkv: Sending chunk 0 length 864338631
2018/07/13 22:44:42 INFO : tv/Shades of Blue (2016) S03E04 (1080p AMZN WEB-DL x265 HEVC 10bit AAC 5.1 Vyndros)/Shades.of.Blue.S03E04.A.Walking.Shadow.1080p.10bit.AMZN.WEB-DL.AAC5.1.HEVC-Vyndros.mkv: Copied (new)
2018/07/13 22:44:42 INFO : tv/Shades of Blue (2016) S03E04 (1080p AMZN WEB-DL x265 HEVC 10bit AAC 5.1 Vyndros)/Shades.of.Blue.S03E04.A.Walking.Shadow.1080p.10bit.AMZN.WEB-DL.AAC5.1.HEVC-Vyndros.mkv: Deleted
2018/07/13 22:44:45 INFO : tv/Teachers.2016.S03E05.1080p.WEB.x264-TBS[TGx]/[TGx]Downloaded from torrentgalaxy.org .txt: Copied (new)
2018/07/13 22:44:45 INFO : tv/Teachers.2016.S03E05.1080p.WEB.x264-TBS[TGx]/[TGx]Downloaded from torrentgalaxy.org .txt: Deleted
2018/07/13 22:44:45 DEBUG : pacer: Rate limited, sleeping for 1.152809982s (1 consecutive low level retries)
2018/07/13 22:44:45 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/07/13 22:44:45 DEBUG : pacer: Rate limited, sleeping for 2.969791879s (2 consecutive low level retries)
2018/07/13 22:44:45 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/07/13 22:44:47 DEBUG : pacer: Resetting sleep to minimum 10ms on success
Killed