Docker-Compose not working

Hey,

I haven't found an official example and tried to create one by myself then. I already tried some variations but the error is mainly:

2020/04/17 11:41:23 Fatal error: unknown command "rclone mount ....

My try:

version: '3.7'
services:
  rclone:
    image: rclone/rclone:beta
    user: 1000:1000
    restart: always
    container_name: rclone
    volumes:
      - ~/.config/rclone:/config/rclone
      - /mnt/google:/data:shared
      - /etc/passwd:/etc/passwd:ro
      - /etc/group:/etc/group:ro
    cap_add:
      - SYS_ADMIN
    devices:
      - /dev/fuse
    security_opt:
      - apparmor:unconfined
    command: >
      "rclone mount gd:/cloud /data/cloud-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 128M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-cloud.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5576 --stats 0 --timeout 1h --umask 007 --use-mmap
      &&
      rclone mount emp-gd:/ /data/emp-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-emp.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5571 --stats 0 --timeout 1h --use-mmap
      &&
      rclone mount emp-out:/ /data/emp-out-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-emp-out.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5575 --stats 0 --timeout 1h --use-mmap
      &&
      rclone mount ipt-gd:/ /data/ipt-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-ipt.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5572 --stats 0 --timeout 1h --use-mmap
      &&
      rclone mount tnt-gd:/ /data/tnt-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-tnt.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5573 --stats 0 --timeout 1h --use-mmap
      &&
      rclone mount tnthb-gd:/ /data/tnthb-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-tnthb.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5580 --stats 0 --timeout 1h --use-mmap
      &&
      rclone mount tnts-gd:/ /data/tnts-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-tnts.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5574 --stats 0 --timeout 1h --use-mmap
      &&
      rclone mount ts-gd:/ /data/ts-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-ts.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5578 --stats 0 --timeout 1h --use-mmap
      &&
      rclone mount tss-gd:/ /data/tss-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-tss.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5579 --stats 0 --timeout 1h --use-mmap
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5576
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5571
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5575
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5572
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5573
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5580
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5574
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5578
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5579"

I also tried with sh -c, with and without rclone in the commands..

Just a guess, but is the " at the start of "rclone mount being literally interpreted? Try removing it...

Hey,

thanks for your answer. But this is most likely not the cause. I tried several formattings as well as ' and " from various pages (and most of them actually repeat over the years tho) but that doesn't work.

So I just got my productive 10 minutes and found some things which leads me to a theory..:

If we check the rclone Dockerfile we notice it uses an ENTRYPOINT and no CMD, while the entrypoint also is just rclone. That also fits the example in the install instructions where everything after the image rclone/rclone acts as as COMMAND. Thus the executing result will be ENTRYPOINT rclone + COMMAND mount dropbox:Photos /data/mount & will be rclone mount dropbox:Photos /data/mount &.

Theory: In that case I think the best solution on my side is to overwrite the ENTRYPOINT in the docker-compose executing all my commands.
A better solution might be using CMD ["/bin/sh"] as in official Alpine Dockerfile. That way my command via sh-c would probably have worked and now maybe gets confused by the ENTRYPOINT?

Not too sure on that yet... But next I will check if I have success overwriting the ENTRYPOINT. Altough I can't be the only one with that problem?

You can certainly override the ENTRYPOINT with plain docker, so I guess that is possible with docker compose, though I have 0 experience of using it!

Or you could build your own container

Something like (totally untested!!)

FROM rclone/rlone:beta
ADD /myscript.sh /
CMD ["/myscript.sh"]

Now you can put whatever you want in myscript.sh and then use docker build to create your own image.

Then you don't have to worry about a complicated command in the compose file :slight_smile:

I was even about to try that...

Creating network "rclone_default" with the default driver
Building rclone
Step 1/3 : FROM rclone/rlone:beta
ERROR: Service 'rclone' failed to build: pull access denied for rclone/rlone, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

Yeah I do have to use an empty entrypoint in the docker-compose to get rid of the entrypoint locking to "rclone".
However I don't get the command to work to mount everything and do the cache warmup.

Anyway I honestly can't spend more time on such a simple issue and I even assume that 80% of the users aren't advanced enough to make it work.
If that is the official rclone Image I really would like to ask the creator to make it actually usable and working...

Here is how you set the ENTRYPOINT with docker compose

Typo; should be rclone:beta

I know how to set the entrypoint... You just wouldn't want to use such a long command (i.e. multiple commands) in the entrypoint syntax. Overwriting entrypoint with "empty" works just fine tho. Just the command doesn't.

I don't really understand why not! I'm not a Docker expert just an occasional user.

Well if you can figure out how to improve it I'd be grateful! Open source projects rely on knowledgeable people dropping in to fix difficult stuff. The Docker file is here: https://github.com/rclone/rclone/blob/master/Dockerfile

Well in theory it works that way, either way, using entrypoint: "" and command as well as maintaining my own dockerfile and loading all the commands as the entrypoint Script.

But I simply don't get further as:

2020/04/29 05:55:16 NOTICE: Serving remote control on http://127.0.0.1:5576/
2020/04/29 05:55:16 ERROR : rc: "vfs/refresh": error: couldn't find method "vfs/refresh"

Will need to think about a theory that could work out to improve the Container but I'm not good at it.

You'll get that if the mount hasn't started at all or hasn't started yet. The vfs rc commands aren't plumbed in until the mount has started I don't think.

Thx, good info. Will think about a solution for that too...

This is caused by the ENTRYPOINT being defined in the Dockerfile, and this will run by default even if a command is given. You're getting the error because it's being interpreted as <entrypoint> <command>, or rclone rclone mount..., as you noted in your message earlier in the week.

Also, your > notation is breaking the argument list. You can remove it - quotes work across multiple lines in yaml.

The ideal fix to this would be to change ENTRYPOINT to CMD in the Dockerfile.

I considered opening a PR to fix this and the example compose files, but unfortunately it means that anybody who built their own compose files based on those examples would also need to update theirs to work with the latest container. As it currently is however, the example compose files do work, which is why you're likely the only one having this issue.

Alternatively, the compose file in the OP could be modified to work without changing the entrypoint like so (I think - I don't have a setup to test this handy, so if you could give it a go and let me know that would be great).

version: '3.7'
services:
  rclone:
    image: rclone/rclone:beta
    user: 1000:1000
    restart: always
    container_name: rclone
    volumes:
      - ~/.config/rclone:/config/rclone
      - /mnt/google:/data:shared
      - /etc/passwd:/etc/passwd:ro
      - /etc/group:/etc/group:ro
    cap_add:
      - SYS_ADMIN
    devices:
      - /dev/fuse
    security_opt:
      - apparmor:unconfined
    command: "mount gd:/cloud /data/cloud-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 128M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-cloud.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5576 --stats 0 --timeout 1h --umask 007 --use-mmap
      &&
      rclone mount emp-gd:/ /data/emp-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-emp.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5571 --stats 0 --timeout 1h --use-mmap
      &&
      rclone mount emp-out:/ /data/emp-out-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-emp-out.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5575 --stats 0 --timeout 1h --use-mmap
      &&
      rclone mount ipt-gd:/ /data/ipt-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-ipt.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5572 --stats 0 --timeout 1h --use-mmap
      &&
      rclone mount tnt-gd:/ /data/tnt-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-tnt.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5573 --stats 0 --timeout 1h --use-mmap
      &&
      rclone mount tnthb-gd:/ /data/tnthb-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-tnthb.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5580 --stats 0 --timeout 1h --use-mmap
      &&
      rclone mount tnts-gd:/ /data/tnts-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-tnts.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5574 --stats 0 --timeout 1h --use-mmap
      &&
      rclone mount ts-gd:/ /data/ts-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-ts.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5578 --stats 0 --timeout 1h --use-mmap
      &&
      rclone mount tss-gd:/ /data/tss-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-tss.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5579 --stats 0 --timeout 1h --use-mmap
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5576
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5571
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5575
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5572
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5573
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5580
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5574
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5578
      &&
      rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5579"

Also, if I remember correctly, mount doesn't run in daemon mode by default, so you'd need to add --daemon to each mount to get the next one to run.

I don't think that would that keep the existing uses of the Docker image working would it? https://rclone.org/install/#install-with-docker

I just re-read the docker docs and it says

An ENTRYPOINT allows you to configure a container that will run as an executable.

Which I would say describes the rclone container....

Maybe we need to be providing a second container which doesn't set ENTRYPOINT?

Correct - Anybody using the current image would break, which is why I didn't PR it :smiley:

To be honest I think the entrypoint is fine how it currently is. It's not hard to override, and only causes problems if you're trying to do something overly complex (like mount 9 drives in a single container). If rclone is being used as a standalone program as intended, it works fine.

If users are trying to do complex tasks with docker+rclone, they should be experienced enough to write their own dockerfiles and understand the ramifications of a bunch of FUSE mounts in a container. There are better ways to handle the OPs use-case without even having to overwrite the entrypoint, and provide a more stable environment, eg:

version: '3.7'
services:
  rclone_gd: &template
    image: rclone/rclone:beta
    user: 1000:1000
    restart: always
    container_name: rclone
    volumes:
      - ~/.config/rclone:/config/rclone
      - /mnt/google:/data:rshared
      - /etc/passwd:/etc/passwd:ro
      - /etc/group:/etc/group:ro
    cap_add:
      - SYS_ADMIN
    devices:
      - /dev/fuse
    security_opt:
      - apparmor:unconfined
    command: "mount gd:/cloud /data/cloud-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 128M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-cloud.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5576 --stats 0 --timeout 1h --umask 007 --use-mmap"

  rclone_emp_gd: 
    <<: *template
    command: "mount emp-gd:/ /data/emp-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-emp.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5571 --stats 0 --timeout 1h --use-mmap"

  rclone_emp_out: 
    <<: *template
    command: "mount emp-out:/ /data/emp-out-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-emp-out.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5575 --stats 0 --timeout 1h --use-mmap"

  rclone_ipt_gd: 
    <<: *template
    command: "mount ipt-gd:/ /data/ipt-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-ipt.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5572 --stats 0 --timeout 1h --use-mmap"

  rclone_tnt_gd: 
    <<: *template
    command: "mount tnt-gd:/ /data/tnt-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-tnt.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5573 --stats 0 --timeout 1h --use-mmap"

  rclone_tnthb_gd: 
    <<: *template
    command: "mount tnthb-gd:/ /data/tnthb-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-tnthb.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5580 --stats 0 --timeout 1h --use-mmap"

  rclone_tnts_gd: 
    <<: *template
    command: "mount tnts-gd:/ /data/tnts-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-tnts.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5574 --stats 0 --timeout 1h --use-mmap"

  rclone_ts_gd: 
    <<: *template
    command: "mount ts-gd:/ /data/ts-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-ts.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5578 --stats 0 --timeout 1h --use-mmap"

  rclone_tss_gd: 
    <<: *template
    command: "mount tss-gd:/ /data/tss-gd --allow-other --async-read=false --attr-timeout 1000h --buffer-size 32M --dir-cache-time 1000h --drive-chunk-size 32M --log-level INFO --log-file /data/logs/mount-tss.log --poll-interval 15s --rc --rc-addr 127.0.0.1:5579 --stats 0 --timeout 1h --use-mmap"
2 Likes

:slight_smile: and thanks for your explanation.

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.