I'll create a docker-compose example in the next days to reproduce this issue. Is here the correct place to post it or in GitHub issues?
It's a little hard to say honestly since so much in your config has been generalized (which of course might be needed for security/privacy depending on the spesific details).
If you believe you have followed the documentation instructions correctly already then I'll take you at your word on that - but it hard for me to verify for you.
I'd post any further information or resources here. Then if an issue is needed later because we are fairly sure there is a problem in rclone that needs to be corrected - then you can just link the thread in addition to a short summary.
One more thing you can definitely do right off the bat is do a full debug-log. Add this to your command:
--log-level DEBUG
--log-file=mylogfile.txt
Try to log the problem as it occurs while keeping it as short as possible because debug output is very verbose. For example I might suggest you do the same thing as you tried - but do it with a new test folder that is just a few files large. Then perform the sync twice to demonstrate that it works first - then fails after. That debug log should give us lot of insight of what is happening under the hood - and hopefully reveal the problem. I'm not a an expert in decoding the debug logs, but I'll give it a shot.
yes, i agree with thestigma,
and if you cannot post a version of your config, that is a problem.
you can change the ipaddress or domain name to something else but leave everything else alone.
and create a second endpoint for testing, without the bucket name in the endpoint.
Maybe there was some confusion because of the long thread - I did actually.
Plus I've removed usernames/passwords/access and secret keys, but they are correct and correctly tell me invalid credentials if I type them wrong.
[SFTP]
type = sftp
user = username
pass = password
host = sftp.foo.com[S3]
type = s3
region = us-east-1
server_side_encryption =
acl = private
provider = Minio
access_key_id = access_key
secret_access_key = secret_key
endpoint = minio.foo.com
env_auth = false
location_constraing =
The sync command is now:
rclone -v --sftp-use-insecure-cipher sync SFTP:/foo/ S3:bucket/bar/foo/
I still owe you the verbose log output - got distracted with something else.
Behaviour:
First run works, if bucket/bar/foo/ doesn't exist.
2nd run, when bucket/bar/foo/ exists outputs:
Failed to create file system for "S3:bucket/bar/foo/": is a file not a directory
if you look at the top of this post, for some reason, you content has been flagged.
"This post was flagged by the community and is temporarily hidden."
and if you compare the two configs, there are different,
the first config has endpoint = [endpoint.com/bucket/](http://endpoint.com/bucket/)
which we were complaining about
and now the recent config is endpoint = [minio.foo.com](http://minio.foo.com)
change S3:bucket/bar/foo/ to S3:bucket2/bar2/foo2/ and test.
Can you tell us what system is at the end of this address and where this info came from?
If I understand correctly, this system uses some form of private server (that rclone interfaces with) - which then in return talks to a cloud-storage backend server on the net presumably?
I just want to nail down that we aren't trying to make rclone connect directly to the cloud-storage backend when we perhaps should be connecting via the server running at the workplace.
Usually you DO connect directly to the service-provider (with Wasabi for example), but documentation seems to indicate that may not be the case for minio - possibly due to API limitations or other compatibility reasons.
I may be off on my understanding here because this system is unknown to me. I am just trying to make it jive with what I can read from the documention to find the potential problem source. We may indeed be barking up the wrong tree, but I try not make assumptions when troubleshooting. The debug-log would help shed some light - but there is no rush 
No idea why a few post here were flagged... If you did it yourself intentionally, keep in mind there is an edit-button you can use.
I've unhidden the posts 
It would be really useful to see the second run with -vv --dump bodies which will show exactly what rclone requested from minio. Feel free to redact sensitive info from the log.
From my experience the worst leaking of sensitive info in debug logs should be files and folder names.
Correct me if I am wrong, because then I want to be aware of it
That's it mostly... A log with -vv --dump bodies can leak host names and object IDs which may or may not be of concern.
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.