Issue using rclone copy to existing bucket

Help and Support
1

I’m not sure what I’m doing wrong here. I’m specifying an existing bucket in the configured gcloud project. I’m simply trying to copy a single file into that bucket. It appears to be trying to create a bucket? I’m confused.

[user@some-server tmp]$ ls foo
foo
[user@some-server tmp]$ rclone copy foo some-gcloud-project:some-gcloud-bucket
2017/08/09 17:50:32 ERROR : foo: Failed to copy: can't make bucket without project number
2017/08/09 17:50:32 ERROR : Attempt 1/3 failed with 1 errors and: can't make bucket without project number
2017/08/09 17:50:33 ERROR : foo: Failed to copy: can't make bucket without project number
2017/08/09 17:50:33 ERROR : Attempt 2/3 failed with 1 errors and: can't make bucket without project number
2017/08/09 17:50:34 ERROR : foo: Failed to copy: can't make bucket without project number
2017/08/09 17:50:34 ERROR : Attempt 3/3 failed with 1 errors and: can't make bucket without project number
2017/08/09 17:50:34 Failed to copy: can't make bucket without project number
[user@some-server tmp]$ rclone config
Current remotes:

Name                 Type
====                 ====
some-gcloud-project  google cloud storage

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> e
Choose a number from below, or type in an existing value
 1 > some-gcloud-project
remote> 1
--------------------
[some-gcloud-project]
type = google cloud storage
client_id =
client_secret =
project_number =
service_account_file = /opt/rclone/some-gcloud-project-service-account.json
object_acl =
bucket_acl =
location = us-central1
storage_class = MULTI_REGIONAL
--------------------
2

I just had a look at the code…

// Mkdir creates the bucket if it doesn't exist
func (f *Fs) Mkdir(dir string) error {
	f.bucketOKMu.Lock()
	defer f.bucketOKMu.Unlock()
	if f.bucketOK {
		return nil
	}
	_, err := f.svc.Buckets.Get(f.bucket).Do()
	if err == nil {
		// Bucket already exists
		f.bucketOK = true
		return nil
	}

	if f.projectNumber == "" {
		return errors.New("can't make bucket without project number")
	}

So it shouldn’t give you that error unless the bucket didn’t exist

Can you see the bucket if you do rclone lsd some-gcloud-project:

I suppose it might be in the wrong region or something…

3

That’s the weird thing; I can list things just fine.

[user@some-server ~]$ rclone -vv lsd some-gcloud-project:some-gcloud-bucket
2017/08/09 20:03:35 DEBUG : rclone: Version "v1.37" starting with parameters ["rclone" "-vv" "lsd" "some-gcloud-project:some-gcloud-bucket"]
2017/08/09 20:03:35 INFO  : Storage bucket some-gcloud-bucket: Modify window is 1ns
           0 2017-08-09 20:03:35        -1 storage
2017/08/09 20:03:35 DEBUG : Go routines at exit 6
2017/08/09 20:03:35 DEBUG : rclone: Version "v1.37" finishing with parameters ["rclone" "-vv" "lsd" "some-gcloud-project:some-gcloud-bucket"
4

Strange!

If you do

rclone copy foo some-gcloud-project:some-gcloud-bucket -vv --dump-bodies --retries 1

Then can you see what http transactions it does?

5

I see you made an issue - let’s carry this on there!

6

For anyone that wants to continue following this:
https://github.com/ncw/rclone/issues/1590