Public GCS bucket and "Entry doesn't belong in directory "" (same as directory)"

Help and Support
1

What is the problem you are having with rclone?

Hi, I am getting a quite confusing message

: Entry doesn't belong in directory "" (same as directory) - ignoring

when listing a public bucket on Google Cloud Storage.

From other threads mentioning this message it seems like this is sort of a "bug" that is successively being removed for different storage providers / use cases.

Is there something to be worried about it here? Can I do something to get rid of it?

What is your rclone version (output from rclone version)

Present both in

rclone v1.53.3
- os/arch: linux/amd64
- go version: go1.15.5

as well as in following beta

rclone v1.54.0-beta.5058.35a4de203
- os/arch: linux/amd64
- go version: go1.15.6

Which OS you are using and how many bits (eg Windows 7, 64 bit)

KDE Neon (20.04, 64 bit)

Which cloud storage system are you using? (eg Google Drive)

Google Cloud Storage, public bucket, anonymous

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone lsd gcs:seldon-models/sklearn

also present when copying, e.g.

rclone copy gcs:seldon-models/sklearn/iris/ iris/

The rclone config contents with secrets removed.

[gcs]
type = google cloud storage
anonymous = true

A log from the command with the -vv flag

2021/01/18 16:44:52 DEBUG : rclone: Version "v1.53.3" starting with parameters ["/home/rskolasinski/.asdf/installs/rclone/1.53.3/bin/rclone" "lsd" "gcs:seldon-models/sklearn" "-vv"]
2021/01/18 16:44:52 DEBUG : Using config file from "/home/rskolasinski/.config/rclone/rclone.conf"
2021/01/18 16:44:52 DEBUG : Creating backend with remote "gcs:seldon-models/sklearn"
2021/01/18 16:44:52 ERROR : : Entry doesn't belong in directory "" (same as directory) - ignoring
           0 2021-01-18 16:44:52        -1 adult_shap
           0 2021-01-18 16:44:52        -1 income
           0 2021-01-18 16:44:52        -1 iris
           0 2021-01-18 16:44:52        -1 iris-0.23.2
           0 2021-01-18 16:44:52        -1 moviesentiment
           0 2021-01-18 16:44:52        -1 wine
2021/01/18 16:44:52 DEBUG : 3 go routines active
2

It is probably a 0 length directory marker so I think it is safe to ignore.

3

Are there any plans to detect such scenarios and silence the warning error?

4

Hmm, I just looked at the code.

We do do that already.

Can you post the result of this and I'll look through the directory listing and work out what is really there:

rclone lsd gcs:seldon-models/sklearn -vv --dump bodies

Thanks

5

Sure, here it comes

$ rclone lsd gcs:seldon-models/sklearn -vv --dump bodies
2021/01/19 14:31:04 DEBUG : rclone: Version "v1.53.3" starting with parameters ["/home/rskolasinski/.asdf/installs/rclone/1.53.3/bin/rclone" "lsd" "gcs:seldon-models/sklearn" "-vv" "--dump" "bodies"]
2021/01/19 14:31:04 DEBUG : Using config file from "/home/rskolasinski/.config/rclone/rclone.conf"
2021/01/19 14:31:04 DEBUG : Creating backend with remote "gcs:seldon-models/sklearn"
2021/01/19 14:31:04 ERROR : : Entry doesn't belong in directory "" (same as directory) - ignoring
           0 2021-01-19 14:31:04        -1 adult_shap
           0 2021-01-19 14:31:04        -1 income
           0 2021-01-19 14:31:04        -1 iris
           0 2021-01-19 14:31:04        -1 iris-0.23.2
           0 2021-01-19 14:31:04        -1 moviesentiment
           0 2021-01-19 14:31:04        -1 wine
2021/01/19 14:31:04 DEBUG : 3 go routines active

BTW, this is a public bucket so you should be able to see these resources

6

Hmm that should have printed stuff - I see a bug in the anonymous gcs code!

Useful!

{
  "kind": "storage#objects",
  "prefixes": [
    "sklearn/adult_shap/",
    "sklearn/income/",
    "sklearn/iris-0.23.2/",
    "sklearn/iris/",
    "sklearn/moviesentiment/",
    "sklearn/wine/"
  ],
  "items": [
    {
      "kind": "storage#object",
      "id": "seldon-models/sklearn//1563134000900850",
      "selfLink": "https://www.googleapis.com/storage/v1/b/seldon-models/o/sklearn%2F",
      "mediaLink": "https://storage.googleapis.com/download/storage/v1/b/seldon-models/o/sklearn%2F?generation=1563134000900850&alt=media",
      "name": "sklearn/",
      "bucket": "seldon-models",
      "generation": "1563134000900850",
      "metageneration": "1",
      "contentType": "text/plain",
      "storageClass": "MULTI_REGIONAL",
      "size": "11",
      "md5Hash": "apnFdauH+MfR7R5S5+NJzg==",
      "crc32c": "XkI+Dw==",
      "etag": "CPLV8PaXteMCEAE=",
      "eventBasedHold": false,
      "timeCreated": "2019-07-14T19:53:20.900Z",
      "updated": "2019-07-14T19:53:20.900Z",
      "timeStorageClassUpdated": "2019-07-14T19:53:20.900Z"
    }
  ]
}

The problem is that this has an object called "sklearn/" - rclone would normally ignore this if it was 0 length.

It has the contents placeholder!

I can widen that check easily enough

Try this

v1.54.0-beta.5063.fac740fc0.fix-gcs-dir on branch fix-gcs-dir (uploaded in 15-30 mins)

7

Thanks @ncw! That was really fast.
I just tried the beta build you linked and it works well! No error anymore :wink:

8

Thanks for testing :slight_smile:

I've merged this to master now which means it will be in the latest beta in 15-30 mins and released in v1.54

9

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.