Transferring the empty directories also, from google drive to AWS S3 bucket using put_object method supported by S3

What is the problem you are having with rclone?

When I am transferring the drive contents to AWS S3, the copy command is not able to transfer the empty directories. I have tried the --create-empty-src-dirs flag, but it doesn't work to create directories on S3 where reason said was that S3 doesn't support empty directories. I tried using put_object method which properly uploads an empty folder as a file of 0bytes. Is there any way to do this as some flag in rclone?

Run the command 'rclone version' and share the full output of the command.

Are you on the latest version of rclone? You can validate by checking the version listed here: Rclone downloads
--> * os/version: ubuntu 22.04 (64 bit)

  • os/kernel: 5.19.0-1022-aws (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.20.2
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Downloading the files from Google drive and uploading it to AWS S3.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone -P  copy remote_g:opFolder_new --drive-chunk-size 32M remote_s3:drive2cloudbackup/hello2 --create-empty-src-dirs 

The rclone config contents with secrets removed.

Current remotes:

Name                 Type
====                 ====
remote_g             drive
remote_s3            s3

rclone does not support empty directories on S3.
often requested feature and much discussed in the forum.

is this a one time transfer or something to be done on a schedule?

you could write a script that:

  1. for each empty dirs on gdrive, copy a file to it.
  2. rclone copy
  3. delete those files from aws

I think you missed the description. Let me reiterate this for you. I had written a script that can upload an empty folder as a 0-byte object using put_object(Bucket=bucket_name, Key=key). The attached screenshot shows two empty folders which were transferred from google drive to AWS S3 here as 0B objects. This whole thing is getting executed without even the need of copying a test file and then delete it.
Now, my query is if this feature has been programmed into rclone copy command or, if was it missed.

yes, that is possible using script. rclone cannot create an empty dir on s3.

rclone backend features aws:
	"Name": "aws",
	"Root": "",
	"String": "S3 root",
	"Precision": 1,
	"Hashes": [
	"Features": {
		"About": false,
		"BucketBased": true,
		"BucketBasedRootOK": true,
		"CanHaveEmptyDirectories": false,


maybe i am confused. what i see is one folder.
and two objects/files of size zero bytes and storage class as standard.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.