Transferring the empty directories also, from google drive to AWS S3 bucket using put_object method supported by S3

[Kartik_Jhawar]

What is the problem you are having with rclone?

When I am transferring the drive contents to AWS S3, the copy command is not able to transfer the empty directories. I have tried the --create-empty-src-dirs flag, but it doesn't work to create directories on S3 where reason said was that S3 doesn't support empty directories. I tried using put_object method which properly uploads an empty folder as a file of 0bytes. Is there any way to do this as some flag in rclone?

Run the command 'rclone version' and share the full output of the command.

Are you on the latest version of rclone? You can validate by checking the version listed here: Rclone downloads
--> * os/version: ubuntu 22.04 (64 bit)

• os/kernel: 5.19.0-1022-aws (x86_64)
• os/type: linux
• os/arch: amd64
• go/version: go1.20.2
• go/linking: static
• go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Downloading the files from Google drive and uploading it to AWS S3.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone -P  copy remote_g:opFolder_new --drive-chunk-size 32M remote_s3:drive2cloudbackup/hello2 --create-empty-src-dirs 

The rclone config contents with secrets removed.

Current remotes:

Name                 Type
====                 ====
remote_g             drive
remote_s3            s3

[asdffdsa (jojothehumanmonkey)]

rclone does not support empty directories on S3.
often requested feature and much discussed in the forum.

https://forum.rclone.org/t/how-to-rclone-copy-empty-folders-directories/14081

https://github.com/rclone/rclone/issues/4423

is this a one time transfer or something to be done on a schedule?

you could write a script that:

1. for each empty dirs on gdrive, copy a file to it.
2. rclone copy
3. delete those files from aws

[Kartik_Jhawar]

I think you missed the description. Let me reiterate this for you. I had written a script that can upload an empty folder as a 0-byte object using put_object(Bucket=bucket_name, Key=key). The attached screenshot shows two empty folders which were transferred from google drive to AWS S3 here as 0B objects. This whole thing is getting executed without even the need of copying a test file and then delete it.
Now, my query is if this feature has been programmed into rclone copy command or, if was it missed.

[asdffdsa (jojothehumanmonkey)]

@Kartik_Jhawar

I had written a script that can upload an empty folder as a 0-byte object using put_object(Bucket=bucket_name, Key=key)

yes, that is possible using script. rclone cannot create an empty dir on s3.

https://forum.rclone.org/t/create-empty-directories/32297

@ncw

Not currently, no. You'll have to put a file in.

---

How to Rclone Copy Empty Folders / Directories

/usr/bin/rclone copy testDrive: s3-rclone:/blah.blah.com/ --create-empty-src-dirs That is the command I am trying to run. From GSuite/Shared Drive to S3 bucket. I copy the shared drive with empty folders but do not see the folders in s3 once rclone copy is executed. Let me know if my command is correct or if this is even possible Edit: rclone v1.50.2 os/arch: linux/amd64 go version: go1.13.4

--create-empty-src-dirs not working for S3

---

rclone backend features aws:
{
	"Name": "aws",
	"Root": "",
	"String": "S3 root",
	"Precision": 1,
	"Hashes": [
		"md5"
	],
	"Features": {
		"About": false,
		"BucketBased": true,
		"BucketBasedRootOK": true,
		"CanHaveEmptyDirectories": false,

[asdffdsa (jojothehumanmonkey)]

@Kartik_Jhawar

The attached screenshot shows two empty folders

maybe i am confused. what i see is one folder.
and two objects/files of size zero bytes and storage class as standard.