How to Purge A Directory After X days

Use delete and rmdir but it doesnt' help because each directory have sub folder inside (although sub folder is empty)
It's : My Drive > HOCVPS_BACKUP > YYYY-MM-DD > mysql (or nginx)
Level 1: HOCVPS_BACKUP
Level 2: Folder backup for each day, YYYY-MM-DD
Level 3: File zip compress site, MySQL folder, Nginx folder
Level 4: mysql backup file in MySQL folder and Nginx conf file in Nginx folder
So, delete help to delete file zip compress site, mysql backup file, nginx config file
But rmdir cannot remove YYYY-MM-DD cause YYYY-MM-DD have 2 empty folder inside
Here is my code:

/usr/sbin/rclone -q --min-age 2d delete "remote:$SERVER_NAME"
/usr/sbin/rclone -q --min-age 2d rmdirs "remote:$SERVER_NAME"

It saids:

2017/01/16 16:09:15 Failed to rmdir: directory not empty: *drive.ChildReference{(*drive.ChildReference)(0xc4201a6770), (*drive.ChildReference)(0xc4201a6850), (*drive.ChildReference)(0xc4201a6930), (*drive.ChildReference)(0xc4201a6a10), (*drive.ChildReference)(0xc4201a6af0), (*drive.ChildReference)(0xc4201a6bd0)}