Do one thing, Use Rclone move instead of Copy.
move the files from mydrive to teamdrive,
Then try rclone copy from teamdrive to mydrive. (there is no restriction to copy from shared drives to my drive)
So, It Works (takes time and resources, but does online without downloading them)!!
Edit: I checked my theory, Got the same "Sharing restrictions cannot be set on a shared drive item"
at first, then moved my files to shared drive and re copied them to my drive.
IT WORKED!!