rclone does not support modtime for directories. has been discussed in the forum, many times.
https://github.com/rclone/rclone/issues/1669