Help with understanding Rclone and Google Drive

Hi,
I am making a back up from my pc and since the internet google drive is really slow in uploading, I have set up RClone to do the trick. Now, I'm a little bit of a noob with script etc. So I have noticed that if i put big mp4 files in my Rclone folder, they take some time to show up the webbrowser Google Drive and I was wondering why this is.

If I put a big file in my Rclone folder with Windows Explorer, the standard Windows Explorer popup shows up and he copies it with around 100-200 mb's and then it takes sometime to show up in the webbrowser Google Drive but the files are there in my Windows Explorer Rclone Google Drive. This is what I found to be weird since I thought that Rclone shows the files that are in your Drive.

So if the files are in your Drive, why doesnt Google show it to you on the web browser? I thought that maybe the files had to be processed or something, so I put a lot of big files in Rclone and when it was finished transferring, they were indeed in my Google Drive according to Rclone. I closed cmd to test this theory and went to work, 4 hours later I checked my Google Drive on Firefox and they weren't there. Then I booted up Rclone and the files were there.
I waited some minutes and then the files started slowly showing up in Google Drive like usual.
But if they are stored in the Google Drive according to Rclone but they aren't there according to Google themselves, then where are the files stored? And why do they only show up if I have Rclone on?

This is probably a dumb question but I was just wondering, also sorry if I made English mistakes since it is not my first language.

I am on the latest version of Rclone and I am on Windows Pro 10 64 bits, the Google Drive is unlimited storage since I have a school account with an edu email. I think the account is part of the Google Workspace or G-Suite but I don't know for sure.

What is your rclone version (output from rclone version)

1.54.0

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount --vfs-cache-mode full gdrive:/ g:


#### The rclone config contents with secrets removed.
Full Access, no advanced config and auto config, no team folder and the rest default.

hello and welcome to the forum,
when you posted, there should have been a template of questions.
if you answer them, we can better help you.
since this is your first time posting, that you are a new user, i have re-posted some of the questions that you did not answer.


STOP and READ YOUR POST WILL BE REMOVED IF IT IS LOW QUALITY:
Please show the effort you've put in to solving the problem and please be specific -- people are volunteering their time to help you! Low effort posts are not likely to get good answers! DO NOT REDACT any information except passwords/keys/personal info. You should use 3 backticks to begin and end your paste to make it readable. Or use a service such as https://pastebin.com or https://gist.github.com/

What is your rclone version (output from rclone version)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Paste command here

The rclone config contents with secrets removed.

Paste config here

A log from the command with the -vv flag

Paste  log here

I think I did it right now but I didn't understand the last one.

My guess is that what you're seeing as really fast is the copy to the cache folder. From there, rclone then uploads it to the remote but as far as Explorer is concerned, it has been copied.

You may see something different if you didn't use --vfs-cache-mode full but you lose resiliency if the upload needs to be restarted (which does happen).

Is there a reason you have to do this in a mounted drive and not just using the command line with copy or sync? Those are more robust and reliable for uploads

ok. that is helpful.
i also use rclone on windows.

if you are doing a backup, i would use rclone sync, not rclone mount
if you want to have an both full backup and an incremental backup, you can use this flag.
https://rclone.org/docs/#backup-dir-dir

if you can better describe what kind of backup you need, we can help with the rclone command, to get that done.

I was'nt trying to fully back up my computer but I have some movies and series on my computer that take up a lot of space so I was trying to upload them.
I didn't know that there were more reliable ways for uploading to I will try the copy command line.
I just did ctrl+c and ctrl+v in the Explorer map because I saw someone doing this in a Youtube tutorial.

ok,

that is a common use of rclone, to stream media from gdrive.

once you start to upload a file, it will take time.
it depends on your internet connection, upload speed.
you can test that at https://www.speedtest.net/.
perhaps run the test and post the results.

Alright, my ping is 4ms, my download speed is 191.77 mbps and my upload speed is 245.50 mbps. Pretty fast right?

sure, pretty fast.

@jwink3101 is correct and well written.

another more complex option is to have two or more mounts, which is what i do.
one mount for streaming media.
rclone mount --vfs-cache-mode full gdrive: g:

one mount without the vfs, for uploading files. copy the files to gdrive using h:, not g:
rclone mount gdrive: h:
once the file is uploaded, it is really uploaded and will appear in gdrive website without delay.

Thanks for the help, so do you have 2 cmd windows open at the same time?

yes, but there are ways around that.

for now, one way or another get the media to gdrive.
just keep in mind, that windows explorer might seem to copy the file quickly.
in fact, rclone, in the background, will upload the file from the rclone cache at your internet upload speeds.
so do not close the cmd window until you see the files in gdrive website.

Alright thanks for all the help guys!

Out of curiosity, why do this? It seems like there are very read disadvantages to uploading via mount or serve. Off the top of my head:

  • Retries if something break
  • Must be uploaded sequentially. (my understanding is that on most remotes that do multi-part upload, they are done concurrently)
  • Lack of VFS refresh on your other remote (this is an issue however you chose to do it I guess)

well, just giving options to a newbie rcloner based on the perceived OP skill set.

imho, it works well, it is reliable and simple to understand, and the OP can know when a file has been really uploaded to gdrive, one of the OP main concerns and source of confusion.

perhaps, over time, as the OP learns about rclone, command line and scripting, then rclone sync/copy/move will be the ultimate solution.

i do not upload much media, so the vfs refresh is not an issue.
i have a little script that will do a vfs/refresh via rc, and trigger jellyfin to do a quick media scan.

When you said this, I already figured out how to do the copy command but I just wanted to know what jojothehumanmonkey had to say.
For larger files, I'm using the copy command now and I use the mount command to get easy acces to all my files.

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.