Simple source from list of URLs

I have a list of a few hundred URLs that I would like to fetch via rclone and then mount to a folder on my computer. I can format the list however I need to – JSON, XML, CSV, TEXT, etc

Is there an existing module that I can use, or could one be created easily?

What do the URLs point to? Are they pointing to files? In which case you can use rclone copyurl to fetch them.

Or do they point to something else?

They point to files, but I do not want to store the files on my computer because they are large – I just want to mount references to them on my drive so that I can sync them in various apps.

Are the file on drive? So what you are saying is that you want to import them into your drive?

Correct. I just want to take a list of URLs (I can format the list however it needs to be), and mount them to my local filesystem.

It would be worth seeing if https://github.com/odeke-em/drive can do this.

Thanks, but now I’m really confused… the utility looks to be focused in syncing files specifically for Google Drive. I just need to take an arbitrary list of URLs and mount them.

I experimented with using HTTP (https://rclone.org/http/), and created a “fake” directory listing. And that actually did work, however, it seems to only work with files relative to that directory – if I put a fully qualified http:// url in the list it ignores it. I wonder if HTTP can be updated to support fully qualified URLs – or could a new source be added to support a basic list of URLs?

Sorry I misunderstood you, I thought you wanted them on google drive!

Hmm. changing the http backend to work with fully qualified URLs (as an option) might work…

Fancy having a look at that?

Without sounding too lazy I was hoping that someone would have a solution or be able to point me in the right place to make this change. But it seems like there isn’t a whole lot of interest in this so I guess I’ll have to give it a go myself. I’ll report back if I can figure it out.

Great - thanks!

Let me know if you need any help.

1 Like

I wanted to do something similar. (Parsing a website for video files, then mounting those files)

I managed to get the files to mount by creating fake urls on localhost, and having my own http server… Then just using php to redirect the requests at the desired resources. Though, that’s really the dumb way to do it. It was just a quick setup to see how mounting would work with Plex.

I might work on this more too… but I was getting like load times for videos around 4-10 seconds… So I’m not sure if I’m going to continue.

edit: If you’re going to build your own from source (and you’re on Windows), you’ll need to do these steps (for being able to mount).

I did try quickly changing things to get stuff to work, but it really wasn’t intended for this. (as it was made for recursively parsing links from directories)