Create mapping output file when migrating between file providers

Help and Support
1

Hi,

I'm sure this feature exists, but I really can't find it, despite my best google-fu.

Currently, my files are in Google Drive but are being indexed using their file IDs in my application. I'd like to move to Google Cloud Storage, but that strips the file IDs from the files - that's fine; I'd like to move to filepath routing anyway, but I need to create a 'look-up table' that maps the file IDs in Google Drive to the filepaths in Google Cloud Storage. Is there a way for rclone to generate these 'lookups' in-flight as the files are being migrated? Even as a CSV with two columns, [FileID, GCSFilepath] would suffice for me to be able to complete the migration.

Is this supported?

Many thanks,

Wiktor

2

hello and welcome to the forum,

not sure about inflight but does this help?
rclone lsf --format="ip" --files-only --absolute -R remote:

which would output

1Oo_pi0CTgA7rCZomvZw-jD0gkAsGXnzn;/folder01/file.ext
1xp494BG0Tu9ukUtt6L0QzK4ELDjF_TJt;/folder01/folder02/file.ext
3

Hey, this is super useful. Would you see this being used before the migration (sync), or after the migration? Ideally, I'd like these generated as we're migrating, but I guess if we can run this after the migration, we should be fine.

Many thanks!

4

good!

and there is https://rclone.org/drive/#copyid

not sure if rclone can do that, i do not use google products much, so others can chime in.
tho, off top of head, could use rclone sync --dump=headers which should contain the ID and parse that.

that would be up to you and your use case.

1 Like
5

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.