--files-from does not work when copy in --drive-shared-with-me

What is the problem you are having with rclone?

Copy command is working fine with --include-from but it does not work with --files-from

What is your rclone version (output from rclone version)

1.49.5

Which OS you are using and how many bits (eg Windows 7, 64 bit)

ubuntu 64

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone --config /usr/local/settings/rclone.conf copy remote1: /home/content --drive-shared-with-me --files-from=/content/upload.txt --stats-one-line -vv

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

2019/10/08 15:33:16 DEBUG : rclone: Version "v1.49.5-178-geff11b44-beta" starting with parameters ["rclone" "--config" "/usr/local/sessionSettings/rclone.conf" "copy" "mnc:" "/content" "--drive-shared-with-me" "--files-from=/content/upload.txt" "--dry-run" "--stats-one-line" "-vv"]
2019/10/08 15:33:16 DEBUG : Using config file from "/usr/local/sessionSettings/rclone.conf"
2019/10/08 15:33:16 DEBUG : .config: Excluded
2019/10/08 15:33:16 DEBUG : upload.txt: Excluded
2019/10/08 15:33:18 DEBUG : Copy: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/Public.Enemies.2009.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/Quantum of Solace (2008) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/Dr. No (1962) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/Casino Royale (2006) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/Casino Royale (1967) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/Die Another Day (2002) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/For Your Eyes Only (1981) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/From Russia with Love (1963) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/GoldenEye (1995) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/Moonraker (1979) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/Licence to Kill (1989) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/Goldfinger (1964) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/On Her Majesty's Secret Service (1969) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/The World Is Not Enough (1999) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/Never Say Never Again (1983) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/Live and Let Die (1973) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/Octopussy (1983) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/Skyfall (2012) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/Spectre (2015) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/You Only Live Twice (1967) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/Tomorrow Never Dies (1997) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/Thunderball (1965) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/The Spy Who Loved Me (1977) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/The Man with the Golden Gun (1974) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/The Living Daylights (1987) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/Diamonds Are Forever (1971) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/007.James.Bond.Collection.1962-2015.MULTi.1080p.BluRay.REMUX.AVC-GLaDOS/A View to a Kill (1985) MULTi 1080p BluRay REMUX AVC - GLaDOS.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 DEBUG : Copy/Movies/Public.Enemies.2009.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON/Public.Enemies.2009.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv: Excluded from sync (and deletion)
2019/10/08 15:33:18 INFO  : Local file system at /content: Waiting for checks to finish
2019/10/08 15:33:18 INFO  : Local file system at /content: Waiting for transfers to finish
2019/10/08 15:33:18 INFO  :          0 / 0 Bytes, -, 0 Bytes/s, ETA -
2019/10/08 15:33:18 DEBUG : 5 go routines active
2019/10/08 15:33:18 DEBUG : rclone: Version "v1.49.5-178-geff11b44-beta" finishing with parameters ["rclone" "--config" "/usr/local/sessionSettings/rclone.conf" "copy" "mnc:" "/content" "--drive-shared-with-me" "--files-from=/content/upload.txt" "--dry-run" "--stats-one-line" "-vv"]

Replace "--include-from" to "--files-only" works fine.

What is the content of this file - can you post a bit?

Note that --files-from is just a plain list of files rooted at the root of the transfer. Whereas --include-from the file can contain wildcards etc.

The file only contains the text Public.Enemies.2009.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv as I was testing the filter options with --drive-shared-with-me tag. It doesnt copy the file as --include-from or --filter-from does.

The problem I am facing with --filter-from and --include-from is that after transfer the items listed in the upload.txt, rclone keeps searching through my huge tree of shared links, which is like forever. So I try to experiment with --files-from so it might stop after complete the task. Or fo you have any alternative way that might come in handy?

When using the --files-from option, this needs to contain the path, so

dir/subdir/Public.Enemies.2009.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv

The path is from the root of the sync.

The file is in the root of the source, so I think path is not neccessary. I've test it with lsf tag, and it shows correctly the file, however when I replace it with copy it doesnt work.

here is the tests.

# upload.txt
Prison.Break.S05E01.Ogygia.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv
Prison.Break.S05E02.Kaniel.Outis.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv
Prison.Break.S05E03.The.Liar.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv
Prison.Break.S05E04.The.Prisoners.Dilemma.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv
Prison.Break.S05E05.Contingency.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv
Prison.Break.S05E06.Phaeacia.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv
Prison.Break.S05E07.Wine-Dark.Sea.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv
Prison.Break.S05E08.Progeny.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv

rclone lsf mnc: --drive-shared-with-me --files-from upload.txt

$ rclone lsf mnc: --drive-shared-with-me --files-from upload.txt
Prison.Break.S05E01.Ogygia.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv
Prison.Break.S05E02.Kaniel.Outis.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv
Prison.Break.S05E03.The.Liar.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv
Prison.Break.S05E04.The.Prisoners.Dilemma.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv
Prison.Break.S05E05.Contingency.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv
Prison.Break.S05E06.Phaeacia.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv
Prison.Break.S05E07.Wine-Dark.Sea.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv
Prison.Break.S05E08.Progeny.1080p.BluRay.REMUX.AVC.DTS-HD.MA.5.1-EPSiLON.mkv

rclone copy mnc: --drive-shared-with-me --files-from upload.txt ./ --dry-run -vv --stats-one-line

$ rclone copy mnc: --drive-shared-with-me --files-from upload.txt ./ --dry-run -vv --stats-one-line
2019/10/10 14:47:40 DEBUG : rclone: Version "v1.49.5" starting with parameters ["F:\\OneDrive\\System\\rClone\\rclone.exe" "copy" "mnc:" "--drive-shared-with-me" "--files-from" "upload.txt" "./" "--dry-run" "-vv" "--stats-one-line"]
2019/10/10 14:47:40 DEBUG : Using config file from "C:\\Users\\Rhaps\\.config\\rclone\\rclone.conf"
2019/10/10 14:47:40 DEBUG : Bloodstained Ritual of the Night.lnk: Excluded
2019/10/10 14:47:40 DEBUG : CCleaner.lnk: Excluded
2019/10/10 14:47:40 DEBUG : class yenhoa: Excluded
2019/10/10 14:47:40 DEBUG : command.txt: Excluded
2019/10/10 14:47:40 DEBUG : CPUZ.lnk: Excluded
2019/10/10 14:47:40 DEBUG : desktop.ini: Excluded
2019/10/10 14:47:40 DEBUG : EVKey.lnk: Excluded
2019/10/10 14:47:40 DEBUG : Examples: Excluded
2019/10/10 14:47:40 DEBUG : FINAL FANTASY VIII REMASTERED.lnk: Excluded
2019/10/10 14:47:40 DEBUG : foobar2000.lnk: Excluded
2019/10/10 14:47:40 DEBUG : Geart rlab.ipynb: Excluded
2019/10/10 14:47:40 DEBUG : Git.lnk: Excluded
2019/10/10 14:47:40 DEBUG : GPUZ.lnk: Excluded
2019/10/10 14:47:40 DEBUG : HWiNFO64.lnk: Excluded
2019/10/10 14:47:40 DEBUG : Internet Download Manager.lnk: Excluded
2019/10/10 14:47:40 DEBUG : Irfanview.lnk: Excluded
2019/10/10 14:47:40 DEBUG : LOGGING.CSV: Excluded
2019/10/10 14:47:40 DEBUG : Magnet.txt: Excluded
2019/10/10 14:47:40 DEBUG : monarch - Chrome.lnk: Excluded
2019/10/10 14:47:40 DEBUG : MPV.lnk: Excluded
2019/10/10 14:47:40 DEBUG : Mqueue.txt: Excluded
2019/10/10 14:47:40 DEBUG : myapprclone: Excluded
2019/10/10 14:47:40 DEBUG : New Electro & House 2012 Dance Mix #17.aac: Excluded
2019/10/10 14:47:40 DEBUG : New folder: Excluded
2019/10/10 14:47:40 DEBUG : qBittorrent.lnk: Excluded
2019/10/10 14:47:40 DEBUG : RAM.png: Excluded
2019/10/10 14:47:40 DEBUG : Rclone Browser.lnk: Excluded
2019/10/10 14:47:40 DEBUG : rClone Scripts Laptop.bat: Excluded
2019/10/10 14:47:40 DEBUG : rClone Scripts.bat: Excluded
2019/10/10 14:47:40 DEBUG : RLabClone: Excluded
2019/10/10 14:47:40 DEBUG : Security Learning.txt: Excluded
2019/10/10 14:47:40 DEBUG : Steam.lnk: Excluded
2019/10/10 14:47:40 DEBUG : The Witcher III.lnk: Excluded
2019/10/10 14:47:40 DEBUG : Turn off Monitor.lnk: Excluded
2019/10/10 14:47:40 DEBUG : UFMagnet.txt: Excluded
2019/10/10 14:47:40 DEBUG : upload.txt: Excluded
2019/10/10 14:47:40 DEBUG : Visual Studio Code.lnk: Excluded
2019/10/10 14:47:40 DEBUG : XLaunch.lnk: Excluded
2019/10/10 14:47:40 DEBUG : Xming.lnk: Excluded
2019/10/10 14:47:40 DEBUG : Yamaha Fes: Excluded
2019/10/10 14:47:43 DEBUG : Copy: Excluded from sync (and deletion)
2019/10/10 14:47:43 DEBUG : Colab Notebooks: Excluded from sync (and deletion)
2019/10/10 14:47:43 DEBUG : Colab Notebooks/MNC 19/09: Excluded from sync (and deletion)
2019/10/10 14:47:43 DEBUG : Colab Notebooks/MNC 08/09: Excluded from sync (and deletion)
2019/10/10 14:47:43 DEBUG : Colab Notebooks/blackpeal_template.ipynb: Excluded from sync (and deletion)
2019/10/10 14:47:43 DEBUG : Colab Notebooks/MNC 05/09: Excluded from sync (and deletion)
2019/10/10 14:47:43 DEBUG : Copy/Anime: Excluded from sync (and deletion)
2019/10/10 14:47:43 DEBUG : Copy/Movies: Excluded from sync (and deletion)
2019/10/10 14:47:43 DEBUG : Copy/TV Shows: Excluded from sync (and deletion)
2019/10/10 14:47:43 DEBUG : Copy/18+: Excluded from sync (and deletion)
2019/10/10 14:47:43 INFO  : Local file system at \\?\F:\OneDrive\DSKDesktop: Waiting for checks to finish
2019/10/10 14:47:43 INFO  : Local file system at \\?\F:\OneDrive\DSKDesktop: Waiting for transfers to finish
2019/10/10 14:47:43 INFO  :          0 / 0 Bytes, -, 0 Bytes/s, ETA -
2019/10/10 14:47:43 DEBUG : 11 go routines active
2019/10/10 14:47:43 DEBUG : rclone: Version "v1.49.5" finishing with parameters ["F:\\OneDrive\\System\\rClone\\rclone.exe" "copy" "mnc:" "--drive-shared-with-me" "--files-from" "upload.txt" "./" "--dry-run" "-vv" "--stats-one-line"]

Hmm, this is a bug.

You can work-around with --no-traverse.

I bisected this problem down to this commit

Can you please make a new issue on github about this and I will investigate further. Put a link to this page in please!

I have submited an issue on github through this link. https://github.com/rclone/rclone/issues/3619

1 Like