I use rclone to transfer data between my VPS’s and Backblaze B2. All VPS are Ubuntu 17.04. One VPS is a head for Plex. I mount B2 as read-only on that instance. Everything works except for Plex media scanning. The scanner wants to scan everything on every pass because the mount doesn’t reflect accurate modification times. I can view the correct mod times for both files and folders on Backblaze’s web interface for file browsing. Am I doing something wrong or is this a limitation of rclone and/or Backblaze B2? Is there a solution so that I can fix the Plex Media Scanner? I have tried searching but haven’t found this exact issue.
I just checked my b2. The files have the correct modification times but I believe the problem for Plex may be the directory times. Those are the date/time that you mount the rclone b2 fs. I’d say open a git issue but I’m not sure if b2 has a better way to handle it. Most of the providers directories are really just an object and not really a directory so they don’t really represent something you can store a creation/modification time against. I’m not sure about b2 though specifically. That would be a question for @ncw.
I’d guess if the directory mod times were correct, the scanner wouldn’t need to descend the tree to determine if there were new files… I’m totally guessing though…
I’d guess one way would be to have an enhancement that the directory timestamp represents the max timestamp of the files/folders in a directory but that sounds really expensive to implement as you’d need to fill descend the tree in order to deduce a directories’ timestamp. I took a look at the b2 API and I see nothing about directory attributes…
Thank you. What is strange is that the directory doesn’t show when it was mounted (it uses some complete different date and time) and individual file modification times aren’t right either. I am attaching some images.
When I run “ls -l” it tells me that every folder was last modified on August 30th, 1754. I mounted the file system today.
Here is the correct mod times.
(I am only allowed to attach one image per post, so I have to make multiple posts. I apologize for the clutter.)
Files appear to show the original created on date, not the time it was uploaded to Backblaze B2.
Interesting. Mine does in fact show the time it was mounted for each directory and the correct mod times for all files I checked. I’m using the latest beta if that makes a difference.
Pick a directory and do a stat on it. What does it show?
Attached is the results of stat on my Movies directory. I will update to the beta and see if anything changes.
Thanks. I was just curious if all three times were the same. Try the latest beta? But really even if it behaves like mine does it still won’t fix your Plex scan problem anyway…
I updated to the latest beta. Stat now shows when I mounted the Remote instead of some weird 1754 date.
Created on date would be right for the files b2 doesn’t have the ability to update timestamp so any file update will be a new upload…
Sorry, I should clarify. That is the original date the file was created on a local file system, not the date it was created on B2. I only began using B2 about a week ago. That file was uploaded some time in July 2017 to B2. I understand the B2 limitations, but the date from stat and ls -l doesn’t match the date reflected on the B2 dashboard for its creation (which would be modification time since you can’t rename or move files, only replace).
Ah I see. I think that is expected behavior. Rclone stores it’s sync’d dates in a tag on b2 so they mirror what is on the local filesystem. The b2 gui shows the actual upload dates because it doesn’t read or care about that custom tag.
What you are saying is that you want to see the dates of when the files are uploaded to b2 rather than the original dates. You could put a issue in and see if a flag could be added to list the uploaded dates instead while the synced mod dates would still be used internally in rclone for syncing…
I guess the individual file dates are largely irrelevant for my issue so long as it doesn’t change. Plex will see a file as new and then leave it alone in the future if the modification time is the same. The real issue is directory modification date and time. Plex will trigger a rescan of the entire folder / library if it sees a change at all ie any time I reboot the VPS or remount the Remote it will try and rescan my entire 50K song library for no reason.
Yes… What I do with kodi is I don’t scan at all. Instead I call the an API to scan just a particular folder when I know it is new. I believe Plex had similar ability. I seem to remember others posting a way to do that…
Thank you for your hard work in developing rclone. It works great and I recognize all your efforts. I am just curious about Backblaze B2 for my own knowledge. If Backlaze can display the correct most recent modification time for “folders”* in their web dashboard, is the limitation related to their API? Is there no way to display the same information through rclone?
Note: I realize that Backblaze doesn’t have true folders like a traditional file system.
A bucket holds files. There is no hierarchy of folders, just one long, flat list of file names…
Even though there are no folders, many of the tools that work with files in a bucket act like there are folders. The file browser on the Backblaze web site acts like there are folders, and so does the b2 command-line tool. Under the covers, they both just scan through the flat list of files and pretend.
We recommend that you use “/” to separate folder names, just like you would for files on your computer. (Or just like you would use “” if you use Windows.) That way the tools can figure out the implied folder structure.
Source - https://www.backblaze.com/b2/docs/files.html