Times on files initially set incorrect on mount

What is the problem you are having with rclone?

depending on how a file is created in a rclone mount, the initial time of that file is wrong. It inherits the directory time instead of the actual creation time.

What is your rclone version (output from rclone version)

1.52

Which OS you are using and how many bits (eg Windows 7, 64 bit)

linux 64 amd

Which cloud storage system are you using? (eg Google Drive)

gdrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount robgs:camst /data/camst \
 --allow-other \
 --buffer-size 150M \
 --default-permissions \
 --dir-cache-time 485h \
 --gid $gid --uid $uid \
 --log-level DEBUG \
 --umask $UMASK \
 --use-mmap \
 2>&1

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

root#: cat testcase
mkdir /data/camst/test
sleep 30
cd /data/camst/test
rm -f test wrongtime
touch test
while (true); do find . -type f -exec stat --format %n:%w:%y {} \;  ; sleep 0.1; done&
echo why > wrongtime
sleep 2
kill %1

When I run that against the mount You'll see the times change in the middle to the correct time where as initially it was the same time as the folder.

root# ./testcase
mkdir: cannot create directory '/data/camst/test': File exists
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:12:06.416000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:30:29.454000000 -0400  #HERE
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:30:29.454000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:30:29.454000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:30:29.454000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:30:29.454000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:30:29.454000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:30:29.454000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:30:29.454000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:30:29.454000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:30:29.454000000 -0400
./test:-:2020-06-05 12:30:27.327000000 -0400
./wrongtime:-:2020-06-05 12:30:29.454000000 -0400

I checked older versions and they all work like this back to 1.50. I should note that if a file is created with a different command like 'touch' the times are correct immediately. I assume this is something to do with streaming the file with echo into the mount.

This time difference messes with my scripts that look at times using find. I tried looking at the code but got lost :cry:

This is part of the smoke and mirrors rclone uses to make you think you've got a real file system.

The problem is that rclone doesn't know the modtime for the file until after it had been uploaded, but it has to appear in the directory immediately it is opened.

I haven't checked the code but using the directory time if the time from the object isn't available seems plausible.

We don't know the modtime until after the upload has finished. Rclone could use the current time for an open file. Would that fix your scripts?

I'm not sure. I think so but i'd like to test it and see what that would look like. Can you help me with where this code is as I can't seem to find it.

It is this line you want to change I think

change it to return time.Now() and seem what happens!

Yes that helps my scripts although i'd need to increase the duration its checking just a bit but that's no big deal.

It makes sense to me that if the modtime is unknown to set it to the current time. But if the file is being moved, why couldn't rclone know the time of that file's mtime before uploading?

Also, should both of these be changed?

    if d.vfs.Opt.NoModTime {
            return time.Now()
    }

...
if o == nil {
return time.Now()
}

Good.

It could... It would need to set the pending modtime (though that might break other stuff so it might need to be a different variable)

It say possible but getting tricky!

The no modtime one is probably OK as there will never be a modtime so we don't want it changing continuously.

I see. Thanks. For me Now() would suffice and its a simple change. I'll submit a PR.

1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.