How sync files (even few) modified

What is the problem you are having with rclone?

rclone doesn't sync my music files, even modified, unless they are "heavily" modified. I mean, if if I change the lyric or metadata and only few bytes are added, rclone doesn't sync that files.

Run the command 'rclone version' and share the full output of the command.

rclone v1.53.1

  • os/arch: linux/amd64
  • go version: go1.15

Which cloud storage system are you using? (eg Google Drive)

Mega

The command you were trying to run (eg rclone copy /tmp remote:tmp)

The rclone config contents with secrets removed.

[mega]
type = mega

[link]
type = local
links = true

[nolink]
type = local

A log from the command with the -vv flag

I'm not on my main pc, so I cannot provide the request log

hello,
that version of rclone is years old, please update to v1.59.1

and need to see a debug log

Thank you, I will do it. So, you are saying that that behavior is not expected (it is a bug)?
I mean you are saying that a file changed should be sync even with very small changes?

that is correct.

"Sync the source to the destination, changing the destination only. Doesn't transfer files that are identical on source and destination, testing by size and modification time or MD5SUM"

ok, thank you. It is therefore a strange thing (but not the first time) that a music file "only slightly" modified is not synced.
Should I open a bug report (with all the requested data now I don't have here)?

no, zero evidence of a bug

1 Like

This morning I have had yet another proof that small changes are ignored by rclone: several music files changed today haven't been sync.
To provide you a complete documentation what steps should I do?
What syntax should I use to say to rclone to sync any file by date, regardless to its size?

  1. update rclone to latest stable, v1.59.1
  2. run the rclone sync with --dry-run and -vv
  3. find a source file that should have been sync to the dest, but was not sync, let's call it file.ext
  4. provide the output of these commands, make sure to change the dir/file paths to match your setup.
    a. rclone lsf /path/to/source/file.ext --format="hsp" for the source file.
    b. rclone lsf /path/to/dest/file.ext --format="htsp" for the dest file.

https://rclone.org/docs/#ignore-size

1 Like

Thank you. I will follow your instructions (this evening or tomorrow).
But I noticed that the problem is only with musica (mega) folder and not with ebooks (mega) folder: ebooks are synced perfectly. And the curious is that the command I use is the same...

good, that narrows down the problem to one folder.

the goal is to find a single source file that should be synced but does not get synced.
and then use that file in step 4.

1 Like

ok, I will do it.

Here you are:

  1. output of sync with --dry-run and -vv
2022/09/03 06:30:14 DEBUG : rclone: Version "v1.59.1" starting with parameters ["rclone" "sync" "/mnt/musica" "mega:musica" "-l" "--dry-run" "-vv"]
2022/09/03 06:30:14 DEBUG : Creating backend with remote "/mnt/musica"
2022/09/03 06:30:14 DEBUG : local: detected overridden config - adding "{b6816}" suffix to name
2022/09/03 06:30:14 DEBUG : Using config file from "/home/duns/.config/rclone/rclone.conf"
2022/09/03 06:30:14 DEBUG : fs cache: renaming cache item "/mnt/musica" to be canonical "local{b6816}:/mnt/musica"
2022/09/03 06:30:14 DEBUG : Creating backend with remote "mega:musica"
2022/09/03 06:30:19 DEBUG : .directory: Sizes identical
2022/09/03 06:30:19 DEBUG : .directory: Unchanged skipping
(...)
2022/09/03 06:30:19 DEBUG : classica1/von Weber/von Weber, sonata per pianoforte n.2, in la bem mag, op.39, 1.mp3: Sizes identical
2022/09/03 06:30:19 DEBUG : classica1/von Weber/von Weber, sonata per pianoforte n.2, in la bem mag, op.39, 1.mp3: Unchanged skipping
2022/09/03 06:30:19 DEBUG : classica1/von Weber/von Weber, sonata per pianoforte n.2, in la bem mag, op.39, 2.mp3: Sizes identical
2022/09/03 06:30:19 DEBUG : classica1/von Weber/von Weber, sonata per pianoforte n.2, in la bem mag, op.39, 2.mp3: Unchanged skipping
2022/09/03 06:30:19 DEBUG : classica1/von Weber/von Weber, il franco cacciatore, ouverture, 2.mp3: Sizes identical
2022/09/03 06:30:19 DEBUG : classica1/von Weber/von Weber, il franco cacciatore, ouverture, 2.mp3: Unchanged skipping
2022/09/03 06:30:19 DEBUG : leggera/english-speakers singers/Sinatra/New york, new york.mp3: Sizes identical
2022/09/03 06:30:19 DEBUG : leggera/english-speakers singers/Sinatra/New york, new york.mp3: Unchanged skipping
2022/09/03 06:30:19 DEBUG : leggera/english-speakers singers/Sinatra/My Way.mp3: Unchanged skipping
2022/09/03 06:30:19 DEBUG : classica1/Haydn/sinfonie/sinf_092/Haydn, sinfonia n.92, 1.mp3: Unchanged skipping
2022/09/03 06:30:19 DEBUG : mega root 'musica': Waiting for transfers to finish
2022/09/03 06:30:19 DEBUG : Waiting for deletions to finish
2022/09/03 06:30:19 INFO  : There was nothing to transfer
2022/09/03 06:30:19 NOTICE: 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Checks:              6962 / 6962, 100%
Elapsed time:         4.6s

2022/09/03 06:30:19 DEBUG : 8 go routines active

I've cut many identical cases.

  1. rclone lsf /path/to/source/file.ext --format="hsp" for the source file.
rclone lsf file.ext --format="hsp"
85db645fc34c7497b2ff5f4a1eaa4b74;4321522;file.ext
  1. rclone lsf /path/to/dest/file.ext --format="htsp" for the dest file:
rclone lsf file.ext --format="htsp"
05a178f45662ebc0cc99ec91b3cff863;2022-08-25 13:47:38;4321522;file.ext

sorry, my bad, that --format value is missing t

for the source file, please post the output of
rclone lsf /path/to/source/file.ext --format="htsp"

1 Like

Here the output (for the source file)

rclone lsf file.ext --format="htsp"
85db645fc34c7497b2ff5f4a1eaa4b74;2022-08-27 17:08:03;4321522;file.ext

the destination one was:

rclone lsf file.ext --format="htsp"
05a178f45662ebc0cc99ec91b3cff863;2022-08-25 13:47:38;4321522;file.ext

good output,

based on that, i agree with you,
rclone sync should copy/overwrite the source file to the dest.

let's try a couple commands
rclone copy -vv --dry-run /path/to/source/file.ext /path/to/dest
rclone check -vv /path/to/source/file.ext /path/to/dest


i assume that you have updated rclone......

1 Like

Yes I have updated.

rclone copy -vv --dry-run "[1971] Anche per te.mp3" mega:musica/leggera/cantautori/Battisti
2022/09/03 20:55:34 DEBUG : rclone: Version "v1.59.1" starting with parameters ["rclone" "copy" "-vv" "--dry-run" "[1971] Anche per te.mp3" "mega:musica/leggera/cantautori/Battisti"]
2022/09/03 20:55:34 DEBUG : Creating backend with remote "[1971] Anche per te.mp3"
2022/09/03 20:55:34 DEBUG : Using config file from "/home/duns/.config/rclone/rclone.conf"
2022/09/03 20:55:34 DEBUG : fs cache: adding new entry for parent of "[1971] Anche per te.mp3", "/mnt/musica/leggera/cantautori/Battisti"
2022/09/03 20:55:34 DEBUG : Creating backend with remote "mega:musica/leggera/cantautori/Battisti"
2022/09/03 20:55:58 DEBUG : [1971] Anche per te.mp3: Sizes identical
2022/09/03 20:55:58 DEBUG : [1971] Anche per te.mp3: Unchanged skipping
2022/09/03 20:55:58 NOTICE: 
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:                 1 / 1, 100%
Elapsed time:        24.2s

2022/09/03 20:55:58 DEBUG : 9 go routines active

Mega doesn't support timestamp or hashes so if the sizes are the same, it will skip.

That's documented here:

Mega (rclone.org)

1 Like

Help me to understand correctly: I have to avoid [ and ] in filenames?

Rename them? I'm not sure what you are asking.

1 Like

I am confused: to sum up, what should I do to avoid the problem (lack of sync)?