That's a 'sync' from source to destination and that's how it works already.
in this case, what's keeping rclone from deleting a file that's not in the source?
That's been answered in the other post:
If you have a directory and something else dropped something there, rclone isn't going to delete because it's not empty and something else dropped a file in there i.e your tmp files.
That's so rclone doesn't destroy something it wasn't instructed to do.
yeah. manually uploading a random .pdf file into a crypt remote using a browser results in the following:
2022/06/30 14:03:14 DEBUG : A-AF31-201635-TT/OE-28275.pdf: Skipping undecryptable file name: illegal base32 data at input byte 2
i don't see anything about the temp files in the logs for issue directories, but i assume it's a similar situation.
well, i'll leave it to you guys. for now, i'll just use an rclone filter to sweep the directory for temp files and delete them. cheers.
jared
That error means you are mixing a crypt remote and non crypted remote and data is mixed together.
You'd have to share more details.
well yeah. my thinking is rclone has the same reaction to the temp files, because they were put there by something other than rclone's crypt.
What else would be writing data into the crypt location?
that's the million dollar question here. we are sir walter raleigh and the source of those temp files is el dorado. you are a developer at rclone. i highly assume if rclone's crypt was depositing ~tmp files by standard procedure you'd know about it.
I'm not a developer. I don't work for rclone.
Rclone doesn't make "tmp" files and drop them in your crypt.
Rclone only copies what you ask it so I really don't know why you'd have non crypt files in your crypt location as that's not rclone doing that.
you certainly contribute a lot 
all i can think of is the inner workings of sharepoint then. and as i mentioned, i'm not familiar with the inner plumbing over there. it must be something about how they process files through microsoft graph.
Your example though isn't about a temp file.
That's looks to be a normal PDF file that is in your crypt remote location.
You'd want to figure out how you are mixing things up to get that in there and fix what is causing it.
hi jared, hope all is well with that super server you are building.
https://forum.rclone.org/t/tmp-files-on-sharepoint-remote/25385
whenever i backup on windows, using rclone, 7zip, fastcopy, i always use VSS as the source.
in the past, i used a simple four line .cmd script to create a temporary snapshot
as detailed at
https://forum.rclone.org/t/i-want-to-share-how-to-enable-vss-for-rclone/11644
and since then i have moved on to a 140+ lines of python code
create a permanent snapshot, mount that snapshot as a local dir, backup from that snapshot and then delete that snapshot.
i know. because as previously mentioned, there is no log of the actual ~tmp file. however, below is a log of the occasion. what happens is, the temp files get created over time in directories. once the directories are deleted from natural causes on the file server, rclone dumps the files but gets stuck on the temp file and therefore can't delete the directory.
not good. got a BIOS issue that i'm waiting for a reply on from the company 
it's the last step though and i'm free of the harbor and sailing
jeepers. that's pretty extensive.
2022/06/30 15:35:13 DEBUG : rclone: Version "v1.58.0" starting with parameters ["rclone" "sync" "--verbose" "-vvv" "--log-file=TBAccount.txt" "E:\\TBAccount\\Costech Update 08-07-2013\\Website\\App_Themes\\Computer\\images" "OneDriveWyEncryptednew:TBAccount\\Costech Update 08-07-2013\\Website\\App_Themes\\Computer\\images"]
2022/06/30 15:35:13 DEBUG : Creating backend with remote "E:\\TBAccount\\Costech Update 08-07-2013\\Website\\App_Themes\\Computer\\images"
2022/06/30 15:35:13 DEBUG : Using config file from "C:\\Users\\Administrator\\AppData\\Roaming\\rclone\\rclone.conf"
2022/06/30 15:35:13 DEBUG : fs cache: renaming cache item "E:\\TBAccount\\Costech Update 08-07-2013\\Website\\App_Themes\\Computer\\images" to be canonical "//?/E:/TBAccount/Costech Update 08-07-2013/Website/App_Themes/Computer/images"
2022/06/30 15:35:13 DEBUG : Creating backend with remote "OneDriveWyEncryptednew:TBAccount\\Costech Update 08-07-2013\\Website\\App_Themes\\Computer\\images"
2022/06/30 15:35:14 DEBUG : Creating backend with remote "OneDriveWy:APT Server/TBAccount/Costech Update 08-07-2013/Website/App_Themes/Computer/v0ancu2jmn60gds35k1237mdoc"
2022/06/30 15:35:14 DEBUG : One drive root 'APT Server/TBAccount/Costech Update 08-07-2013/Website/App_Themes/Computer/v0ancu2jmn60gds35k1237mdoc': Token expired but no uploads in progress - doing nothing
2022/06/30 15:35:14 DEBUG : OneDriveWy: Loaded invalid token from config file - ignoring
2022/06/30 15:35:15 DEBUG : Saving config "token" in section "OneDriveWy" of the config file
2022/06/30 15:35:15 DEBUG : Keeping previous permissions for config file: -rw-rw-rw-
2022/06/30 15:35:15 DEBUG : OneDriveWy: Saved new token in config file
2022/06/30 15:35:21 DEBUG : Creating backend with remote "OneDriveWy:APT Server/TBAccount/Costech Update 08-07-2013/Website/App_Themes/Computer/images"
2022/06/30 15:35:24 DEBUG : fs cache: renaming cache item "OneDriveWyEncryptednew:TBAccount\\Costech Update 08-07-2013\\Website\\App_Themes\\Computer\\images" to be canonical "OneDriveWyEncryptednew:TBAccount/Costech Update 08-07-2013/Website/App_Themes/Computer/images"
2022/06/30 15:35:25 DEBUG : bottom_bg.jpg: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : bottom_bg.jpg: Unchanged skipping
2022/06/30 15:35:25 DEBUG : button_bg.jpg: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : search_btn_hover2.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : search_btn_hover2.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : no-icon-available.png: Unchanged skipping
2022/06/30 15:35:25 DEBUG : shortcuts_bg.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : shortcuts_bg.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : sort-order-descending.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : top-nav-bg.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : top-nav-bg.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : up.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : up.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : Icons/~tmp94_prboap0fdamm43ms9m89n9l77g: Skipping undecryptable file name: illegal base32 data at input byte 0
2022/06/30 15:35:25 DEBUG : PaymentInstruments/checkhelp.jpg: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : PaymentInstruments/cvv_amex_m.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : PaymentInstruments/cvv_amex_m.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : PaymentInstruments/checkhelp.jpg: Unchanged skipping
2022/06/30 15:35:25 DEBUG : PaymentInstruments/cid_amex.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : PaymentInstruments/cid_amex.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : PaymentInstruments/cid.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : PaymentInstruments/cid.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : PaymentInstruments/cvv_amex.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : PaymentInstruments/cvv_amex.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : PaymentInstruments/checkhelp_m.jpg: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : PaymentInstruments/checkhelp_m.jpg: Unchanged skipping
2022/06/30 15:35:25 DEBUG : PaymentInstruments/cvv_visa.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : PaymentInstruments/cvv_visa.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : PaymentInstruments/cvv_visa_m.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : PaymentInstruments/cvv_visa_m.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : ratings/rate_00.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : ratings/rate_00.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : ratings/rate_05.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : ratings/rate_01.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : ratings/rate_01.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : ratings/rate_09.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : ratings/rate_03.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : ratings/rate_03.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : ratings/rate_06.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : ratings/rate_06.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : ratings/rate_02.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : ratings/rate_02.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : ratings/rate_04.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : ratings/rate_04.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : ratings/rate_08.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : ratings/rate_08.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : ratings/rate_05.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : Encrypted drive 'OneDriveWyEncryptednew:TBAccount/Costech Update 08-07-2013/Website/App_Themes/Computer/images': Waiting for checks to finish
2022/06/30 15:35:25 DEBUG : ratings/rate_07.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : ratings/rate_07.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : ratings/rate_09.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : ratings/rate_10.gif: Size and modification time the same (differ by 0s, within tolerance 1s)
2022/06/30 15:35:25 DEBUG : ratings/rate_10.gif: Unchanged skipping
2022/06/30 15:35:25 DEBUG : Encrypted drive 'OneDriveWyEncryptednew:TBAccount/Costech Update 08-07-2013/Website/App_Themes/Computer/images': Waiting for transfers to finish
2022/06/30 15:35:25 DEBUG : Waiting for deletions to finish
2022/06/30 15:35:25 INFO : Icons: Removing directory
2022/06/30 15:35:26 DEBUG : Icons: Failed to Rmdir: directory not empty
2022/06/30 15:35:26 DEBUG : Encrypted drive 'OneDriveWyEncryptednew:TBAccount/Costech Update 08-07-2013/Website/App_Themes/Computer/images': failed to delete 1 directories
2022/06/30 15:35:26 INFO : There was nothing to transfer
2022/06/30 15:35:26 INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Checks: 59 / 59, 100%
Deleted: 0 (files), 1 (dirs)
Elapsed time: 12.1s
2022/06/30 15:35:26 DEBUG : 6 go routines active
That's what I'm trying to get at though, rclone isn't doing that as that's not a function of rclone.
That's not a function of Sharepoint/Onedrive either "naturally". You'd have some other app writing into the area.
If rclone was writing it, you'd get a log entry somewhere as rclone is great, but doesn't make up files
Cant' you find that file and see what created it? There should be an audit trail of the file / owner / etc. I can see all that stuff when I look at OneDrive or Sharepoint and you can see the audit logs. The compliance portal I believe is where you can look but I'm not sure what level of access you have.
You should be able to see the file and who created it at a minimum.
I think those ~tmp files might be something to do with failed mulipart uploads on OneDrive.
I think there might be more forum posts or issues about it but I'm on the move at the moment and can't search.
i wish i was on the move right now. to some beach somewhere.
you mean if rclone has something like a "upload session not found" error or something? cause i do see that on occasion.
But
That's not a tmp file and is being mixed in. That's not rclone doing that unless the OP is copying to the non encrypted remote somehow or something else is syncing it (3rd party program).
Before you go down the path of worrying about tmp files, there is a process issue mixing between the remotes.
but! that was just me making the case that if a 3rd party was mixing in files, rclone has a definite log entry that shows it can see it. which, as the other log above shows, is not the case with tmp files.
what do you mean by mixing between remotes, brother animosity?
You have two remotes:
type = onedrive
token =
drive_id = b!LshYrNCGAUWnjNcIfrBRKsXCnZUGeHTZulsJAXsRKG
drive_type = business
[OneDriveWyEncryptednew]
type = crypt
remote = OneDriveWy:DTD Server
filename_encryption = standard
directory_name_encryption = false
password =
If you have a regular file being copied into a crypt location, that creates the problem you are seeing:
So someone/something/some process/some application copied into "OneDriveWy:DTD Server" instead of "OneDriveWyEncryptednew:"
if you get garbage files in the crypt remote, rclone can't read them, thus it can't delete them.
So in any easy example, if you have a directory that has one 'bad' file and nothing else, it can't be removed because of that bad file.
felix@gemini:~$ rclone sync gcrypt:one gcrypt:two -vvv
2022/07/01 11:41:11 DEBUG : Setting --config "/opt/rclone/rclone.conf" from environment variable RCLONE_CONFIG="/opt/rclone/rclone.conf"
2022/07/01 11:41:11 DEBUG : rclone: Version "v1.58.1" starting with parameters ["rclone" "sync" "gcrypt:one" "gcrypt:two" "-vvv"]
2022/07/01 11:41:11 DEBUG : Creating backend with remote "gcrypt:one"
2022/07/01 11:41:11 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2022/07/01 11:41:11 DEBUG : Creating backend with remote "GD:crypt/vgiitg6j8vuehta15ci1ig135g"
2022/07/01 11:41:12 DEBUG : Creating backend with remote "gcrypt:two"
2022/07/01 11:41:12 DEBUG : Creating backend with remote "GD:crypt/fns8mdipv34944sh8cv8kqpntc"
2022/07/01 11:41:12 DEBUG : fns8mdipv34944sh8cv8kqpntc/hosts: Skipping undecryptable file name: not a multiple of blocksize
2022/07/01 11:41:12 DEBUG : Encrypted drive 'gcrypt:two': Waiting for checks to finish
2022/07/01 11:41:12 DEBUG : Encrypted drive 'gcrypt:two': Waiting for transfers to finish
2022/07/01 11:41:14 INFO : one/hosts: Copied (server-side copy)
2022/07/01 11:41:14 DEBUG : Waiting for deletions to finish
2022/07/01 11:41:14 INFO : two/hosts: Deleted
2022/07/01 11:41:14 INFO : two: Removing directory
2022/07/01 11:41:14 DEBUG : fns8mdipv34944sh8cv8kqpntc: Rmdir: contains file: "hosts"
2022/07/01 11:41:14 DEBUG : two: Failed to Rmdir: directory not empty
2022/07/01 11:41:14 DEBUG : Encrypted drive 'gcrypt:two': failed to delete 1 directories
2022/07/01 11:41:14 INFO :
Transferred: 316 B / 316 B, 100%, 0 B/s, ETA -
Checks: 1 / 1, 100%
Deleted: 1 (files), 1 (dirs)
Transferred: 1 / 1, 100%
Elapsed time: 3.0s
2022/07/01 11:41:14 DEBUG : 9 go routines active
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.