OneDrive fails to list certain files (1.65)

What is the problem you are having with rclone?

Listing (lsjson) on OneDrive causes a ERROR : Documents: error listing: unknown object type <nil> error

NOTE: This is NOT with the new --onedrive-delta flag. I found it in testing that but this is without it.

The "Documents:" turns out to be a false flag. It is just a folder as found below.

Run the command 'rclone version' and share the full output of the command.

rclone v1.65.0
- os/version: ubuntu 23.04 (64 bit)
- os/kernel: 6.2.0-37-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.4
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

OneDrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone lsjson onedrive: -vv -R 1> out 2> err

The rclone config contents with secrets removed.

[onedrive]
type = onedrive
token = XXX
drive_id = XXX
drive_type = personal
client_id = XXX
client_secret = XXX

A log from the command with the -vv flag

This is the STDERR log.

2023/11/26 14:47:13 DEBUG : Setting --password-command "rclone-pass-store echo" from environment variable RCLONE_PASSWORD_COMMAND="rclone-pass-store echo"
2023/11/26 14:47:13 DEBUG : rclone: Version "v1.65.0" starting with parameters ["rclone" "lsjson" "onedrive:" "-vv" "-R" "--filter" "- nsc/**"]
2023/11/26 14:47:13 DEBUG : Creating backend with remote "onedrive:"
2023/11/26 14:47:13 DEBUG : Using config file from "/home/jwinokur/.config/rclone/rclone.conf"
2023/11/26 14:47:13 DEBUG : nsc: Excluded
2023/11/26 14:47:14 DEBUG : Justin's Notebook: OneNote file not shown in directory listing
2023/11/26 14:47:14 ERROR : Documents: error listing: unknown object type <nil>
2023/11/26 14:51:40 DEBUG : 21 go routines active
2023/11/26 14:51:40 Failed to lsjson with 2 errors: last error was: error in ListJSON: unknown object type <nil>

The STDOUT log is all of the files and is not super helpful but I can share if needed. The last 10 lines are:

{"Path":"backups/P2/r73g50pt4oip0gbu1rs9r6en38/vrhe7a47opefnd5nstm49uj0j87ocj5uqtttpnr3lq41erl637hg/5ntj3hr10h78dhbhivi1mem3lo/u7c2o3id0cj8fetbldn88mt1k2tm2eofn0apmu8qho3mopogkia0","Name":"u7c2o3id0cj8fetbldn88mt1k2tm2eofn0apmu8qho3mopogkia0","Size":629,"MimeType":"application/octet-stream","ModTime":"2023-09-06T09:48:55Z","IsDir":false,"ID":"1af35bc3b364a58c#1AF35BC3B364A58C!491835"},
{"Path":"backups/P2/r73g50pt4oip0gbu1rs9r6en38/vrhe7a47opefnd5nstm49uj0j87ocj5uqtttpnr3lq41erl637hg/5ntj3hr10h78dhbhivi1mem3lo/uggaqng5gjkgg040tkuol2m88quct2fmppt4mol4h622u98a0j00","Name":"uggaqng5gjkgg040tkuol2m88quct2fmppt4mol4h622u98a0j00","Size":663,"MimeType":"application/octet-stream","ModTime":"2023-08-08T13:03:51Z","IsDir":false,"ID":"1af35bc3b364a58c#1AF35BC3B364A58C!487834"},
{"Path":"backups/P2/r73g50pt4oip0gbu1rs9r6en38/vrhe7a47opefnd5nstm49uj0j87ocj5uqtttpnr3lq41erl637hg/5ntj3hr10h78dhbhivi1mem3lo/ugmv01qqgqcnb3s8bgm91go4ej1hrvov2kol73dq9d9ts4msadtg","Name":"ugmv01qqgqcnb3s8bgm91go4ej1hrvov2kol73dq9d9ts4msadtg","Size":618,"MimeType":"application/octet-stream","ModTime":"2023-07-15T10:14:44Z","IsDir":false,"ID":"1af35bc3b364a58c#1AF35BC3B364A58C!482177"},
{"Path":"backups/P2/r73g50pt4oip0gbu1rs9r6en38/vrhe7a47opefnd5nstm49uj0j87ocj5uqtttpnr3lq41erl637hg/5ntj3hr10h78dhbhivi1mem3lo/uhi0kebj1pmbdgtf4e4l1nar4lf9h38aqrqpfdegviebnrmgg26g","Name":"uhi0kebj1pmbdgtf4e4l1nar4lf9h38aqrqpfdegviebnrmgg26g","Size":557,"MimeType":"application/octet-stream","ModTime":"2023-06-10T13:03:24Z","IsDir":false,"ID":"1af35bc3b364a58c#1AF35BC3B364A58C!475767"},
{"Path":"backups/P2/r73g50pt4oip0gbu1rs9r6en38/vrhe7a47opefnd5nstm49uj0j87ocj5uqtttpnr3lq41erl637hg/5ntj3hr10h78dhbhivi1mem3lo/upkg9u0bis6nm2bd3au02td10aemprllbhdngkbh175ck5ik3ks0","Name":"upkg9u0bis6nm2bd3au02td10aemprllbhdngkbh175ck5ik3ks0","Size":630,"MimeType":"application/octet-stream","ModTime":"2023-09-04T07:23:48Z","IsDir":false,"ID":"1af35bc3b364a58c#1AF35BC3B364A58C!491425"},
{"Path":"backups/P2/r73g50pt4oip0gbu1rs9r6en38/vrhe7a47opefnd5nstm49uj0j87ocj5uqtttpnr3lq41erl637hg/5ntj3hr10h78dhbhivi1mem3lo/ve425mpirvclqdbhti339j4tt5r89fkcsfnqb5m5ufuqjk2m0jng","Name":"ve425mpirvclqdbhti339j4tt5r89fkcsfnqb5m5ufuqjk2m0jng","Size":626,"MimeType":"application/octet-stream","ModTime":"2023-09-04T07:35:24Z","IsDir":false,"ID":"1af35bc3b364a58c#1AF35BC3B364A58C!491452"},
{"Path":"backups/P2/r73g50pt4oip0gbu1rs9r6en38/vrhe7a47opefnd5nstm49uj0j87ocj5uqtttpnr3lq41erl637hg/5ntj3hr10h78dhbhivi1mem3lo/vhg0ed9fbdp5iaftedukr5ujjbo088ub0qn1u9dubb51o29ao2cg","Name":"vhg0ed9fbdp5iaftedukr5ujjbo088ub0qn1u9dubb51o29ao2cg","Size":703,"MimeType":"application/octet-stream","ModTime":"2023-07-27T13:15:27Z","IsDir":false,"ID":"1af35bc3b364a58c#1AF35BC3B364A58C!486385"},
{"Path":"backups/P2/r73g50pt4oip0gbu1rs9r6en38/vrhe7a47opefnd5nstm49uj0j87ocj5uqtttpnr3lq41erl637hg/5ntj3hr10h78dhbhivi1mem3lo/vjc21rmg7pitcck0vlkiuabr4070he0nppm0agf4nlhi0bb7qv10","Name":"vjc21rmg7pitcck0vlkiuabr4070he0nppm0agf4nlhi0bb7qv10","Size":667,"MimeType":"application/octet-stream","ModTime":"2023-09-04T07:36:14Z","IsDir":false,"ID":"1af35bc3b364a58c#1AF35BC3B364A58C!491474"},
{"Path":"backups/P2/r73g50pt4oip0gbu1rs9r6en38/vrhe7a47opefnd5nstm49uj0j87ocj5uqtttpnr3lq41erl637hg/5ntj3hr10h78dhbhivi1mem3lo/vr06jsues4vbafi3g42fo76bdv2q1qteg479tb11otscctm80efg","Name":"vr06jsues4vbafi3g42fo76bdv2q1qteg479tb11otscctm80efg","Size":627,"MimeType":"application/octet-stream","ModTime":"2023-09-04T07:24:25Z","IsDir":false,"ID":"1af35bc3b364a58c#1AF35BC3B364A58C!491553"},
{"Path":"backups/P2/r73g50pt4oip0gbu1rs9r6en38/vrhe7a47opefnd5nstm49uj0j87ocj5uqtttpnr3lq41erl637hg/5ntj3hr10h78dhbhivi1mem3lo/vunrt9o4bugvpq1paf9pipugvg1o4fc1j53ug34rbdvi9aqdbk2g","Name":"vunrt9o4bugvpq1paf9pipugvg1o4fc1j53ug34rbdvi9aqdbk2g","Size":619,"MimeType":"application/octet-stream","ModTime":"2023-09-04T07:35:53Z","IsDir":false,"ID":"1af35bc3b364a58c#1AF35BC3B364A58C!491414"}

You'll note this is an encrypted name but for all intents and purposes, I am not using crypt. It is just backing up crypted files too.


I ran this agin with 1.64 and, using Python to read the malformed JSON, was able to combine the file lististing

The following work in 1.64 but are not in 1.65

Documents/DO NOT USE -- USE personal_files
Documents/DO NOT USE -- USE personal_files.txt
Documents/shared

Note that these are just files/dirs in that as a warning. There is nothing special about them.

The full 1.64 listing for these is:

[
 {
  "Path": "Documents/DO NOT USE -- USE personal_files",
  "Name": "DO NOT USE -- USE personal_files",
  "Size": -1,
  "MimeType": "inode/directory",
  "ModTime": "2023-10-20T17:26:12Z",
  "IsDir": true,
  "ID": "1af35bc3b364a58c#1AF35BC3B364A58C!498268"
 },
 {
  "Path": "Documents/DO NOT USE -- USE personal_files.txt",
  "Name": "DO NOT USE -- USE personal_files.txt",
  "Size": 13,
  "MimeType": "text/plain",
  "ModTime": "2023-10-20T17:26:26Z",
  "IsDir": false,
  "ID": "1af35bc3b364a58c#1AF35BC3B364A58C!498269"
 },
 {
  "Path": "Documents/shared",
  "Name": "shared",
  "Size": -1,
  "MimeType": "inode/directory",
  "ModTime": "2022-01-04T23:24:18Z",
  "IsDir": true,
  "ID": "1af35bc3b364a58c#1AF35BC3B364A58C!310840"
 }
]

I tried to create a file called "onedrive:newtmp/test -- ing.txt" but it works fine.

1 Like

I got the same error using v1.65.

I am having a similar issue, albeit with the Sync command.

Reverting back to 1.64.2 from 1.65 resolves the issue.

1 Like

Mount command is also affected.

Oh, looks like this is a bug introduced when making --onedrive-delta.

It is very easy to fix though - please try this

v1.66.0-beta.7551.0872f62ad.fix-onedrive-list on branch fix-onedrive-list (uploaded in 15-30 mins)

3 Likes

That worked! Hopefully that makes it to 1.65.1 soon!

Thanks, as alwasy!

I've merged this to master now which means it will be in the latest beta in 15-30 minutes and released in v1.65.1

Thanks for testing!

BTW I forgot to say, this is triggered by OneNote files (and possibly other non-file based objects) so you don't meet this bug unless you try to list one of those things.

1 Like

Hi,

as I was affected by this too, I would like to report that:

  • v1.66.0-beta.7551.0872f62ad.fix-onedrive-list - indeed solves my issue
  • v1.66.0-beta.7558.e3d0bff9c (latest beta at the time of writing) - unfortunately does not

Thanks for the fix!

1 Like

Good catch.

fix-onedrive-list is not merged with master yet...

Attn. @ncw

I don't know what happened there - thanks for the heads up - but I really did merge it this time! So it will be in the latest beta in 30 mins and I'll release it in the point release.

Now it is in master. Thank you for taking care of it..

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.