Dropbox "error reading source directory" errors

Help and Support
1

What is the problem you are having with rclone?

When syncing from Dropbox to local I get regular HTML errors like this:

</body>
</html>
2024/11/05 21:10:23 ERROR : xxx/xxx/xxx/xxx: error reading source directory: <!DOCTYPE html>
<html>
<head><meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Dropbox - 504</title>
<link href="https://cfl.dropboxstatic.com/static/metaserver/static/css/error.css" rel="stylesheet" type="text/css"/>
<link rel="shortcut icon" href="https://cfl.dropboxstatic.com/static/images/favicon.ico"/>
</head>
<body>
<div class="figure">
<img src="https://assets.dropbox.com/www/en-us/illustrations/spot/target-miss.svg" alt="Error: 504"/>
</div>
<div id="errorbox">
<h1>Error (504)</h1>Something went wrong. Don't worry, your files are still safe and the Dropbox team has been notified. Check out our <a href="https://status.dropbox.com">Status Page</a> to see if there is a known incident, our <a href="https://www.dropbox.com/help">Help Center</a> and <a href="https://forums.dropbox.com">forums</a> for help, or head back to <a href="https://www.dropbox.com/home">home</a>.
</div>

</body>
</html>

It seems to happen during checking rather than transfers but I cannot be certain.

It happens randomly, it doesn't happen every time (although it happens most times) and it is different folders each time. It normally succeeds within the three retries.

Run the command 'rclone version' and share the full output of the command.

rclone v1.68.1
- os/version: centos 7.9.2009 (64 bit)
- os/kernel: 3.10.0-1160.119.1.el7.x86_64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.23.1
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Dropbox

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync dropboxpaul: /data/dropbox/paul --tpslimit 10 --tpslimit-burst 0 --progress --fast-list

I have tried reducing the --tpslimit, removing --fast-list and setting --checkers to 4 (instead of the default 8) but these made no difference.

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[dropboxpaul]
type = dropbox
client_id = XXX
client_secret = XXX
token = XXX

A log from the command that you were trying to run with the -vv flag

The dropbox account contains over 70,000 files with only a handful to update on each run so the log would be impossibly large to paste and would also contain lots of sensitive information from the filenames.

If there is a way of catching just the errors without all the irrelevant and sensitive information please let me know and I will do so.

This log is with -v (single v) but I don't think it adds anything useful:

rclone sync dropboxpaul: /data/dropbox/paul --tpslimit 10 --tpslimit-burst 0 --fast-list --progress -v
2024/11/05 21:25:24 INFO  : Starting transaction limiter: max 10 transactions/s with burst 1
2024/11/05 21:34:19 ERROR : xxx/xxx/xxx/xxx: error reading source directory: <!DOCTYPE html>
<html>
<head><meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Dropbox - 504</title>
<link href="https://cfl.dropboxstatic.com/static/metaserver/static/css/error.css" rel="stylesheet" type="text/css"/>
<link rel="shortcut icon" href="https://cfl.dropboxstatic.com/static/images/favicon.ico"/>
</head>
<body>
<div class="figure">
<img src="https://assets.dropbox.com/www/en-us/illustrations/spot/target-miss.svg" alt="Error: 504"/>
</div>
<div id="errorbox">
<h1>Error (504)</h1>Something went wrong. Don't worry, your files are still safe and the Dropbox team has been notified. Check out our <a href="https://status.dropbox.com">Status Page</a> to see if there is a known incident, our <a href="https://www.dropbox.com/help">Help Center</a> and <a href="https://forums.dropbox.com">forums</a> for help, or head back to <a href="https://www.dropbox.com/home">home</a>.
</div>

</body>
</html>
2024/11/05 21:34:19 ERROR : Local file system at /data/dropbox/paul: not deleting files as there were IO errors
2024/11/05 21:34:19 ERROR : Local file system at /data/dropbox/paul: not deleting directories as there were IO errors
2024/11/05 21:34:19 ERROR : Attempt 1/3 failed with 1 errors and: <!DOCTYPE html>
<html>
<head><meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Dropbox - 504</title>
<link href="https://cfl.dropboxstatic.com/static/metaserver/static/css/error.css" rel="stylesheet" type="text/css"/>
<link rel="shortcut icon" href="https://cfl.dropboxstatic.com/static/images/favicon.ico"/>
</head>
<body>
<div class="figure">
<img src="https://assets.dropbox.com/www/en-us/illustrations/spot/target-miss.svg" alt="Error: 504"/>
</div>
<div id="errorbox">
<h1>Error (504)</h1>Something went wrong. Don't worry, your files are still safe and the Dropbox team has been notified. Check out our <a href="https://status.dropbox.com">Status Page</a> to see if there is a known incident, our <a href="https://www.dropbox.com/help">Help Center</a> and <a href="https://forums.dropbox.com">forums</a> for help, or head back to <a href="https://www.dropbox.com/home">home</a>.
</div>

</body>
</html>
2024/11/05 21:42:24 INFO  : There was nothing to transfer
2024/11/05 21:42:24 ERROR : Attempt 2/3 succeeded
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:            132179 / 132179, 100%
Elapsed time:      17m0.1s
2024/11/05 21:42:24 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Checks:            132179 / 132179, 100%
Elapsed time:      17m0.1s

2024/11/05 21:42:24 INFO  : Dropbox root '': Committing uploads - please wait...

In this case it happened once during the 17 minute comparison and was successful on the second attempt. Because I have run it a few times while testing there was nothing to transfer this time.

Thank you for any help or suggestions etc.
Paul

1 Like
2

Have you tried increasing the number of low-level retries from the default value of 10?

I had intermittently experienced the problem but it became much more frequent this weekend. Increasing the default number from 10 to 50 appears to have solved the problem for me. This value was chosen arbitrarily and I didn't test different values, so I'm not sure what an optimum should be.

Hope this helps!

1 Like
3

It is interesting that you have also seen a significant increase in these errors recently too. Must be something their end, although the Linux and Windows desktop apps still work fine.

I haven't tried the low-level-retries option. I will change it to 50 in my bash script now and see how it goes over the next few days.

Thank you for that suggestion.
Paul

4 
transaction limiter: max 4 transactions/s with burst 1

I got the same kinda error in this last week (I perform a sync job planned every Saturday). It seems so randomic and rare (in my case I found a couple in the log with >10k files to sync)

I added option concerning a delay on subsequent attempts ( --retries-sleep 60). It doesn't fix the rise of such errors, but they remain limited to 1/3 failed attempts.

By now...

Edit: it seems an internal DropBox issue, I encountered the same error just a minute ago while uploading a file by browser through the web interface

5

that is a server-side error, should contact dropbox.

6

After increasing low-level-retries to 50 it worked fine for 6 days then had the errors again. So that may have just been coincidence, if it is indeed a Dropbox issue.

Is there any way of hiding the HTML in the rclone output and just logging the errors? That would make it easier to read.

Interestingly I have never seen issues like this with the Dropbox app on Linux or Windows, but that may hide the errors and just quietly retry until it succeeds.

7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.