Please start a new thread and we will get to the bottom of this. It is hard to tell what can be wrong based on your short description.
1 Like
Hello, I have a question please and I wonder if you can help me out.
I am currently on the enterprise plan, what is the best way to configure Box.com.
I don't feel that single sign on is efficient, have you tried to configure it with creating app method?
Or simply leave the app details like id and secret blank and just choose enterprise?
The other thing if I am using chunker with 49Gi, why do we still need to use the vfs-chunk-read flag?
Thank You
I am not sure what you mean by "efficient" but for any Box account (personal or enterprise) you can use single sign on or app method. It depends on level of account control you require - e.g. you might want some app id to only access certain folders etc.
These are completely not related things.
You use chunker because your account has max file limits. Otherwise you can not store files larger than 50 GB (and if you do not have such files you do not even need chunker).
vfs-chunk-read flag is used for mounts and its purpose is described here
1 Like
I hope I'm still on topic.
how efficient is chunker? can I use the cheapest business plan and let chunker chop up everything above 4.9G without any drawbacks?
I assume on a local mount, the files would look un-chunked?
how does that work for video files when you need some metainfo (e.b. mediainfo) and the file is chunked? is that no issue cause the meta data is usually at the beginning of the file?
how about scanning through a video file that is chunked? will rclone have to download the whole 80GB file first or is that all done on the fly like that file had not been chunked?
Chunker is fully transparent and files look and behave like un-chunked.
On different remotes for different reasons I used 256MiB chunks and all works fine for me.
Does it have some impact on performance? I am sure it does. It is negligible for my use but might be critical for yours.
Only way to see and measure real impact is to test. Create Box test account and run it against your expected workload.
thanks, will do!
@kapitainsky: (to avoid some misunderstanding from my side)
This config “order” will have crypted files on box share without the chunker filenames (*.rclone_chunk.###) visible” on it?
correct. the whole purpose is to hide from cloud provider everything that can be hidden - including chunking information.
but I still advice always do some testing before going live with BIG data:)
1 Like
I did added everything, run a test COPY with one client_id and client_secret, everything works just fine
I can also mount the chunker_remote: as a normal mount point and direct delete files if needed
If I want to have an additional extra client_id only for MOUNT, do I need to add box 3x again (box_mount + crypt_mount + chunker_mount)? with the same password and password2 and filename_encoding and hash_type?
will that work?
Even though I think
box_mount + crypt + chunker
would work I would create
box_mount + crypt_mount + chunker_mount
The reason is that this way 100% I avoid potentially some edge cases that maybe were not taken into account when implementing these remotes. For such cases I am lazy:) and prefer option that works 100% without any chance of failure.
1 Like
I'm very new to flags, and rclone in general, so I'm getting a bit lost in this thread. I'm currently not setting up a Box mount yet, just the remotes for migrations, Chunk > Crypt > Remote. Are there any flags are advanced configurations you'd advise I enable/add for better performance, Kap?
Once I begin to build mounts on the chunk box remote, does anyone else have any flag suggestions for better performance primarily utilizing the mount for Plex?
Additionally, I've simply been setting up these remotes with my user, no custom app. Is a custom app suggested? Does anyone have a good walk-through build for a well performing app? From the beginning of creating an app, I'm not sure the option that would be best between Custom App, Limited Access App, or Box Custom Skill.
EDIT: Scratch that, writing and explaining it out has shown me the error of my ways.
1 Like
Depends on issues you are facing. I usually do not look at extra flags until I need them (and it happens very rarely). Defaults should work perfectly fine for most cases.
Use VFS caching "full" mode for best experience. I am not Plex user so not sure what are this software specific requirements.
I have no issues with not using custom app. But I use box for backups - maybe Plex requires something special.
1 Like
Thanks for this guide @kapitainsky. Big files are successfully being transferred to box.com without issue by chunks but my issue is they are being transferred and showing as chunks, instead of original file. Any hint will be appreciated!
My rclone.conf is below,
[box]
type = box
token = {"access_token":"XXX","token_type":"bearer","refresh_token":"XXX","expiry":"XXX"}
[crypt_remote]
type = crypt
remote = box:
filename_encoding = base32768
#filename_encryption = off
#directory_name_encryption = false
password = XXXX
password2 = XXXX
[chunker_remote]
type = chunker
remote = crypt_remote:
chunk_size = 4.883Gi
name_format = *.rcc###
hash_type = sha1all
They were being transferred as expected, and I used below command to sync.
rclone sync gdrive:XXX chunker_remote:XXX --transfers 24 --checkers 24 --tpslimit 24 --fast-list --checksum --progress
Transferred: 37.627 GiB / 37.627 GiB, 100%, 1.110 GiB/s, ETA 0s
Checks: 5 / 5, 100%
Renamed: 5
Transferred: 2 / 2, 100%
Elapsed time: 17m29.4s
If you show an example we will figure out what is wrong.
Thanks @kapitainsky , attached is the screenshot from my mac to box.com.
Instead of assembling back chunks to the original, it is just showing the chunks as they are...
what is that picture? rclone mount? If yes - what is your mount command?
Sorry, if it was not clear. The screenshot is the result of rclone sync with my configuration as uploaded. I've mounted with box's mac app with mac finder.
Ok. And why do you expect rclone created chunks to be assembled? Does Box provide such functionality?
Until not - if you want to see assembled parts you have to access your box using rclone chunker_remote you created e.g.:
rclone ls chunker_remote:
Ohhh my bad. I thought that rclone will do reassemble files after uploading to the box, which was my misunderstanding. So chunker will divide files by certain sizes so that it can be uploaded and reassembling will happen only when downloading back.
Thanks so much @kapitainsky.
1 Like