What is the problem you are having with rclone?
Many Apple computers with M1 (and M1 Pro) CPU will have a kernel panic when running rclone sync.
One example of such panic is given below, and is always a matter of "watchdog timeout: no checkins from watchdogd in x seconds".
We've encountered this on countless M1 computers (iMacs, mac Minis, macBooks), with different versions of macOS.
If we run rclone sync on a M1 computer and it leads to a kernel panic, it will always lead to a kernel panic everytime we run a rclone sync.
According to -vv logs, it isn't related to the data transfered: runing rclone sync twice on a same computer with the same data set shows that there's no correlation between the folder/file being processed and the kernel panic.
The only workaround found so far is to format the hard drive and re-install macOS to the most up-to-date version.
We had two examples yesterday:
MacBook Pro M1 Pro:
Out of the box:
System version: 12.2.1 (21D62)
Kernel version: Darwin 21.3.0
-> kernel panic
OS reinstall:
System version: macOS 12.3 (21E230)
Kernel version: Darwin 21.4.0
-> no panic
The exact same for a Mac Mini M1: same versions prior to re-install leading to panic, same version after re-install with rclone sync running without problem.
Run the command 'rclone version' and share the full output of the command.
rclone v1.57.0
- os/version: darwin 12.2 (64 bit)
- os/kernel: 21.3.0 (arm64)
- os/type: darwin
- os/arch: arm64
- go/version: go1.17.2
- go/linking: dynamic
- go/tags: cmount
Which cloud storage system are you using? (eg Google Drive)
None.
We transfer files between our computers an a smb:// share (vice et versa).
The command you were trying to run (eg rclone copy /tmp remote:tmp)
/Volumes/our-nas/.../rclone-v1.57.0-osx-arm64/rclone sync -P \
--log-file="$HOME/Desktop/rcloneTest.log" --log-level INFO \
--size-only --no-unicode-normalization --max-backlog=1000000 --checkers=8 --transfers=4 \
"/Volumes/our-nas/.../SomeFolder" "$HOME/Desktop/rclone-TEST"
The rclone config contents with secrets removed.
We don't use a config file.
And as shown in the above rclone command, we call rclone binary on our NAS, in order to be able to run it on any machine we want to get the data on.
A log from the command with the -vv flag
The log files with log-level INFO are way to long to fit this post.
And there's nothing informative in it.
A summary of the log file:
- 1 heading line "NOTICE: Config file "/Users/username/.config/rclone/rclone.conf" not found - using defaults"
- 6891 lines "INFO : path/to/file : Copied (new)"
- 14829 lines "NOTICE: path/to/file : Can't follow symlink without -L/--copy-links"
- 1 line "NOTICE: path/to/file : Can't transfer non file/directory" (it is not the last one, it's the 920 out of 21722).
Last line of the log file simply is the last file rclone could transfer before the computer crashed.
Sorry but I can't even make a pastebin out of the log, it's too big.
The most recent panic log
panic(cpu 0 caller 0xfffffe00286cbd0c): watchdog timeout: no checkins from watchdogd in 94 seconds (123 total checkins since monitoring last enabled)
Debugger message: panic
Memory ID: 0x6
OS release type: User
OS version: 21D62
Kernel version: Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T6000
Fileset Kernelcache UUID: FA4EB485BA9DC1EBAA5D0E80232A48CC
Kernel UUID: BADF56F4-2876-3FF4-AC12-F25E78B09AA1
iBoot version: iBoot-7429.81.3
secure boot?: YES
Paniclog version: 13
KernelCache slide: 0x0000000020378000
KernelCache base: 0xfffffe002737c000
Kernel slide: 0x0000000020bac000
Kernel text base: 0xfffffe0027bb0000
Kernel text exec slide: 0x0000000020c94000
Kernel text exec base: 0xfffffe0027c98000
mach_absolute_time: 0xc4acc74fc
Epoch Time: sec usec
Boot : 0x62330b97 0x000ba2ae
Sleep : 0x62332238 0x00035462
Wake : 0x623325ce 0x00059323
Calendar: 0x62332afc 0x00016b14
Zone info:
Foreign : 0xfffffe002fb94000 - 0xfffffe002fba8000
Native : 0xfffffe100047c000 - 0xfffffe300047c000
Readonly : 0xfffffe14cd148000 - 0xfffffe1666adc000
Metadata : 0xfffffe6c0feec000 - 0xfffffe6c1be30000
Bitmaps : 0xfffffe6c1be30000 - 0xfffffe6c219e4000
Largest zones:
Zone Name Cur Size Free Size
kext. kalloc.32768 5376M 0K
vnodes 61M 5K
APFS_INODES 60M 27460K
vnode pager structur 6M 877K
kalloc.type2.32 5M 1181K
CORE 0 PVH locks held: None
CORE 1 PVH locks held: None
CORE 2 PVH locks held: None
CORE 3 PVH locks held: None
CORE 4 PVH locks held: None
CORE 5 PVH locks held: None
CORE 6 PVH locks held: None
CORE 7 PVH locks held: None
CORE 0 is the one that panicked. Check the full backtrace for details.
CORE 1: PC=0xfffffe0027d394a4, LR=0xfffffe0027d39494, FP=0xfffffe60b7debdc0
CORE 2: PC=0xfffffe0027d1ef50, LR=0xfffffe0027d1ef4c, FP=0xfffffe60b536bef0
CORE 3: PC=0xfffffe0027d1ef50, LR=0xfffffe0027d1ef4c, FP=0xfffffe60396abef0
CORE 4: PC=0xfffffe0027d1ef4c, LR=0xfffffe0027d1ef4c, FP=0xfffffe60b5193ef0
CORE 5: PC=0xfffffe0027d1ef50, LR=0xfffffe0027d1ef4c, FP=0xfffffe60a56cbef0
CORE 6: PC=0xfffffe0027d1ef4c, LR=0xfffffe0027d1ef4c, FP=0xfffffe60b5f03ef0
CORE 7: PC=0xfffffe0027d1ef50, LR=0xfffffe0027d1ef4c, FP=0xfffffe60b50d3ef0
Total cpu_usage: 15754410
Thread task pri cpu_usage
0xfffffe1b33929860 kernel_task 95 571
0xfffffe1b33dfe9a0 kernel_task 95 13746
0xfffffe1b33929040 kernel_task 0 5091940
0xfffffe1b34164920 kernel_task 0 5730812
0xfffffe1b34161860 kernel_task 0 1040278
Panicked task 0xfffffe20005cc678: 0 pages, 540 threads: pid 0: kernel_task
Panicked thread: 0xfffffe1b33929040, backtrace: 0xfffffe6017cbf830, tid: 102
lr: 0xfffffe0027cea4e8 fp: 0xfffffe6017cbf8a0
lr: 0xfffffe0027cea1b8 fp: 0xfffffe6017cbf910
lr: 0xfffffe0027e2a2bc fp: 0xfffffe6017cbf930
lr: 0xfffffe0027e1c6c8 fp: 0xfffffe6017cbf9b0
lr: 0xfffffe0027e1a118 fp: 0xfffffe6017cbfa70
lr: 0xfffffe0027c9f7f8 fp: 0xfffffe6017cbfa80
lr: 0xfffffe0027ce9e2c fp: 0xfffffe6017cbfe20
lr: 0xfffffe0027ce9e2c fp: 0xfffffe6017cbfe90
lr: 0xfffffe00284fd7a0 fp: 0xfffffe6017cbfeb0
lr: 0xfffffe00286cbd0c fp: 0xfffffe6017cbfee0
lr: 0xfffffe00286cb0f4 fp: 0xfffffe6017cbff20
lr: 0xfffffe00286c8b68 fp: 0xfffffe6017cbff40
lr: 0xfffffe0028ec2534 fp: 0xfffffe6017cbffd0
lr: 0xfffffe0027e1d21c fp: 0xfffffe6017cbffe0
lr: 0xfffffe0027c9f86c fp: 0xfffffe6017cbfff0
lr: 0xfffffe0027d1ef4c fp: 0xfffffe60b42ebef0
lr: 0xfffffe0027d1f164 fp: 0xfffffe60b42ebf20
lr: 0xfffffe0027ca8e78 fp: 0x0000000000000000
Kernel Extensions in backtrace:
com.apple.driver.AppleInterruptControllerV2(1.0d1)[84EF9B26-ADAB-34FA-8D4D-B5D075A9993C]@0xfffffe0028ec0b00->0xfffffe0028ec3437
dependency: com.apple.driver.AppleARMPlatform(1.0.2)[D5F48829-453D-3EB2-96FC-933E1C883CE2]@0xfffffe002867f2b0->0xfffffe00286c7a4f
com.apple.driver.AppleARMWatchdogTimer(1.0)[D17395CE-C4B1-3CA9-B28B-B4DBEBCA8674]@0xfffffe00286c7a50->0xfffffe00286cc1db
dependency: com.apple.driver.AppleARMPlatform(1.0.2)[D5F48829-453D-3EB2-96FC-933E1C883CE2]@0xfffffe002867f2b0->0xfffffe00286c7a4f
last started kext at 2808715228: com.apple.filesystems.smbfs 4.0 (addr 0xfffffe0027b94c70, size 64483)
loaded kexts:
com.apple.filesystems.smbfs 4.0
com.apple.fileutil 20.036.15
com.apple.filesystems.autofs 3.0
com.apple.driver.AppleBiometricServices 1
com.apple.driver.CoreKDL 1
com.apple.driver.usb.realtek8153patcher 5.0.0
com.apple.iokit.SCSITaskUserClient 452.60.2
com.apple.driver.AppleTopCaseHIDEventDriver 5020.1
com.apple.driver.SEPHibernation 1
com.apple.driver.BCMWLANFirmware4387.Hashstore 1
com.apple.driver.DiskImages.ReadWriteDiskImage 493.0.0
com.apple.driver.DiskImages.UDIFDiskImage 493.0.0
com.apple.driver.DiskImages.RAMBackingStore 493.0.0
com.apple.driver.DiskImages.FileBackingStore 493.0.0
com.apple.filesystems.apfs 1933.80.3
com.apple.driver.AppleUSBDeviceNCM 5.0.0
com.apple.driver.AppleThunderboltIP 4.0.3
com.apple.driver.AppleFileSystemDriver 3.0.1
com.apple.nke.l2tp 1.9
com.apple.filesystems.tmpfs 1
com.apple.filesystems.lifs 1
com.apple.IOTextEncryptionFamily 1.0.0
com.apple.filesystems.hfs.kext 582.60.2
com.apple.security.BootPolicy 1
com.apple.BootCache 40
com.apple.AppleFSCompression.AppleFSCompressionTypeZlib 1.0.0
com.apple.AppleFSCompression.AppleFSCompressionTypeDataless 1.0.0d1
com.apple.driver.ApplePMP 1
com.apple.AppleEmbeddedSimpleSPINORFlasher 1
com.apple.driver.AppleSmartIO2 1
com.apple.driver.AppleCS42L84Audio 530.2
com.apple.driver.AppleSN012776Amp 530.2
com.apple.driver.AppleT6000SOCTuner 1
com.apple.driver.AppleT6000CLPCv3 1
com.apple.driver.AppleSmartBatteryManager 161.0.0
com.apple.driver.AppleALSColorSensor 1.0.0d1
com.apple.driver.AppleAOPVoiceTrigger 100.1
com.apple.driver.ApplePMPFirmware 1
com.apple.driver.AppleM68Buttons 1.0.0d1
com.apple.driver.AppleProResHW 128.2.0
com.apple.driver.AppleJPEGDriver 4.7.9
com.apple.driver.AppleAVE2 530.3.0
com.apple.driver.AppleAVD 555
com.apple.AGXG13X 188.10
com.apple.driver.AppleSDXC 3.1.1
com.apple.driver.AppleSamsungSerial 1.0.0d1
com.apple.driver.AppleSerialShim 1
com.apple.driver.AppleMobileDispT600X-DCP 140.0
com.apple.driver.AppleSPMIPMU 1.0.1
com.apple.driver.usb.AppleSynopsysUSB40XHCI 1
com.apple.driver.AppleMCDP29XXUpdateSupport 1
com.apple.driver.AppleDPDisplayTCON 1
com.apple.driver.AppleEventLogHandler 1
com.apple.driver.AppleS5L8960XNCO 1
com.apple.driver.AppleT6000PMGR 1
com.apple.driver.AppleS8000AES 1
com.apple.driver.AppleS8000DWI 1.0.0d1
com.apple.driver.AppleInterruptControllerV2 1.0.0d1
com.apple.driver.AppleT8110DART 1
com.apple.driver.AppleBluetoothModule 1
com.apple.driver.AppleBCMWLANBusInterfacePCIe 1
com.apple.driver.AppleS5L8920XPWM 1.0.0d1
com.apple.driver.AudioDMAController-T600x 100.51
com.apple.driver.AppleT6000DART 1
com.apple.driver.AppleSPIMC 1
com.apple.driver.AppleS5L8940XI2C 1.0.0d2
com.apple.driver.AppleT6000 1
com.apple.iokit.IOUserEthernet 1.0.1
com.apple.driver.usb.AppleUSBUserHCI 1
com.apple.iokit.IOKitRegistryCompatibility 1
com.apple.iokit.EndpointSecurity 1
com.apple.driver.AppleDiskImages2 126.60.3
com.apple.AppleSystemPolicy 2.0.0
com.apple.nke.applicationfirewall 402
com.apple.kec.InvalidateHmac 1
com.apple.kec.AppleEncryptedArchive 1
com.apple.driver.driverkit.serial 6.0.0
com.apple.iokit.IOAVBFamily 1010.2
com.apple.plugin.IOgPTPPlugin 1000.11
com.apple.iokit.IOEthernetAVBController 1.1.0
com.apple.kext.triggers 1.0
com.apple.driver.AppleMesaSEPDriver 100.99
com.apple.iokit.IOBiometricFamily 1
com.apple.driver.usb.IOUSBHostHIDDevice 1.2
com.apple.driver.usb.cdc.ecm 5.0.0
com.apple.driver.usb.cdc.acm 5.0.0
com.apple.driver.usb.serial 6.0.0
com.apple.driver.AppleHIDKeyboard 228
com.apple.driver.AppleActuatorDriver 5430.21
com.apple.driver.AppleMultitouchDriver 5430.21
com.apple.driver.AppleHSBluetoothDriver 5020.1
com.apple.driver.IOBluetoothHIDDriver 9.0.0
com.apple.driver.usb.cdc 5.0.0
com.apple.driver.AppleUSBAudio 414.7
com.apple.iokit.IOAudioFamily 300.10
com.apple.vecLib.kext 1.2.0
com.apple.driver.AppleSEPHDCPManager 1.0.1
com.apple.driver.AppleTrustedAccessory 1
com.apple.iokit.AppleSEPGenericTransfer 1
com.apple.driver.AppleConvergedIPCOLYBTControl 1
com.apple.driver.AppleConvergedPCI 1
com.apple.driver.AppleBluetoothDebug 1
com.apple.driver.DiskImages.KernelBacked 493.0.0
com.apple.driver.AppleXsanScheme 3
com.apple.driver.usb.networking 5.0.0
com.apple.driver.AppleThunderboltUSBDownAdapter 1.0.4
com.apple.driver.AppleThunderboltDPInAdapter 8.5.1
com.apple.driver.AppleThunderboltDPAdapterFamily 8.5.1
com.apple.driver.AppleThunderboltPCIDownAdapter 4.1.1
com.apple.nke.ppp 1.9
com.apple.driver.AppleBSDKextStarter 3
com.apple.filesystems.hfs.encodings.kext 1
com.apple.driver.AppleHIDTransportSPI 5400.30
com.apple.driver.AppleHIDTransport 5400.30
com.apple.driver.AppleInputDeviceSupport 5400.30
com.apple.driver.AppleBTM 1.0.1
com.apple.driver.AppleDCPDPTXProxy 1.0.0
com.apple.driver.DCPDPFamilyProxy 1
com.apple.driver.AppleDiagnosticDataAccessReadOnly 1.0.0
com.apple.driver.AppleCSEmbeddedAudio 530.2
com.apple.driver.ApplePassthroughPPM 3.0
com.apple.driver.AppleAOPAudio 102.2
com.apple.driver.AppleEmbeddedAudio 530.2
com.apple.iokit.AppleARMIISAudio 100.1
com.apple.driver.AppleSPU 1
com.apple.AGXFirmwareKextG13XRTBuddy 188.10
com.apple.AGXFirmwareKextRTBuddy64 188.10
com.apple.driver.DCPAVFamilyProxy 1
com.apple.iokit.IOGPUFamily 35.11
com.apple.iokit.IONVMeFamily 2.1.0
com.apple.driver.AppleNANDConfigAccess 1.0.0
com.apple.driver.AppleStockholmControl 1.0.0
com.apple.iokit.IOMobileGraphicsFamily-DCP 343.0.0
com.apple.driver.AppleDCP 1
com.apple.driver.AppleFirmwareKit 1
com.apple.iokit.IOMobileGraphicsFamily 343.0.0
com.apple.driver.usb.AppleUSBHostPacketFilter 1.0
com.apple.driver.AppleDialogPMU 1.0.1
com.apple.driver.AppleSPMI 1.0.1
com.apple.driver.AppleUSBXDCIARM 1.0
com.apple.driver.AppleUSBXDCI 1.0
com.apple.iokit.IOUSBDeviceFamily 2.0.0
com.apple.driver.usb.AppleSynopsysUSBXHCI 1
com.apple.driver.usb.AppleUSBXHCI 1.2
com.apple.driver.AppleEmbeddedUSBHost 1
com.apple.driver.usb.AppleUSBHub 1.2
com.apple.driver.usb.AppleUSBHostCompositeDevice 1.2
com.apple.driver.AppleT6000TypeCPhy 1
com.apple.driver.AppleT8103TypeCPhy 1
com.apple.driver.AppleHPM 3.4.4
com.apple.driver.AppleSART 1
com.apple.driver.ApplePMGR 1
com.apple.driver.AppleARMWatchdogTimer 1
com.apple.driver.AppleDisplayCrossbar 1.0.0
com.apple.iokit.IODisplayPortFamily 1.0.0
com.apple.driver.AppleTypeCPhy 1
com.apple.driver.AppleThunderboltNHI 7.2.8
com.apple.driver.AppleT6000PCIeC 1
com.apple.iokit.IOThunderboltFamily 9.3.3
com.apple.driver.ApplePIODMA 1
com.apple.driver.AppleT600xPCIe 1
com.apple.driver.AppleMultiFunctionManager 1
com.apple.driver.AppleBluetoothDebugService 1
com.apple.driver.AppleBCMWLANCore 1.0.0
com.apple.iokit.IO80211Family 1200.12.2b1
com.apple.driver.IOImageLoader 1.0.0
com.apple.driver.AppleOLYHAL 1
com.apple.driver.corecapture 1.0.4
com.apple.driver.AppleEmbeddedPCIE 1
com.apple.driver.AppleMCA2-T600x 600.95
com.apple.driver.AppleEmbeddedAudioLibs 100.9.1
com.apple.driver.AppleFirmwareUpdateKext 1
com.apple.driver.AppleH13CameraInterface 4.87.0
com.apple.driver.AppleH10PearlCameraInterface 17.0.3
com.apple.driver.AppleGPIOICController 1.0.2
com.apple.driver.AppleFireStormErrorHandler 1
com.apple.driver.AppleMobileApNonce 1
com.apple.iokit.IOTimeSyncFamily 1000.11
com.apple.driver.DiskImages 493.0.0
com.apple.iokit.IOGraphicsFamily 593
com.apple.iokit.IOBluetoothSerialManager 9.0.0
com.apple.iokit.IOBluetoothHostControllerUSBTransport 9.0.0
com.apple.iokit.IOBluetoothHostControllerUARTTransport 9.0.0
com.apple.iokit.IOBluetoothHostControllerTransport 9.0.0
com.apple.driver.IOBluetoothHostControllerPCIeTransport 9.0.0
com.apple.iokit.IOBluetoothFamily 9.0.0
com.apple.driver.FairPlayIOKit 68.13.1
com.apple.iokit.CSRBluetoothHostControllerUSBTransport 9.0.0
com.apple.iokit.BroadcomBluetoothHostControllerUSBTransport 9.0.0
com.apple.driver.AppleSSE 1.0
com.apple.driver.AppleSEPKeyStore 2
com.apple.driver.AppleUSBTDM 532.40.7
com.apple.iokit.IOUSBMassStorageDriver 209.40.6
com.apple.iokit.IOPCIFamily 2.9
com.apple.iokit.IOSCSIBlockCommandsDevice 452.60.2
com.apple.iokit.IOSCSIArchitectureModelFamily 452.60.2
com.apple.driver.AppleIPAppender 1.0
com.apple.driver.AppleFDEKeyStore 28.30
com.apple.driver.AppleEffaceableStorage 1.0
com.apple.driver.AppleCredentialManager 1.0
com.apple.driver.KernelRelayHost 1
com.apple.iokit.IOUSBHostFamily 1.2
com.apple.driver.AppleUSBHostMergeProperties 1.2
com.apple.driver.usb.AppleUSBCommon 1.0
com.apple.driver.AppleSMC 3.1.9
com.apple.driver.RTBuddy 1.0.0
com.apple.driver.AppleEmbeddedTempSensor 1.0.0
com.apple.driver.AppleARMPMU 1.0
com.apple.iokit.IOAccessoryManager 1.0.0
com.apple.driver.AppleOnboardSerial 1.0
com.apple.iokit.IOSkywalkFamily 1.0
com.apple.driver.mDNSOffloadUserClient 1.0.1b8
com.apple.iokit.IONetworkingFamily 3.4
com.apple.iokit.IOSerialFamily 11
com.apple.driver.AppleSEPManager 1.0.1
com.apple.driver.AppleA7IOP 1.0.2
com.apple.driver.IOSlaveProcessor 1
com.apple.driver.AppleBiometricSensor 2
com.apple.iokit.IOHIDFamily 2.0.0
com.apple.iokit.CoreAnalyticsFamily 1
com.apple.driver.AppleANELoadBalancer 5.35.2
com.apple.driver.AppleH11ANEInterface 5.35.0
com.apple.AUC 1.0
com.apple.iokit.IOAVFamily 1.0.0
com.apple.iokit.IOHDCPFamily 1.0.0
com.apple.iokit.IOCECFamily 1
com.apple.iokit.IOAudio2Family 1.0
com.apple.driver.AppleIISController 100.1
com.apple.driver.AppleAudioClockLibs 100.9.1
com.apple.driver.AppleM2ScalerCSCDriver 265.0.0
com.apple.iokit.IOSurface 302.11.1
com.apple.driver.IODARTFamily 1
com.apple.security.quarantine 4
com.apple.security.sandbox 300.0
com.apple.kext.AppleMatch 1.0.0d1
com.apple.driver.AppleMobileFileIntegrity 1.0.5
com.apple.security.AppleImage4 4.2.0
com.apple.kext.CoreTrust 1
com.apple.iokit.IOCryptoAcceleratorFamily 1.0.1
com.apple.driver.AppleARMPlatform 1.0.2
com.apple.iokit.IOStorageFamily 2.1
com.apple.iokit.IOSlowAdaptiveClockingFamily 1.0.0
com.apple.iokit.IOReportFamily 47
com.apple.kec.pthread 1
com.apple.kec.Libm 1
com.apple.kec.corecrypto 12.0
** Stackshot Succeeded ** Bytes Traced 51378 (Uncompressed 183984) **
Ending note
It feels like it's much likely some trouble with multi-thread handling of the M1 chip.
An interesting case would be to find two instances of the same mac model with same system & kernel version, one leading to panic and not the other.
I'll keep this post up-to-date if I were to find such case.