#### What is the purpose of this change?
In short there are two parts:
- One… part is making the build and publish of ARMv5/ARMv6 builds more accurate/correct: Make the existing ARMv6 build into a more correct ARMv5 build, and add ARMv6 as a new build to complement ARMv5 and ARMv7.
- The other part is making the ARM world of rclone a little less confusing, make the version command report more relevant information, and the selfupdate command and install.sh script handle the different variants more correct.
Some background:
When building 32-bit ARM binaries in go, in addition to `GOARCH=arm`, the `GOARM` variable can be set to value 5, 6 or 7 to specify compatibility level, or "ARM architectural family": ARMv5, ARMv6 or ARMv7. The decisive factor is not really the actual ARM architecture version of the processor, it is the floating point hardware support (or lack of such). Quote from https://go.dev/doc/install/source#environment regarding GOARM variable:
> This sets the ARM floating point co-processor architecture version the run-time should target. If you are compiling on the target system, its value will be auto-detected.
Specifically, if the processor supports the `VFPv3` instruction set it can be built with `GOARM=7` (ARMv7 compatibility), or if only `VFPv1` is supported `GOARM=6` (ARMv6 compatibility). If no VFP instruction sets are supported it must be built with `GOARM=5` (ARMv5 compatibility). ARMv5 is the lowest supported by go, ARMv8 is 64-bit ARM architecture, aka AArch64, and have a separate `GOARCH=arm64` and therefore `GOARM` is not relevant, hence `GOARM` can have value `5`, `6` or `7`. This PR is about 32-bit ARM builds, i.e. `GOARCH=arm`, and considering different values of `GOARM`.
##### rclone arm builds
There is a lot of confusion around ARM 32-bit architectures, in general, in golang, and in rclone... (I don't *think* it is just in my head...)
Officially, rclone provides ARMv6 and ARMv7 builds, here: https://rclone.org/downloads.
The ARMv6 downloads have name suffix "-arm", e.g. "https://downloads.rclone.org/v1.60.0/rclone-v1.60.0-linux-arm.zip".
The ARMv7 downloads have name suffix "-arm-v7", e.g. "https://downloads.rclone.org/v1.60.0/rclone-v1.60.0-linux-arm-v7.zip".
The ARMv7 build is built with GOARM=7: ✔️ https://github.com/rclone/rclone/blob/7042a11875b6011f7cd5e847c6d14c19b89979e5/bin/cross-compile.go#L86
The ARMv6 build is built without setting GOARM: ❌
This is actually wrong! It seems to be a correct assumption according to the go documentation: "GOARM=6: use VFPv1 only; default if cross compiling" (https://go.dev/doc/install/source#environment) - but in reality it is not! One proof is the following parts of the go source:
https://github.com/golang/go/blob/661e931dd120484cc1318792bc93ace00bbde088/src/cmd/dist/build.go#L140-L144
https://github.com/golang/go/blob/661e931dd120484cc1318792bc93ace00bbde088/src/cmd/dist/util.go#L392-L422
And another proof is to test a source build locally, on a non-ARM system:
```bash
$ GOARCH=arm go build -o rclone-arm
$ GOARCH=arm GOARM=5 go build -o rclone-armv5
$ GOARCH=arm GOARM=6 go build -o rclone-armv6
$ GOARCH=arm GOARM=7 go build -o rclone-armv7
$ sha1sum rclone-arm
4f5941fbf9c817a52185adf4e2b90d63d0420381 rclone-arm
$ sha1sum rclone-armv5
4f5941fbf9c817a52185adf4e2b90d63d0420381 rclone-armv5
$ sha1sum rclone-armv6
32164b308d54277e466a84363692758ff8cf2a23 rclone-armv6
$ sha1sum rclone-armv7
af9721ae9d8cdb1ffe0ac24246df45715d87a245 rclone-armv7
```
This was also discussed in a previous rclone issue https://github.com/rclone/rclone/issues/4553, where the discussion in short progressed something like this, ending with the same conclusion:
> And as far as I can tell, the default is to build for ARMv6 and always has been.
> I think this must mean that leaving GOARM unset is doing an ARMv5 build when cross compiling not an ARMv6 build as we thought.
> We were wrong: the default with go 1.14 is to build for ARMv5, or at least from an Intel Linux"
So: The binaries rclone currently publishes as ARMv6 are actually ARMv5 (no hardfloat / softfloat) builds!
In addition to binary archive releases, rclone also releases distro packages in format .deb and .rpm. These are created using goreleaser's nFPM feature, packing the same rclone binaries as published as binary archive releases. The build without setting GOARM is packaged with nFPM arch parameter set to "arm6", and the GOARM=7 build with arch set to "arm7":
https://github.com/rclone/rclone/blob/7042a11875b6011f7cd5e847c6d14c19b89979e5/bin/cross-compile.go#L89-L94
For the .deb package, both variants are actually mapped to the same Debian architecture `armhf`, which in reality is a ARMv7 (VFPv3 hardfloat) architecture: https://github.com/goreleaser/nfpm/blob/50853d3ecdeeb8dbbe0c39c6ed9f6919bfe50b34/deb/deb.go#L35-L45. So rclone publishes one .deb file labelled ARMv6, which is in reality an ARMv5 binary in an ARMv7 .deb package ❗ The other .deb file, labelled ARMv7, is the actual ARMv7 binary in a matching ARMv7 .deb package. Edit: There is a related issue: https://github.com/rclone/rclone/issues/5107.
For the .rpm package, the "arm6" variant is mapped to architecture armv6hl and "arm7" to "armv7hl":
https://github.com/goreleaser/nfpm/blob/50853d3ecdeeb8dbbe0c39c6ed9f6919bfe50b34/rpm/rpm.go#L56-L69. So in this case the published .rpm labelled ARMv6 is an ARMv5 binary in an ARMv6 .rpm package, while the ARMv7 is correct.
So how to fix this?
First of all, I think its best to explicitely set GOARM value, not rely on some default - which is also the recommended approach (https://github.com/golang/go/wiki/GoArm):
> In cross compilation situations, it is recommended that you always set an appropriate GOARM value along with GOARCH.
Second; which versions do we need? Obviously ARMv7. Keeping the ARMv5 builds would be nice, I think. Probably not many out there, but I know of an embedded device which have an ARMv7 cpu that lacks floating point hardware - which requires an ARMv5 build in go (GOARM=5). Though, Entware does publish a variant of rclone built with GOARM=5, called "rclone_nohf", so a compatible build can be installed using the opkg package manager (unofficial, but listed [here](https://rclone.org/install/#package-manager), by the way). Anyway, due to the ARMv5/v6 issue described above, removing the ARMv5 build could be a breaking change for users now, although it is documented as ARMv6 it may be installed on systems where only a ARMv5 is in fact working. Ok, so ARMv5 and ARMv7 then. But we could also add a ARMv6 build, to have the complete set. Another argument could be the .deb/.rpm package changes mentioned above, some users might miss the "armv6hl" rpm package we had before, although it then contained an ARMv5 compatible binary, they may want still have an "armv6hl" rpm package - though it would now contain an ARMv6 binary and have a different file name. So, reduced risk of breakage. With the other changes in this PR there should be less confusion separating these variants, e.g. with rclone version and selfupdate commands. But still: This may lead to some distruption, due to the changed filenames and the changes in .rpm and .deb packages!? ⚠️
We could also ensure all published file have version-specific suffix, like "-arm-v7", and not one with only "-arm", to reduce confusion. The rclone.org/downloads page documents what the "-arm" suffix file today is, but it could still lead to some confusion, and if only looking at the github release artifacts then the filename is all one has to go with. One reason not to do this, is to reduce risk of breaking existing users workflow. If this is a concern, maybe we need to keep the existing downloads as much as possible same as before, i.e. those with suffix "-arm" are ARMv5 builds - but now correctly labelled as such on the download page.
So, with this PR I suggest the following:
- Keep building ARMv5 builds and publishing them as "-arm" downloads as before (more or less).
- Explicitely set GOARM=5 while building, instead of not setting GOARM and relying on default as was done before.
- Identify it as "arm5" instead of "arm6" when creating distro packages. For .deb this means packages will be built for "armel" ports, instead of "armhf", and for .rpm this means the packages will change from being "armv6hl" to "armv5tel", while in both cases the contained binary is the same ARMv5 build as before. ⚠️
- Properly label them as ARMv5 in the download page. Also mention this being a "no hardfloat" or "softfloat" compatible build, e.g. compatible with ARMv7 architectures that does not have proper VFP hardware floating point support.
- For binary/archive downloads this is simply a relabelling in the download page to identify it with the correct architecture ARMv5, where it before incorrectly was labelled ARMv6, but for the .deb and .rpm packages the package architecture will be changed, though, as described above.
- How much of a breaking change is this, really ❓
- Build and publish "-arm-v6" variants as ARMv6 builds.
- This is a completely new set of builds, although it will be presented on the download page as ARMv6 which we also seemingly had before.
- Build and publish "-arm-v7" variants as ARMv7 builds.
- No change.
##### rclone version
Currently the `rclone version` command reports the value from `runtime.GOARCH` next to the label `os/arch`, which for any ARM 32-bit architectures is the value `arm`. This has no indication regarding which version of ARM, i.e. GOARM.
Unfortunately we cannot retrieve the `GOARM` value a program was built with in runtime (though the go runtime does it internally, for example [here](https://github.com/golang/go/blob/master/src/runtime/os_linux_arm.go)), so there is no easy way to report the ARM version/compatibility level a running rclone was built with. But we can (via x/sys/cpu package) detect the floating point hardware support of the current CPU, and from that deduce the highest value of `GOARM` we can build go programs with while still being able to run on the cpu.
The presented value for `os/kernel` from rclone version command does already include the information from the command `uname -m`. This command shows the architecture of the OS/kernel currently running. In code, the third party package github.com/shirou/gopsutil reports this value as `KernelArch`. For ARM it can report values such as armv6l, armv7l, armv8l, arm64 and aarch64, which may indicate ARM version (and additional info), but it can also be ambiguous and unreliable, depending on the cpu manufacturer, Linux distribution etc. It can simply have value "arm", or it can have value "armv7l" for a processor based on ARMv7 but without floating point hardware - which means it in go needs to be built in ARMv5 compatibility mode (GOARM=5) and not ARMv7 (GOARM=7). And also the same principle applies for ARMv7 processors with VFP support but not VFPv3 - which in go needs to be built as ARMv6 (GOARM=6).
This PR introduces check on the VFP instruction set support, and adds information about the supported ARM version:
```
- os/arch: arm (ARMv7 compatible)
- os/arch: arm (ARMv6 compatible)
- os/arch: arm (ARMv5 compatible, no hardfloat)
```
Note that the ARM version described is based on the VFP instruction set, so a ARMv7 CPU without VFPv3 support will be reported as "ARMv6 compatible" if it has VFP support or "ARMv5 compatible" if neither.
Note that there is (another) possible confusion here: The suffix represents capabilities of the machine's processor, not how the running program was built. A rclone program reporting `arm (ARMv7 compatible)` may in reality be an ARMv6 build. In contrast, the value `arm`, coming from `runtime.GOARCH`, represents the target architecture of the running program. Though, at least for ARM, I guess it necessarily must match the actual hardware, or else rclone would not run at all. Anyway, the point is that this new "os/arch" value for ARM is strictly speaking a mix of different information, but in reality I think it makes enough sense to combine these two and not e.g. add a info new line instead.
The "no hardfloat" label on ARMv5 is supposed to avoid the above mentioned possible confusion a bit. Not sure if we should use the text "no hardfloat", or just "nohf" (or "NOHF"), or perhaps "softfloat", here ❓ There are many terms in use out there, so not sure which is most clear for a "regular user".. In any case, what is used here should be coordinated with the label used on the download page (described above).
Sources:
https://go.dev/src/runtime/os_linux_arm.go
https://github.com/golang/go/wiki/GoArm
https://en.wikipedia.org/wiki/Uname
https://stackoverflow.com/questions/45125516/possible-values-for-uname-m
##### rclone selfupdate
Rclone did previously publish two variants of ARM 32-bit binaries for Linux:
- For ARMv6 (or in reality ARMv5): rclone-v1.59.2-linux-**arm**.zip
- For ARMv7: rclone-v1.59.2-linux-**arm-v7**.zip
The `rclone selfupdate` command only checked `runtime.GOARCH` when deciding which download to pick. But for any ARM 32-bit architecture this has value `arm`. This means it would always pick the ARMv6 (ARMv5) download. If the user originally had the ARMv7 build, running selfupdate would "downgrade" it. It wouldn't stop working, as the ARM builds are backwards compatible, but still - not ideal. ~If running selfupdate on a custom ARMv5 build, it would "upgrade" it to ARMv6 variant, which will not run on ARMv5 hardware.~
This PR introduces check on the VFP instruction sets, same as in the version command changes described above, and picks the "best" match. Note that this means if running the ARMv6 variant of rclone on a system which is considered compatible with ARMv7, i.e. VFPv3 floating point hardware is detected, then selfupdate will now "upgrade" the rclone installation from a ARMv6 variant to a ARMv7 variant.
##### install.sh
The install.sh script uses `uname -m` to decide if the "arm-v7", "arm-v6" or "arm" (ARMv5/softfloat) download should be used. This could perhaps be made more accurate, and in-line with selfupdate and version commands, if it looked in `cat /proc/cpuinfo` and checked for vfp in "Features" list. To do this we would have to use sed, grep or similar tools, I'm not sure if we can assume one such being present, in general?
This PR only adds support for arm-v6 to the install.sh.
#### Was the change discussed in an issue or in the forum before?
Somewhat related https://github.com/rclone/rclone/pull/6505.
#### Checklist
- [x] I have read the [contribution guidelines](https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#submitting-a-new-feature-or-bug-fix).
- [ ] I have added tests for all changes in this PR if appropriate.
- [ ] I have added documentation for the changes if appropriate.
- [x] All commit messages are in [house style](https://github.com/rclone/rclone/blob/master/CONTRIBUTING.md#commit-messages).
- [x] I'm done, this Pull Request is ready for review :-)
