But not with march=native?

The distirbuted binaries use two standard instruction sets for x86-64 and one for arm like “march=x86-64-v3”

https://wiki.gentoo.org/wiki/Gentoo_binhost/Available_packag...

You can have your own binary host or even just compile packages on another host on demand. -march=native is a concern in both cases.