Elasticsearch crashes (SIGILL in libvec.so) on kNN search when the VM/hypervisor hides AVX flags

What happened

Elasticsearch 8.17.4 in Docker (OpenJDK 23, Ubuntu 20.04 container on rocky-linux host) crashed with a SIGILL in the native libvec.so library whenever a kNN search ran on a densevector field. The hserr log showed the crash inside sqr7u_2, and startup logs reported vec_caps=2 (AVX-512 capable CPU). Elastic engineers traced it to virtualization: the CPU (Intel Xeon Gold 6226) supports AVX2/AVX-512, but the hypervisor/VM did not expose those flags to the container OS, so the vector library issued instructions the environment could not execute.

The fix that worked

Check that AVX2/AVX-512 is actually exposed at every layer: bare metal, hypervisor/VM, and container (lscpu flags inside the container must show avx/avx2/avx512). If your hypervisor hides those flags, either enable them in the hypervisor/VM config or start Elasticsearch with the native vector library disabled: -Dorg.elasticsearch.nativeaccess.enableVectorLibrary=false. An Elastic maintainer confirmed the fix direction is to make Elasticsearch fall back gracefully with a warning instead of crashing when the OS-level flags are missing.

In the thread Elasticsearch crashes (SIGILL in libvec.so) on kNN search when the VM/hypervisor hides AVX flags, the question was: Elasticsearch 8.17.4 in Docker (OpenJDK 23, Ubuntu 20.04 container on rocky-linux host) crashed with a SIGILL in the native libvec.so library whenever a kNN search ran on a densevector field. The hserr log showed the crash inside sqr7u_2, and startup logs reported vec_caps=2 (AVX-512 capable CPU). Elastic engineers traced it to virtualization: the CPU (Intel Xeon Gold 6226) supports AVX2/AVX-512, but the hypervisor/VM did not expose those flags to the container OS, so the vector library issued instructions the environment could not execute. The verified resolution: Check that AVX2/AVX-512 is actually exposed at every layer: bare metal, hypervisor/VM, and container (lscpu flags inside the container must show avx/avx2/avx512). If your hypervisor hides those flags, either enable them in the hypervisor/VM config or start Elasticsearch with the native vector library disabled: -Dorg.elasticsearch.nativeaccess.enableVectorLibrary=false. An Elastic maintainer confirmed the fix direction is to make Elasticsearch fall back gracefully with a warning instead of crashing when the OS-level flags are missing. Original discussion: https://vectle.com/threads/thr_Hbr1HBHAFD8K9qXf6kYU9Q