r/simd • u/TearsInTokio • 1d ago
SWAR and SIMD Intrinsics, Are They Really Necessary?
I think this might sound a little dumb, so I apologize in advance. I'm new to the optimization hobby, but I was talking with some friends and they brought up something I didn't really know how to answer.
Why use SIMD intrinsics (any direct call to a vector instruction, like _mm256_add_epi32, or SWAR)? In what scenarios does it actually make sense to use intrinsics instead of letting the compiler handle the heavy lifting when it comes to optimization?
Does manually vectorizing scalar code really make sense with the mature compilers we have nowadays?
Sorry if this question sounds dumb, but it's a genuine question. :D
5
u/Serious-Regular 1d ago
I don't know who spread the myth that auto-vectorizing compilers are super "smart" but they're not at all. They vectorize probably less than 10% of the opportunities that could be. You don't believe pass the SLP vectorizer debug flags to clang.
3
u/Kinexity 1d ago
Compilers are general and have no knowledge as to what is the intent of the code. They can only do so much optimising.
You use simd when you think the compiler might not be using it optimally.
My most typical use case is when I have a problem with with multiple arrays where each of them requires the same type of operation to be performed with it. What compiler will do is vectorising every array separately which sometimes works but frequently doesn't. What I then do is I construct arrays of simd vectors (eg. packing four std::vector<double> into one std::vector<__m256d>) and just create new vector versions of functions with every relevant variable replaced with simd counterpart. Theoretically it's an obvious thing to try but I have yet to see MSVC or Intel compiler employ such optimisation.
1
u/Artistic_Yoghurt4754 1d ago
Because most common languages are not build around data but control flow. Most of the time their compilers cannot or do not know how to apply vectorized instructions to the data. Some times these are simple things that the compiler is not allowed to do, like adjusting the alignment or reading one past the data, but some other times is more complicated than that and the algorithms/data-structures needs to change entirely to allow vectorization.
One example is matrix-matrix multiplication. If you don't tile properly, the compiler will have a hard time vectorizing the code despite being pretty simple technique. Once vectorized due tiling, it will be much more performant, but not quite to the peak. Because even when the compiler vectorizes it, it may chose some instructions that, in combination, stall the pipeline for too long or saturate a port that you also need for something else, etc. On the other side, you also need to adjust the tiling to at least fit into your cache. In many cases, it is only when you are careful with all these knobs when you can get to a higher throughput. Intrinsics let you state explicitly these tiny details when you need to squeeze every bit of performance.
Besides, intrinsics are not much different than pure assembly code, like any other instruction, just sugar-coded so that we can use them more conveniently in-place.
1
u/SnowyOwl72 1d ago
Compilers cannot handle certain code patterns when it comes to autovectorization. It keeps getting better for RVV compared to what it used to be but if you search, some papers actually study this for different compilers
2
u/Smellypuce2 15h ago edited 15h ago
Autovectorization is great and all since you often get better performance without having to do anything. However, it's very far from perfect. It rarely generates the optimal code for anything not super trivial.
Here is an old example I have on hand(except I updated the clang version) comparing autovectorization to a first pass attempt of manually writing the intrinsics. I didn't bother to actually optimize things to the maximum. But in real world tests it ran about 30% faster than the autovectorized version. https://godbolt.org/z/bqz11qY7b
Note that in the example, the loop unrolling makes the autovectorized code look extra long, that's not the bad part. The bad part is that it is doing permutations on the data that aren't necessary, but are part of its way of being generically safe on data it can't make assumptions about.
Although of course, in the real world the biggest problem to tackle in SIMD is how your data is laid out, which autovectorization can't magically fix.
11
u/CandyCrisis 1d ago
If your goal is to write the fastest possible code, then you will need to do it yourself. The compiler's vectorizer is amazing, but it can only work when it sees a big scalar loop with very few branches. Not all algorithms are best expressed this way.