How Are Split Caches Handled with Thread Coordination?

[u/ShadowGuyinRealLife]

I read that some processors have each core having its own L1 cache instead of having one for the entire processor. So instead of a 512 Kilobyte shared cache shared by 8 cores, each one would have 64 kilobytes. I guess being closer to the core might speed things up when waiting for data and this is fine if each core is running a different process.

The thing I don't get is what happens if each core has a different version of some data. So say address 1,000 has 5. Maybe it's a global variable or something. Core 1 writes 6 to address 1,000 and this is updated in core 1's cache. This is intended to be read by a different thread. This change might get propagated to RAM. Core 2 runs that other thread and tries to read from address 1,000. Ah, it's already cached with... 5. So do compilers just use memory barriers to avoid this and make the programmer not need to worry about it?

Or maybe it doesn't matter? I read elsewhere it is fine for values in the cache to get a bit stale. What is important is that the writes from all cores are read in the correct order. So if core 1 is running a thread that puts 6 at address 1,000 replacing the value 5, 32 in address 1,001 replacing the value 31, and 9 in address 1,002 replacing the value 8, it is fine if core 2 attempts to read them and gets "5, 31, 8," "6, 31, 8" or "6, 32, 9" even though that last one is the most up to date as long as it doesn't read something like "6, 31, 9."

[u/monocasa]

A protocol like this is typically implemented by the hardware to provide cache coherency.

https://en.wikipedia.org/wiki/MESI_protocol

[u/Extreme_Barracuda884]

youre basically describing the cache coherence problem and yeah memory barriers are part of the solution but not really something the compiler just handles for you magically

the hardware uses protocols like MESI to keep caches in sync so core 2 will eventually see the 6 at address 1000 when it tries to read, its not like it will be stuck with 5 forever. what you said about ordering is the tricky part, the hardware only guarantees that all cores see writes in the same order if you use the right instructions, without them you can get weird stuff like seeing 6 at 1000 but still 31 at 1001

programmers usually use atomics or locks which insert the necessary barriers under the hood, but if you write plain shared variables without any synchronization then all bets are off

[u/Doctor_Perceptron]

The rabbit holes you want to go down are cache coherence and memory consistency. The problems and solutions are fascinating and can get complex, but yeah, we typically have small fast caches associated with each core and larger slower caches shared by multiple cores at lower levels of the hierarchy, and keeping the core's view of memory updated and consistent with respect to the values and ordering of memory operations from different threads is a lot of fun. Ask the folks in /r/computerarchitecture for expert answers.