I disagree. Without forward progression at thread level you cannot have fine grained control over the progress of your algorithm. You are limited to the SIMD group as the source of your parallelism but the speed up of the GPU is that you can use parallel execution of the SIMD group itself to achieve parallelism - that’s a factor of 32 for both Nvidia and Apple GPUs! Without that you are limited to having an entire SIMD group dedicated to a single control flow, great for divergence but not so great for overall performance making a huge class of algorithms simply out of reach or hellishly complicated. That’s what Bryce and
Olivier’s talks focused on - Olivier who is now at Apple.
In fact I think you have it reversed: being able to coordinate amongst thread group blocks is nice to have. It enables things like single pass scans which are faster than the standard two pass scans. It means that you sometimes don’t have to launch multiple kernels and can reduce your overhead - that’s the example given by Nvidia. Certain Atomics become available and more germane to Nvidia than Apple perhaps, multiple GPUs can be coordinated more easily together. There’s good reason why developers should care. I agree. But fundamentally they are the same class of algorithms as before. Without forward progress at the thread level though huge classes of algorithms, basically anything requiring complex synchronization through mutexes and locks like hash tables and linked lists are simply out of practical reach on the GPU (other use cases as well). They are either incredibly cumbersome to program or simply not performant enough to warrant their use as your forced into lock-free or SIMD group aware versions. I’ll see if I get track down Bryce’s talk and there was another guy as well who goes over the “why you should care”. But if memory serves Olivier’s talk linked to by
@Jimmyjames does a really decent job of explaining it as well and emphasizing just how difficult an engineering challenge it was to achieve and why they bothered to do it.
Now I’ll admit for my own work I don’t make use of this mutex capability (yet), but I almost certainly rely (through a library) on the single pass scan technique. But one of the algorithms I’d love to work on, the CPU version is filled with these kinds of control flows mostly in the use of hash tables. Now I believe people had worked on hash tables prior to this development but it’s either Bryce’s or Olivier’s talk or both that explains why this new system is so very much nicer for that in particular.
