Regarding dedicated OS cores — I can certainly see a dedicated low-latency ordered control queue to facilitate inter-core syscalls, that should be fairly cheap to implement given the presumably low bandwidth requirements. What I do wonder about are three things:
- Cost of cache synchronization between dedicated OS cores and client cores (sounds expensive!)
I think this mostly doesn't need to happen... though I'm waving my hands quite furiously here. The theory is that most data-heavy calls talk to the OS core for control but it in turn passes pointers to a userspace process which is running back on the same user CPU for actual data movement. Thus my reference to a split driver and zero-copy. My vague idea was to have the NoC have design support for syscalls, such that a limited amount of arguments can be passed directly over the NoC rather than being referred to as memory addresses which are in the cache. It seems that this exists already, give or take (inter-core fifos for registers).
Hm. Thinking about this more... I just realized something significant. This model means that ONLY the OS cluster is writing the TLB! That means that all invalidations are generated only on that cluster, and can be pushed cheaply to all other clusters. This drastically simplifies cache design, and reduces latency.
So in fact I think this design would find cache sync much cheaper than typical designs.
Also, this changes my earlier response. I think it's enough of a benefit that the 1-OS-core-per-cluster alternative is completely outclassed, and probably not worth considering.
- What actually happens on the client core when a syscall is emitted — since the execution is asynchronous, the thread either needs to wait or relinquish control until the syscall is processed. For this to be efficient, one needs some sort of scheduling apparatus running on the core itself
First sentence, exactly right. Second, I don't think so. Perhaps every syscall is in fact a sleep instruction that indicates "wait for a message from the OS core with a new address to start executing from" (I mentioned this earlier; it would also specify which register types would be restored, and from where). Sometimes, the response from the OS means "resume now without context switch" and it does instead of yielding. And of course some (probably more and more over time, if this idea is successful) return immediately without sleeping, as they're totally async. This is entirely implemented in hw, there's no OS overseeing it.
- How are memory page faults handled? Are they trapped on the client core and forwarded to the OS core via the syscall queue?
For a TLB miss, that's the potential performance problem, going across the NoC to the OS Cores. But... why would you have to do that? The HW page walker exists in every core. Let it do its thing, and return, while holding the thread suspended (or not even, as it can execute out of order).
For a real fault, the latency between clusters, on the order of 100ns, will be entirely hidden by the latency to disk which is more than two orders of magnitude larger. (Or, for a minor fault, by the latency of the page copy/zero-fill/whatever.)
Intuitively, I'd say that it could be quite hard to balance (how much resources do the OS cores need to process the syscall queue, and what happens in edge cases?), dealing with caches might be a problem, and will introduce a whole new level of complexity in handling context switches and exceptions. There are certainly cases where asynchronous handling of requests is preferable, and we already have dedicated hardware (e.g. DMA transfers) doing that.
Yes, balance seems like a question you can only answer with massive telemetry. Fortunately, Apple should have that, so they can probably know how many cores you'd need for this to work well (and thus whether it would be cost-effective). However there's some literature saying 5-10% of compute resources are typically sufficient, assuming all the heavy driver lifting is done in userspace. On the high end of that, or maybe a little bit more, for some IO-heavy tasks like RDBMSes. That would make 2 cores plenty for even the M5 max.
I suspect but can't prove (or even argue in detail) that you'd win more than you lose on caching, context switches, and exceptions. As the world goes more and more multiprocessor, code gets more and more multiprocessing-based and more and more async, which I think would work well with this sort of CPU.
BTW, if I remember correctly, Apple was already using some semi-custom stuff to handle certain traps without incurring a context switch. Asahi people described a special execution mode that hides client registers and allows the OS code to execute in a shadow space. To me this kind of solution (dedicated register/cache spaces for OS code rather than dedicated cores) sounds preferable as it avoids the headache of synchronization and can't be beat in terms of latency.
Assuming I know what you're talking about (User Mode Interrupts, and hw thread switching, for drivers to execute, not the OS itself), that is extensively used in this design, exactly as it is now. It's important for this design because it means user code can talk directly to the driver without the OS intermediating, so you don't have those long-latency syscalls every time you read or write.
This was actually one of the things that got me wondering if Apple might be headed in this direction, because no-copy no-kernel drivers are foundational if you're going to do it.
also not certain on performance consequences in practice. I can think of arguments both for why it might be fast and why it might be slow and not sure which would weigh the strongest.
