So the 96GB of RAM only on the top tier Max die could be to save cost. If they have to put the RAM on the silicon package it can be harder to predict how many packages to make with how much memory on which level of cut-down Max. If you say 96 only on Max you eliminate one whole chip configuration from manufacturing. It's a bit harder to separate RAM from SoC than it used to be so the operational cost of more configs can be higher.
While doing a high end M2 Pro Mac mini gets near a Mac Studio in cost, I think it's more complicated to compare them than just saying the Studio is better. If your work is generally purely CPU bound and you don't require extra I/O from the Studio then the M2 Pro should work better for that than the M1 Max. Anything that uses the Neural Engine (whatever serious work that might be...) would also run better on the M2 Pro. Though anything using the GPU cores would likely be better off with the Studio and M1 Max.
As for the die shots provided by Apple, how trustworthy do we think they are in reflecting the true nature of the chip? I'd like to see one actually de-lidded. Would having this peculiar 19 core GPU thing not make it significantly harder to design? I mean just it being an odd number I would think would make it hard to wire up to caches and memory. But I don't know, it may not really be too different to building a 20 core chip and fusing it off in the design complexity; Not really my field. Just don't think I've ever seen a chip that actually has an uneven number of cores, be it CPU or GPU, without it being a case of them physically being there but being fused off.
Sure, storage cards and the I/O make sense, and it’s good to know that there are a bunch that work out of the box. But what people want are GPUs, presumably, and just adding a few slots for storage and I/O doesn’t really give the Mac Pro a reason to exist that couldn’t more easily be served by TB expansion.
There's also things like hardware keys like iLok that act as software license keys in hardware. Very common for music software. I know that people like to put those inside of Mac Pros even though you could get an external setup working for that.
Do people still connect devices like this with fibre channel cards?
Afterburner 2.0 is a possibility.
Apple already made MPX modules a thing, and while it is effectively just PCIe+ it still took RnD and I could see them continue on with MPX modules, perhaps with only MPX capable slots and none that are pure PCIe.
On the x86 side of the fence there's also work to make RAM in CXL form, which is effectively building on top of PCIe. I could see a situation where this could be done for Mac Pro as a form of tiered RAM. You have your caches, L1, L2, SLC, then RAM then CXL RAM, then SSD swap.
People never seemed to like Fusion Drives much but the basic ideas of tiering things out like that for fast but small and slower but bigger has always made sense in computing and extending it with another layer could have meaningful improvements especially on a design like Apple Silicon where we will presumably never get 2019 Mac Pro levels of SoC packaged RAM.
I'm daydreaming with possibilities and speculations that are currently without too much basis here. Ultimately we can't really know much for certain right now.
But as others have mentioned too, there could be fun finds from the Asahi project directing M2 Max that could reveal aspects of future possibilities. IIRC the memory controller for M1 Max was also already quite a bit over-spec for what even the M1 Ultra was capable of. 512GB limit IIRC
