Apple Silicon + nVidia Blackwell

[Altaic]

Just came across this rather interesting development:

It apparently works via a user space eGPU driver for this USB4 to PCIe x16 adapter and I'll hazard a guess that it will soon be featured in this open source project.

A Mac Studio Ultra paired with a RTX 5090 or RTX Pro 6000 would be one hell of a machine learning system. The timing here is also serendipitous since MLX now supports CUDA (on linux), so it's conceivable that it'll soon be possible to run MLX models in macOS on heterogeneous hardware.

Wild times!

 

[dada_dave]

Just came across this rather interesting development:

View attachment 36068

It apparently works via a user space eGPU driver for this USB4 to PCIe x16 adapter and I'll hazard a guess that it will soon be featured in this open source project.

A Mac Studio Ultra paired with a RTX 5090 or RTX Pro 6000 would be one hell of a machine learning system. The timing here is also serendipitous since MLX now supports CUDA (on linux), so it's conceivable that it'll soon be possible to run MLX models in macOS on heterogeneous hardware.

Wild times!

Whoa a user side eGPU driver! I wonder what the performance is like … I’d love to just do CUDA on my Mac, no separate box needed! I thought the memory mapping was going to be a problem, very very cool! Definitely need to read more about this.

 

[Altaic]

Whoa a user side eGPU driver! I wonder what the performance is like … I’d love to just do CUDA on my Mac, no separate box needed! I thought the memory mapping was going to be a problem, very very cool! Definitely need to read more about this.

Not sure if the bulk of the logic will be in Anemll or if it’s already in tinygrad [1], since that’s what Anemll is hooking it up to. Apparently tinygrad just added user space driver capability for these cards on Linux a few days ago.

[1] https://github.com/tinygrad/tinygrad

 

[Altaic]

For those who are interested, Anemll just published a branch of tinygrad with a summary doc.