New ML framework from Apple called MLX

Another interesting feature request for complex64 metal gemm...

github.com

Add Gemm for complex64 · Issue #140 · ml-explore/mlx

Not sure if this is an issue of mine, but I'm unable to use mx.matmul or use the @ operator on two mx.arrays, it throws the following error: ` libc++abi: terminating due to uncaught exception of ty...
github.com github.com

A different MPS backend isn't a huge tradeoff (and probably preferable as it will be more stable today in general - I wouldn't really look to MLX for anything mission critical important right now - more something I'd like to tinker with). The performance differential between MLX and MPS isn't as huge on M3 owing to the new chip improvements as it would be with other M1/M2 silicon that if you're using MLS anyway, you're not really missing out.
 
tomO2013 said:
I wouldn't really look to MLX for anything mission critical important right now
Click to expand...

Emphasis added.

I can’t imagine Apple releasing a framework like MLX for M1 and M2 improvements alone. I know it seems that way right now, but that’s where the puck has been. No, they have bigger plans. Obviously, MLX is in its infancy. We’ve yet to see the M3 Ultra and the M4 will undoubtedly debut in 2024. I expect MLX to have some major advances in the near future to compliment the Silicon releases. Remember… “Where the puck will be. Not where it’s been”. Just a hunch.
 
Someone has compiled a list of benchmark scores for MLX vs CPU vs MPS.
https://github.com/TristanBilot/mlx-benchmark

M1 Pro

Operationmlx_gpumlx_cpumpscpumps/mlx_gpu speedupmlx_cpu/mlx_gpu speedup
MatMul15.4636.6923.11593.400.491.37
Softmax3.9942.106.7632.290.699.55
Linear13.3135.3132.7698.051.461.65
Conv2d63.722505.3910.21125.39-0.8438.32
BCE4.2928.798.508.490.985.71
Concat6.3190.506.4343.790.0213.35

Some nice improvements except from Conv2d and Concat.
 
MLX seems to iterate quite rapidly. Some nice GEMM improvements in the latest release.
Some benchmarks here: https://github.com/ml-explore/mlx/pull/424#issuecomment-1898815724
1705618322825.png

1705618375557.png
 
tomO2013 said:
Another interesting feature request for complex64 metal gemm...

github.com

Add Gemm for complex64 · Issue #140 · ml-explore/mlx

Not sure if this is an issue of mine, but I'm unable to use mx.matmul or use the @ operator on two mx.arrays, it throws the following error: ` libc++abi: terminating due to uncaught exception of ty...
github.com github.com

A different MPS backend isn't a huge tradeoff (and probably preferable as it will be more stable today in general - I wouldn't really look to MLX for anything mission critical important right now - more something I'd like to tinker with). The performance differential between MLX and MPS isn't as huge on M3 owing to the new chip improvements as it would be with other M1/M2 silicon that if you're using MLS anyway, you're not really missing out.
Click to expand...
It’s alpha (breaking changes will occur for the good of humanity), but entirely useable and quite performant. There are also still some performance cliffs, so I’d expect perf to improve significantly.

Also, they’re looking to hire kernel engineers 🙂


(Awni is the project lead, with three other Apple engineers and quite a few active contributors; they’re one of the most pleasant group of open source developers I’ve come across)
 
Last edited:
Altaic said:
It’s alpha (breaking changes will occur for the good of humanity), but entirely useable and quite performant. There are also still some performance cliffs, so I’d expect perf to improve significantly.

Also, they’re looking to hire kernel engineers 🙂


(Awni is the project lead, with three other Apple engineers and quite a few active contributors; they’re one of the most pleasant group of open source developers I’ve come across)
Click to expand...
Agreed - they are awesome. Super friendly and no attitude. That’s a huge contributing factor to making this a success.
 
Back
Top