Apple to use it’s own server chips

mr_roboto said:
Are you suggesting Linux on Apple Silicon? To me, it's something of a given that once they committed to AS hardware, they'd use a Darwin-based OS. Linux on AS is too rough to use at scale like this. They'd have a lot of extra work to do on fully integrating key AS features and security hardening.
Click to expand...

No. I’m suggesting that doing AS in the cloud is a bunch of custom work to get access to a specific compute resource. And that Apple has a lot of experience at this point managing non-Apple hardware in the datacenter that it’s not completely out of line to just deploy onto more common server hardware.

Again, I’m not suggesting Apple wouldn’t use AS, just that I don’t think it’s a slam dunk proposition.

As I think about it though, I wonder how much they can build on top of whatever they did to enable Xcode cloud…
 
dada_dave said:
Interestingly Apple may be using its own chips for running the new intelligence data centers, but for training Apple Intelligence they used Google TPUs:

www.tomshardware.com

Apple skips Nvidia's GPUs for its AI models, uses thousands of Google TPUs instead

Recently released research paper reveals the details.
www.tomshardware.com www.tomshardware.com

The research paper linked to in the article is really cool:

Click to expand...
I haven’t read the paper yet. I wonder if they choose Google’s TPUs due to better performance or is it just to spite Nvidia?
 
Jimmyjames said:
I haven’t read the paper yet. I wonder if they choose Google’s TPUs due to better performance or is it just to spite Nvidia?
Click to expand...
I'll be honest I'm not sure how the TPU compares with efficiency and performance relative to Nvidia. I know that Apple and Nvidia don't get along but I can't say if that was a factor here or it was simply down to more prosaic issues: cost, availability, and the efficiency/performance for what Apple wanted to do.
 
dada_dave said:
I'll be honest I'm not sure how the TPU compares with efficiency and performance relative to Nvidia. I know that Apple and Nvidia don't get along but I can't say if that was a factor here or it was simply down to more prosaic issues: cost, availability, and the efficiency/performance for what Apple wanted to do.
Click to expand...

It seems like Google has some of the more integrated training solutions ready to go, where the point is to move the result to different hardware for inference. Nvidia doesn’t offer direct solutions to my knowledge, and instead points you at AWS, Azure and others. And Azure would certainly like you to use their framework for both training and inference.

So I think this is a case of pragmatism for the goals involved: only need training on a per-hour basis, need to run the final model on their own hardware, want to minimize how much infrastructure they have to maintain (i.e. no EC instances).
 
Back
Top