May 7 “Let Loose” Event - new iPads

[tomO2013]

On a recent PC build - I revisited my monitor setup.

I ended up buying two high(er) end gaming monitors - BenQ Mobiuz EX270QM (1440p, 240Hz, IPS with local dimming zones and HDR600 certification).

If you don't do any gaming, in day to day (living with them) the high refresh rate is game changing for just working with text even in a code editor etc... and not having to worry about burnin from static content.

I'm personally hoping for an update to Apples studio XDR line, with a minimum of 120Hz refresh rate support. Even from the perspective of being able to edit video content and match the response rate of the device that you are targetting consumption on (e.g. a phone with promotion display) it's really great.

I think I was surprised at how much better the experience was moving from a 32" 4k BenQ @ 60Hz --> two 27" 1440p @ 240Hz (on my mac I run these at 144Hz.

I'm reading a little more about Apple OLED tech in these new iPads. Supposedly the risk of burnin goes down significantly if you aren't doing a lot of HDR content as for a given SDR brightness level each panel has to work half as hard as a more traditional OLED panel. Pretty sweet if that's the case.

 

[dada_dave]

What I find really interesting is that the Neural engine is still only 16 cores but much, much faster (we have to see at what precision of course, but then maybe that's part of it, more flexible precision!). @leman you mentioned patents for new neural engines?

Hmmm according to Tom’s the M3 (like the A17Pro) had a 36 Int8 TOPs NPU and thus the new NPU is only 5% bigger. They just changed the reporting precision (as they had done for the A17Pro).

Alleged Apple M4 Geekbench scores show incremental improvement in machine learning over last gen

Incremental improvement over last gen

www.tomshardware.com

Nevertheless, the M4 focuses heavily on AI processing, with its 16-core neural engine supporting up to 38 trillion operations per second (TOPS). In comparison, the previous-gen M3 also has a 16-core neural engine. Apple's M3 was rated for 18 TOPS at FP16 precision, but the M4 is rated for 38 TOPS with INT8. That means, if equalized to INT8 precision, we're looking at a 5% improvement in TOPS for the M4 over the M3.

I’m ignoring the benchmark since we don’t know if it’s real yet but it’s plausible if the above paragraph is accurate.

 

[amonduin]

Hmmm according to Tom’s the M3 (like the A17Pro) had a 36 Int8 TOPs NPU and thus the new NPU is only 5% bigger. They just changed the reporting precision (as they had done for the A17Pro).

Alleged Apple M4 Geekbench scores show incremental improvement in machine learning over last gen

Incremental improvement over last gen

www.tomshardware.com

I’m ignoring the benchmark since we don’t know if it’s real yet but it’s plausible if the above paragraph is accurate.

How do they know which precision Apple was talking about in this case? I was under the impression that info had been hard to come by.

 

[dada_dave]

How do they know which precision Apple was talking about in this case? I was under the impression that info had been hard to come by.

That’s why I caveated according to Tom’s. I haven’t been able to find that info either. I think they’re assuming that the A17Pro and M4 were using Int8 and the M3 was using Int16. To be fair that makes the most amount of sense and is probably right. But the paragraph reads a lot more sure of the numbers than I am at least.

 

[Cmaier]

I'm reading a little more about Apple OLED tech in these new iPads. Supposedly the risk of burnin goes down significantly if you aren't doing a lot of HDR content as for a given SDR brightness level each panel has to work half as hard as a more traditional OLED panel. Pretty sweet if that's the case.

sources tell me that each OLED is working at less than 100% even at 1600nits.

 

[theorist9]

Apple products with a display, apparently. There's also a thinner iPod Shuffle.

Which shuffle is thinner? I have the first-gen shuffle, and I measured it at 8.4 mm over the body and 8.8 mm over the ring.

And according to Apple's product specs, the 2nd - 4th gens are 10.5, 7.8, and 8.7 mm respectively. But those thicknesses include the built-in clips.

 

[Yoused]

You can use both at the same time, but performance is obviously degraded. In fact (not sure if this has changed) if the system is idle, even CoreML workloads that could run in the Neural Engine run in the GPU instead by default. It's a bit weird that there's no developer API to directly use the Neural Engine.

It is really not strange at all. Apple's ML/AI-related strategy has always been to get the job done. If other units are busy doing stuff, you distribute the workload as broadly as you can, pulling in specialized units when they go idle, but using available alternative capacity when necessary. Ergo, obfuscating the execution path behind the API is the best way to insure that you get your desired result, howsoever.

 

[tomO2013]

100% agree with you yoused.

It's also why IMHO apple products tend not to benchmark well on cross-compiled benchmarks where API calls to foundational Apple specific API's are not utilized (for code portability obvious reasons)...
you end up doing a lot of work on traditional compute and leave horsepower on the table for code portability that would otherwise be available if you created a targeted build.
I do think with Qualcomm taking a similar approach now on the Windows side that we are going to see similar considerations emerge on the windows side.

 

[Jimmyjames]

Hmmm according to Tom’s the M3 (like the A17Pro) had a 36 Int8 TOPs NPU and thus the new NPU is only 5% bigger. They just changed the reporting precision (as they had done for the A17Pro).

Alleged Apple M4 Geekbench scores show incremental improvement in machine learning over last gen

Incremental improvement over last gen

www.tomshardware.com

I’m ignoring the benchmark since we don’t know if it’s real yet but it’s plausible if the above paragraph is accurate.

As @amonduin said, and you caveated, I don’t believe Apple has made it public with regard to the precision level of the M4 NPU. I would assume FP16, but that may be incorrect.

In terms of Geekbench’s ML score, while you can request the test be run on either cpu, gpu or npu, does anyone know if it’s guaranteed to stay on the requested device? There are some strange scores on there, like the M1 iPad Pro getting ~7000 while the M2 gets ~5000. I am wondering if the os can migrate work from one device to the other?

 

[dada_dave]

As @amonduin said, and you caveated, I don’t believe Apple has made it public with regard to the precision level of the M4 NPU. I would assume FP16, but that may be incorrect.

In terms of Geekbench’s ML score, while you can request the test be run on either cpu, gpu or npu, does anyone know if it’s guaranteed to stay on the requested device? There are some strange scores on there, like the M1 iPad Pro getting ~7000 while the M2 gets ~5000. I am wondering if the os can migrate work from one device to the other?

I believe @Andropov was saying that no in fact you can’t guarantee where a program will run. I dunno

You can use both at the same time, but performance is obviously degraded. In fact (not sure if this has changed) if the system is idle, even CoreML workloads that could run in the Neural Engine run in the GPU instead by default. It's a bit weird that there's no developer API to directly use the Neural Engine.

 

[tomO2013]

As @amonduin said, and you caveated, I don’t believe Apple has made it public with regard to the precision level of the M4 NPU. I would assume FP16, but that may be incorrect.

In terms of Geekbench’s ML score, while you can request the test be run on either cpu, gpu or npu, does anyone know if it’s guaranteed to stay on the requested device? There are some strange scores on there, like the M1 iPad Pro getting ~7000 while the M2 gets ~5000. I am wondering if the os can migrate work from one device to the other?

Assuming a worst case scenario and Apple is using FP8 for M4 when comparing with FP16 on M3, the metrics reported in geekbench still show a 5-10% improvement which is nothing to be sniffed at especially when you consider M4 is running in a chassis that is likely thermally constrained.

I'm actually impressed if they pulled an extra 10% performance improvement over M3 (it's not exactly long in the tooth) and no doubt whatever other u-arch under the hood improvements to M4 to allow future scaling on desktop and a Mac Pro bodacious Ultra fusion. I suspect we will know a LOT more about M4 generation after WWDC.

 

[Jimmyjames]

No idea if it’s real but holy moly. 4.4ghz = 3767 geekbench.

iPad16,6 - Geekbench

Benchmark results for an iPad16,6 with an ARM processor.

browser.geekbench.com

 

[jbailey]

Hmmm according to Tom’s the M3 (like the A17Pro) had a 36 Int8 TOPs NPU and thus the new NPU is only 5% bigger. They just changed the reporting precision (as they had done for the A17Pro).

Alleged Apple M4 Geekbench scores show incremental improvement in machine learning over last gen

Incremental improvement over last gen

www.tomshardware.com

I’m ignoring the benchmark since we don’t know if it’s real yet but it’s plausible if the above paragraph is accurate.

On my MacBook Air M3:

CPU: 4197
GPU: 5444
NPU: 8079

On my iPhone 15 Pro (A17Pro):
CPU: 4071
GPU: 3650
NPU: 5996

I'm not sure what I can infer from this but it certainly looks like the NPU on the M3 is at least as good as the A17. Maybe the cross-platform isn't comparable yet? And for the CPU scores, is that the AMX that makes both the M3 and A17 so similar?

 

[Jimmyjames]

GB Compute score = 53792

iPad16,6 - Geekbench

Benchmark results for an iPad16,6 with an ARM processor.

browser.geekbench.com

 

[Cmaier]

GB Compute score = 53792

iPad16,6 - Geekbench

Benchmark results for an iPad16,6 with an ARM processor.

browser.geekbench.com

15% improvement?

 

[Jimmyjames]

15% improvement?

Yes. Around 20% from M3 to M4 for cpu, which I think it outstanding.

 

[quarkysg]

Yes. Around 20% from M3 to M4 for cpu, which I think it outstanding.

It could be clocked higher for Macs.

 

[Jimmyjames]

It could be clocked higher for Macs.

Oh yes! Maybe 3900 for a MacBook Pro16.

 

[Jimmyjames]

M4 may have SME instead of AMX
https://mast.hpc.social/@fclc/112407100533912023

 

[Jimmyjames]

Even better single core. 3810!!

iPad16,6 - Geekbench

Benchmark results for an iPad16,6 with an ARM processor.

browser.geekbench.com