Apple: M1 vs. M2

[theorist9]

Out of curiosity what kinds of calculations are you doing in Mathematica and if another tool could do the same thing faster is that viable as an alternative for your needs?

Symbolic math. Getting symbolic solutions to integrals and differential equations; solving simultaneous sets of equations with or without domain restrictions; root finding; recursions; simplifying complex expressions. Doing those calculations with units. 2D and 3D plotting. Matlab might be better for numerical work, but the conventional wisdom (which was correct last time I checked) is that Mathematica is much stronger for symbolic work. Plus I know Mathematica very well, so it's the path of least resistance for me.

And I do consulting work for students and professionals that need help with Mathematica, so I'll need to use it for that regardless.

 

[Cmaier]

Symbolic math. Getting symbolic solutions to integrals and differential equations; solving simultaneous sets of equations with or without domain restrictions; root finding; recursions; simplifying complex expressions. Doing those calculations with units. 2D and 3D plotting. Matlab might be better for numerical work, but the conventional wisdom (which was correct last time I checked) is that Mathematica is much stronger for symbolic work. Plus I know Mathematica very well, so it's the path of least resistance for me.

And I do consulting work for students and professionals that need help with Mathematica, so I'll need to use it for that regardless.

A little unexpected to me that M1 would lag on symbolic math, since I would think the lack of optimized math libraries would be more relevant to numerical work.

 

[casperes1996]

Symbolic math. Getting symbolic solutions to integrals and differential equations; solving simultaneous sets of equations with or without domain restrictions; root finding; recursions; simplifying complex expressions. Doing those calculations with units. 2D and 3D plotting. Matlab might be better for numerical work, but the conventional wisdom (which was correct last time I checked) is that Mathematica is much stronger for symbolic work. Plus I know Mathematica very well, so it's the path of least resistance for me.

And I do consulting work for students and professionals that need help with Mathematica, so I'll need to use it for that regardless.

Right. Most be some massive sets of equations. I tend to use Maple for all that and it's all basically instant with the stuff I've thrown at it, and I have a feeling Mathematica would be faster, primarily based on Maple being a Java program, haha.
But yeah, I'd rather wait a bit longer on a computer than waiting a lot longer to learn a new tool for that sort of thing as long as we're not talking regular orders of magnitude or anything

 

[theorist9]

A little unexpected to me that M1 would lag on symbolic math, since I would think the lack of optimized math libraries would be more relevant to numerical work.

You raise a very good point. I just checked the WolframMark benchmark, and 14 of its 15 calculations are numerical rather than purely symbolic. So clearly the WolframMark benchmark isn't representative of the kind of work I do—I wasn't aware of that!

When I have a chance I'll compare the results for the one symbolic calculation (a polynomial expansion). Though since that's just one test, I eventually should find a colleague who uses Mathematica and has an M1, and ask them to run some of my calculations.

I should also emphasize I don't know the real reason(s) Mathematica is slower than expected on the M1 with WolframMark. This has been discussed on the Mathematica Stack Exchange, and the thinking there is that it's because of the libraries, but that's speculative (which is why I try to be careful with my language on this, and used the term "perhaps" when I offered that as a potential explanation).

 

[Cmaier]

You raise a very good point. I just checked the WolframMark benchmark, and 14 of its 15 calculations are numerical rather than purely symbolic. So clearly the WolframMark benchmark isn't representative of the kind of work I do—I wasn't aware of that!

When I have a chance I'll compare the results for the one symbolic calculation (a polynomial expansion). Though since that's just one test, I eventually should find a colleague who uses Mathematica and has an M1, and ask them to run some of my calculations.

I should also emphasize I don't know the real reason(s) Mathematica is slower than expected on the M1 with WolframMark. This has been discussed on the Mathematica Stack Exchange, and the thinking there is that it's because of the libraries, but that's speculative (which is why I try to be careful with my language on this, and used the term "perhaps" when I offered that as a potential explanation).

I would imagine that for purely symbolic math m1 will be fast. My understanding of the algorithms behind those kinds of problems is that it’s graph operations and table lookups, which in some ways are similar to benchmarks for things like compiling. If Mathematica is slow for that, it would be a bit of an outlier.

 

[theorist9]

I would imagine that for purely symbolic math m1 will be fast. My understanding of the algorithms behind those kinds of problems is that it’s graph operations and table lookups, which in some ways are similar to benchmarks for things like compiling. If Mathematica is slow for that, it would be a bit of an outlier.

I did more searching and was able to find a somewhat higher score than I'd seen before for the M1 (benchmark score of 3.4 rather than 3.2; my iMac scores a 4.6). This was done on an M1 Pro MBP by someone who was careful to ensure no other processes were working on the machine (e.g., Spotlight indexing). He used Mathematica 13.0.0, while I used13.0.1, but I don't think that will affect things.

I did a detailed breakdown by test. With this, a more complex picture emerges. A lot of the M1 Pro's increased run time is due to poor performance on two particular tests: a singular value decomposition and solving a linear system. The one purely symbolic test (polynomial expansion, highlighted in yellow) takes the same time on both machines—but it's so quick (0.05 s) that it might not be meaningful.

Looking at these times, I think a problem is that this benchmark is several years old, and was developed when processors were much slower. We'd get more meaningful results with a more demanding benchmark; I suspect the reason they haven't changed it is to allow historical comparisons.

 

[Cmaier]

I did more searching and was able to find a somewhat higher score than I'd seen before for the M1 (benchmark score of 3.4 rather than 3.2; my iMac scores a 4.6). This was done on an M1 Pro MBP by someone who was careful to ensure no other processes were working on the machine (e.g., Spotlight indexing). He used Mathematica 13.0.0, while I used13.0.1, but I don't think that will affect things.

I did a detailed breakdown by test. With this, a more complex picture emerges. A lot of the M1 Pro's increased run time is due to poor performance on two particular tests: a singular value decomposition and solving a linear system. The one purely symbolic test (polynomial expansion, highlighted in yellow) takes the same time on both machines—but it's so quick (0.05 s) that it might not be meaningful.

Looking at these times, I think a problem is that this benchmark is several years old, and was developed when processors were much slower. We'd get more meaningful results with a more demanding benchmark; I suspect the reason they haven't changed it is to allow historical comparisons.

View attachment 14967

I’d be very curious to see what folks are seeing with real symbolic workloads on mathematica.

 

[theorist9]

I’d be very curious to see what folks are seeing with real symbolic workloads on mathematica.

Well, if anyone wants to volunteer to install a 7-day free trial of Mathematica on an M1 Pro/Max/Ultra, I could supply some symbolic workloads to run (though don't install it yet—I would need time to put the code together).

Running it would be easy. You wouldn't need to know Mathematica. You'd just do CMD-A to select all the cells in the notebook, and then Shift-Enter to run all the cells. The output will then generate on its own.

 

[Cmaier]

Well, if anyone wants to volunteer to install a 7-day free trial of Mathematica on an M1 Pro/Max/Ultra, I could supply some symbolic workloads to run (though don't install it yet—I would need time to put the code together).

Running it would be easy. You wouldn't need to know Mathematica. You'd just do CMD-A to select all the cells in the notebook, and then Shift-Enter to run all the cells. The output will then generate on its own.

I actually went to their website and started the process of downloading it, before I realized I couldn’t remember my wolfram account password and the “i forgot my password” link wasn’t working.

If you want to gather up a test case, I’m happy to install mathematica trial on my MBP with an M1 Max, or even on one of my family’s MBA’s with an M1 (if they let me ) in the next day or two or five.

 

[theorist9]

I actually went to their website and started the process of downloading it, before I realized I couldn’t remember my wolfram account password and the “i forgot my password” link wasn’t working.

If you want to gather up a test case, I’m happy to install mathematica trial on my MBP with an M1 Max, or even on one of my family’s MBA’s with an M1 (if they let me ) in the next day or two or five.

OK, I'll let you know when I can get you that, but it might be more like a week or so; I'd like to collect a reasonable variety of tests.

 

[Cmaier]

OK, I'll let you know when I can get you that, but it might be more like a week or so; I'd like to collect a reasonable variety of tests.

No problem at all. I won’t download anything until I get the word. (assuming i can navigate their website, which is pretty awful).

 

[casperes1996]

Well, if anyone wants to volunteer to install a 7-day free trial of Mathematica on an M1 Pro/Max/Ultra, I could supply some symbolic workloads to run (though don't install it yet—I would need time to put the code together).

Running it would be easy. You wouldn't need to know Mathematica. You'd just do CMD-A to select all the cells in the notebook, and then Shift-Enter to run all the cells. The output will then generate on its own.

Like Cmaier I'm also up for running any tests you want; 16" M1 Max 24GPU 32GB RAM

 

[Andropov]

I updated a couple graphs I had with the new M2 geekbench data (single core and multi core).

Seems to me Apple is perfectly on track, neither A15 nor M2 appear to diverge from the trend. And they managed to sustain the multicore YoY improvements without increasing core count, unlike others. Also intresing how the mid-tier """efficiency""" cores Intel used in Alder Lake manage to push the multicore score to M1-Pro levels. I wonder if that's something Apple could do too (switching to a three tier core system).

 

[Cmaier]

I updated a couple graphs I had with the new M2 geekbench data (single core and multi core).

View attachment 14989View attachment 14990

Seems to me Apple is perfectly on track, neither A15 nor M2 appear to diverge from the trend. And they managed to sustain the multicore YoY improvements without increasing core count, unlike others. Also intresing how the mid-tier """efficiency""" cores Intel used in Alder Lake manage to push the multicore score to M1-Pro levels. I wonder if that's something Apple could do too (switching to a three tier core system).

I’m slightly fascinated by the dichotomy in tech sites - half of them are “M2 is a nice performance jump” and half are “apple has hit a wall, but at least the GPU is faster”

 

[theorist9]

Like Cmaier I'm also up for running any tests you want; 16" M1 Max 24GPU 32GB RAM

Great, I'll send a notebook to both of you once I assemble it.

The idea here is I want to assemble operations that will ensure Mathematica (MMA) doesn't use numerical libraries. This has gotten me thinking about what defines a symbolic vs. numerical operation when it comes to that criterion. In MMA, I consider an operation symbolic if it uses one of MMA's symbolic solvers, regardless of whether the expression on which it's operating contains numbers. For instance, it can calculate integrals symbolically using Integrate, or numerically using NIntegrate. Likewise, it can solve simultaneous equations symbolically using Solve, or numerically using NSolve. Hence I consider both of these to be symbolic operations, which would not use the numerical libraries, even though they contain numbers:

Furthermore, MMA treats exact numbers (which is what are featured above) as symbolic rather than numerical. While there are exceptions (like 1/2), base-10 non-integers typically can't be exactly represented with a finite binary. For instance, 9/10 is the infinite repeating binary 111001100110011.... However, when you enter 9/10 into MMA (as opposed to 0.9), it treats it as a symbolic entity, as evidence by the fact that you can subsequently convert it to a finite binary with whatever number of digits you please (up to the limits of the system). E.g., if you assign x = 9/10, you can subsequently convert that to a finite binary with 10^8 digits, or 10^9 digits, or 10^10 digits, and so on. Obviously when you make that assignment, MMA isn't storing a finite binary with so many digits. Instead, it is storing that value symbolically, so that subsequent numerical conversion can be done if desired. Thus when these symbolic values are manipulated in MMA (other examples would be Pi and Sqrt[2]), I don't think they would be handled by the numerical libraries. I think they are instead processed within MMA's own software.

For instance, consider this operation:

Here the value of z is stored symbolically, since it can be subsequently be expressed numerically with any degree of precision you like (say 10^10 digits). The fact that this can be done would seem to indicate this is not processed by a numerical library, since (I assume) numerical libraries aren't designed to handle symbolics, and would instead need these to first be converted to some type of float, which means that, once the operation was completed, z would be limited in its precision by the type of float used.

Thus I assume something like this, which uses symbolic numbers only, should not use the numerical libraries either:

BUT: I'm not a MMA developer, so these are just my inferences. Any thoughts on whether what I've said is correct, or if I need to limit myself to expressions that contain no numbers at all in order to ensure the numerical libraries aren't used (other than powers, since x^2 is just x*x)?

[Note: I'm using "precision" to mean the number of digits, since that's (more or less) MMA's convention for the word. I know that's not the standard usage of "precision".]

 

[Cmaier]

Great, I'll send a notebook to both of you once I assemble it.

The idea here is I want to assemble operations that will ensure Mathematica (MMA) doesn't use numerical libraries. This has gotten me thinking about what defines a symbolic vs. numerical operation when it comes to that criterion. In MMA, I consider an operation symbolic if it uses one of MMA's symbolic solvers, regardless of whether the expression on which it's operating contains numbers. For instance, it can calculate integrals symbolically using Integrate, or numerically using NIntegrate. Likewise, it can solve simultaneous equations symbolically using Solve, or numerically using NSolve. Hence I consider both of these to be symbolic operations, which would not use the numerical libraries, even though they contain numbers:

View attachment 15001
Furthermore, MMA treats exact numbers (which is what are featured above) as symbolic rather than numerical. While there are exceptions (like 1/2), base-10 non-integers typically can't be exactly represented with a finite binary. For instance, 9/10 is the infinite repeating binary 111001100110011.... However, when you enter 9/10 into MMA (as opposed to 0.9), it treats it as a symbolic entity, as evidence by the fact that you can subsequently convert it to a finite binary with whatever number of digits you please (up to the limits of the system). E.g., if you assign x = 9/10, you can subsequently convert that to a finite binary with 10^8 digits, or 10^9 digits, or 10^10 digits, and so on. Obviously when you make that assignment, MMA isn't storing a finite binary with so many digits. Instead, it is storing that value symbolically, so that subsequent numerical conversion can be done if desired. Thus when these symbolic values are manipulated in MMA (other examples would be Pi and Sqrt[2]), I don't think they would be handled by the numerical libraries. I think they are instead processed within MMA's own software.

For instance, consider this operation:
View attachment 15006
Here the value of z is stored symbolically, since it can be subsequently be expressed numerically with any degree of precision you like (say 10^10 digits). The fact that this can be done would seem to indicate this is not processed by a numerical library, since (I assume) numerical libraries aren't designed to handle symbolics, and would instead need these to first be converted to some type of float, which means that, once the operation was completed, z would be limited in its precision by the type of float used.

Thus I assume something like this, which uses symbolic numbers only, should not use the numerical libraries either:

View attachment 15002

BUT: I'm not a MMA developer, so these are just my inferences. Any thoughts on whether what I've said is correct, or if I need to limit myself to expressions that contain no numbers at all in order to ensure the numerical libraries aren't used (other than powers, since x^2 is just x*x)?

[Note: I'm using "precision" to mean the number of digits, since that's (more or less) MMA's convention for the word. I know that's not the standard usage of "precision".]

I think you are okay with numbers in the way you have listed above. These are all either still solved symbolically, or also do things like table lookups that shouldn’t require fancy math libraries.

 

[Colstan]

What I find particularly striking is that the M2 is now beating the Mac Pro in Geekbench, not just almost doubling it in single-core, but besting the Mac Pro in multi-core, as well:

M2:

Single-core: 1919
Multi-core: 8929

8-Core Mac Pro:

Single-Core: 1016
Multi-Core: 8027

You have to upgrade to the 12-core Mac Pro, starting at $7,000, to beat the M2 in multi-core. Tell somebody this two years ago and it would be pure insanity for a MacBook to best the Mac Pro in multi-core performance and nearly double it in single-core.

Also, according to the Geekbench Metal benchmark, the M2 GPU is about 50% faster than the M1. The M1 scored 20440, while the M2 is 30627, which is similar to the GFXBench results from earlier in this thread. I have a suspicion that the M2 is a case of Apple under-promising, while over-delivering.

As a side note, this thread is a good example of why I enjoy this forum. When I first started visiting here, it was a refuge for a handful of former MacRumors members, who were either pushed out or simply got sick of dealing with bad moderating, and shameless defense of obvious trolls. At the time, I wasn't sure this forum had enough inertia to propel it long-term. Thankfully, my skepticism wasn't warranted, and this site is able to stand on its own. Most of the quality posters that I enjoyed over at the other place are now active here. In recent times, @theorist9 and @casperes1996 have joined us here, among many others. I hardly post over at MR any longer, not because I got banned, but because the quality individuals have migrated over here. I no longer have to deal with the ne'er-do-wells and malcontents, thanks to this site's sensible forum moderation and the overall improved quality of dialogue.

...that being said, I'm still assuming that the M2 loses to x86 on some random chess benchmark written in 1988, but I think us Mac users will somehow persevere, despite that substantial setback.

 

[Cmaier]

What I find particularly striking is that the M2 is now beating the Mac Pro in Geekbench, not just almost doubling it in single-core, but besting the Mac Pro in multi-core, as well:

M2:

Single-core: 1919
Multi-core: 8929

8-Core Mac Pro:

Single-Core: 1016
Multi-Core: 8027

You have to upgrade to the 12-core Mac Pro, starting at $7,000, to beat the M2 in multi-core. Tell somebody this two years ago and it would be pure insanity for a MacBook to best the Mac Pro in multi-core performance and nearly double it in single-core.

Also, according to the Geekbench Metal benchmark, the M2 GPU is about 50% faster than the M1. The M1 scored 20440, while the M2 is 30627, which is similar to the GFXBench results from earlier in this thread. I have a suspicion that the M2 is a case of Apple under-promising, while over-delivering.

As a side note, this thread is a good example of why I enjoy this forum. When I first started visiting here, it was a refuge for a handful of former MacRumors members, who were either pushed out or simply got sick of dealing with bad moderating, and shameless defense of obvious trolls. At the time, I wasn't sure this forum had enough inertia to propel it long-term. Thankfully, my skepticism wasn't warranted, and this site is able to stand on its own. Most of the quality posters that I enjoyed over at the other place are now active here. In recent times, @theorist9 and @casperes1996 have joined us here, among many others. I hardly post over at MR any longer, not because I got banned, but because the quality individuals have migrated over here. I no longer have to deal with the ne'er-do-wells and malcontents, thanks to this site's sensible forum moderation and the overall improved quality of dialogue.

...that being said, I'm still assuming that the M2 loses to x86 on some random chess benchmark written in 1988, but I think us Mac users will somehow persevere, despite that substantial setback.

Be sure to tell your friends about the high quality technical entertainment to be had over here.

 

[Yoused]

What I find particularly striking is that the M2 is now beating the Mac Pro in Geekbench, not just almost doubling it in single-core, but besting the Mac Pro in multi-core, as well:

M2:

Single-core: 1919
Multi-core: 8929

8-Core Mac Pro:

Single-Core: 1016
Multi-Core: 8027

You have to upgrade to the 12-core Mac Pro, starting at $7,000, to beat the M2 in multi-core. Tell somebody this two years ago and it would be pure insanity for a MacBook to best the Mac Pro in multi-core performance and nearly double it in single-core.

Looking at my chart, the Xeon W has a Single Core/GHz score of 461, which is 25 points (about 3%) higher than the same score for the 2019 A12Z and well below any M-series SoC. Additionally, the M1 Ultra, with 16+4 cores, shows a GB5 multicore score of 23367, which is 17% higher than the 28 core (56 thread) Xeon's score of 20029, though the Ultra runs at 3.2 while the Xeon Mac Pro runs at 2.5.

It looks like Blizzard performance has been improved considerably over Icestorm, so the little cores put a lot more into the M2 than the M1. Given the performance offered by the Studio, it is not entirely obvious to me that Apple even needs to produce a M-series pro. You can still get a $50+K Mac Pro, but the top end Studio at $8K pantses it already, so why would Apple bother? They can clearly sell more of the latter than the former: getting more Macs out there seems like a better strategy than selling a tiny number of niche products.

 

[throAU]

Those onboard media decode engines are going to make the baseline M2 machines the go-to for casual video editing.

I think a lot of people underestimate their impact, and i'm shocked apple didn't make a much bigger thing of them.