MetalFX and the ANE.

[Jimmyjames]

Do we know the extent to which MetalFX (Temporal) uses the ANE and or AI/ML? A cursory search yields some suggestions that it does, including from @leman on Reddit I think, and Philip Turner on the Apple forums. Trying it out on my M4 does show that with MetalFX Quality mode there is some ANE usage, but it seems really low: less than 5%. Can that be correct?

 

[mr_roboto]

Do we know the extent to which MetalFX (Temporal) uses the ANE and or AI/ML? A cursory search yields some suggestions that it does, including from @leman on Reddit I think, and Philip Turner on the Apple forums. Trying it out on my M4 does show that with MetalFX Quality mode there is some ANE usage, but it seems really low: less than 5%. Can that be correct?

Seems plausible to me. M4 has a 38 TOPS ANE, so 5% of that is 1.9 TOPS. If you assume it's upscaling to 10MP, that'd be about 190K math ops per output pixel per second, or 3166 per pixel at 60 FPS.

 

[Jimmyjames]

Seems plausible to me. M4 has a 38 TOPS ANE, so 5% of that is 1.9 TOPS. If you assume it's upscaling to 10MP, that'd be about 190K math ops per output pixel per second, or 3166 per pixel at 60 FPS.

Ahhh ok. Makes sense. Thanks.

 

[diamond.g]

Does ANE utilization go up if you choose MFX Performance?

I also wonder if MFX Denoising uses AI model (like nvidia ray reconstruction).

 

[dada_dave]

Black Myth: Wukong, Hogwarts Legacy, and the Witcher 3 run at 60 FPS on macOS Tahoe 26 with Metal 4

A new video from Blendlogic Tech explores gaming performance on Apple silicon using the latest macOS and tools. The creator tests several popular titles to see how far Mac gaming has come.

www.notebookcheck.net

Nvidia’s framegen apparently works quite well in Hogwarts Legacy through translation layers on an M1 Max. It must be using the ANE. I haven’t watched the original video just read the article so unclear how many other details there are.

 

[diamond.g]

Black Myth: Wukong, Hogwarts Legacy, and the Witcher 3 run at 60 FPS on macOS Tahoe 26 with Metal 4

A new video from Blendlogic Tech explores gaming performance on Apple silicon using the latest macOS and tools. The creator tests several popular titles to see how far Mac gaming has come.

www.notebookcheck.net

Nvidia’s framegen apparently works quite well in Hogwarts Legacy through translation layers on an M1 Max. It must be using the ANE. I haven’t watched the original video just read the article so unclear how many other details there are.

The comparisons to FSR are odd. Apple doesn't hook it through a special path like they do DLSS so of course there is a performance hit.

Wonder if Apple is able to convert the transformer model or if it is just the CNN.

 

[casperes1996]

Wonder if Apple is able to convert the transformer model or if it is just the CNN.

What do you mean by this? They overtake the whole DLSS dll to call back into GPTK to trigger Metal code instead

 

[diamond.g]

What do you mean by this? They overtake the whole DLSS dll to call back into GPTK to trigger Metal code instead

So it isn't running the actual DLSS model at all. Which would explain why the powerlines in CP2077 looked like garbage (that has been fixed in the DLSS Transformer model). So does that mean that Nvidia Reflex doesn't actually work?

 

[casperes1996]

So it isn't running the actual DLSS model at all. Which would explain why the powerlines in CP2077 looked like garbage (that has been fixed in the DLSS Transformer model). So does that mean that Nvidia Reflex doesn't actually work?

It's not running DLSS or any Nvidia specific tech, no. Apple is grabbing the calls to DLSS the game makes and using the inputs given to DLSS to instead run MetalFX. Don't know if Apple hooks Reflex or how Reflex works so no clue on that one. If it is purely implemented in-game then it probably does just work. If it requires a driver/DLL component, then no, actual Reflex most certainly not. I don't know if Apple has a hook to replace that with a Metal implementation of something similar but I would find it very unlikely

 

[diamond.g]

It's not running DLSS or any Nvidia specific tech, no. Apple is grabbing the calls to DLSS the game makes and using the inputs given to DLSS to instead run MetalFX. Don't know if Apple hooks Reflex or how Reflex works so no clue on that one. If it is purely implemented in-game then it probably does just work. If it requires a driver/DLL component, then no, actual Reflex most certainly not. I don't know if Apple has a hook to replace that with a Metal implementation of something similar but I would find it very unlikely

It is in game (SDK), that for lack of better explanation makes games CPU bound for better response time or in game feel (which is hidden if you use a controller).

 

[casperes1996]

It is in game (SDK), that for lack of better explanation makes games CPU bound for better response time or in game feel (which is hidden if you use a controller).

If it's a vendor agnostic game engine thing I see no reason it wouldn't work

 

[diamond.g]

If it's a vendor agnostic game engine thing I see no reason it wouldn't work

Does macOS have any way to measure "pc latency"?

 

[casperes1996]

Does macOS have any way to measure "pc latency"?

Latency is such a huge conversation. There's a bunch of different ways to measure during development, but measuring from the outside perspective we're a smidge more limited. If you just mean present delays then that information is present in the updated Metal 4 HUD. But if you want to know something like when the game thread finished its physics computations for a given frame relative to the call to present, then I don't think you can get that without the game itself providing the instrumentation

 

[Jimmyjames]

Does macOS have any way to measure "pc latency"?

What does “pc latency” mean?

 

[diamond.g]

What does “pc latency” mean?

It sounds like Metal 4 HUD does show this value. What would be missing is the input latency and display latency (which Apple should show/know on mobile systems).

 

[Jimmyjames]

View attachment 35737
It sounds like Metal 4 HUD does show this value. What would be missing is the input latency and display latency (which Apple should show/know on mobile systems).

Ahh ok. That seems like quite a generic measure, encompassing many subsystems.

 

[Jimmyjames]

I just posted a message in the AC Shadows thread about an upscaling mystery within the game. It’s tangentially related to to the ANE so I thought I would mention the post here also.

Post in thread 'Assasin’s Creed Shadows coming to the Mac day 1.'

Wednesday at 11:06 AM

I’ve been looking at changing various settings within the game and have been surprised at how little difference in performance changes in MetalFX seems to make. I’m a complete novice at this but I launched Metal system trace and captured a few seconds of AC Shadows and also some other Resident Evil games.

I’m left with the impression that AC Shadows doesn’t seem to actually use upscaling much! It’s very possible that I am misreading some of the data, but it does seem to line up with the results I see. When I run Metal system trace on the RE games, with the section marked as MetalFX, I...

• Jimmyjames

• 

 

[diamond.g]

Ahh ok. That seems like quite a generic measure, encompassing many subsystems.

It is. Folks that play with mainly controllers are less likely to feel poor framerates/latency than those that primarily use keyboard and mouse. The macOS mouse acceleration sucks for (FPS) gaming crowd dwells here.