Thoughts on ROCm and AMD with PyTorch…

[tomO2013]

I’m actually in the market right now for a new windows and Ubuntu linux dual boot workstation.

I love my mac and will continue to use my M3 Max macbook pro primarily when out and about. However I do also enjoy some gaming on the side and I want to have an easier time of it developing with Windows. I’m using parallels currently on my macbook but it’s not a great developer experience. Sometimes you just want a separate physical dedicated workstation… one for apple dev, one for windows / linux.

When I factored power/performance of the the AMD ‘best’ VS Intel best… it was a no-contest win for AMD Zen 4 and I’m perhaps two mouse clicks away from putting in my order. The 14900k series just eats so much power under heavy load and is so noisy - my M3Max has ruined me in terms of quiet workstation performance).
However one of the big things for me that AMD still has going for it is forward compatibility with future 8000 series CPU’s. Zen 5 should be drop in replaceable with the Zen 4. This is huge.
Intel’s next cpu will require new motherboard etc…

My current dilemma is what GPU to go with…. prices in Canada still average 2800 - 3000 for a 4090 which is insane. A 4080/4080 super is 1800.
An Rx7900XTX is 1300.

I’m thinking of going all AMD, because of the linux driver support is (in my experience anyway vastly superior if you want a stable linux desktop workstation). I do know that I’m giving up CUDA performance on PyTorch for ROCm on AMD. However, I’m prepared to give up performance and take on some time cost for getting the RX7900XTX setup correctly. I simply like the open model driving ROCm and there have been some big improvements in the last year. It’s like the AMD has finally woken up that it’s not just good enough to open source a technology but you need to continually invest and support it.
I‘m thinking with the cost of hardware and AI hardware going through the roof, even subscription cloud pricing is astronomical, we may see a shift towards more open hardware approach’s for those who are willing to sacrifice some performance. In that sense I like the idea of supporting with my dollars ROCm by buying AMD hardware even if it doesn’t make the most technical sense from a performance perspective. I know … I’m nuts!

A final consideration for me going all AMD…. 5000 series rtx will drop later this year, dropping prices like a boat anchor on 4000 series cards. I’ll loose significant less on reselling a RX7900XTX later…

Are there any other thoughts that you guys have here?

 

[Nycturne]

A final consideration for me going all AMD…. 5000 series rtx will drop later this year, dropping prices like a boat anchor on 4000 series cards. I’ll loose significant less on reselling a RX7900XTX later…

Assuming the 5000 series isn't more expensive than the 4000s. They did eat up a lot of the "gains" with higher pricing on the 4000s, so not sure what their game here will be.

That all said, I have an AMD system at the moment, along with an AMD-based miniPC for astrophotography work and would go with them again. I just might avoid getting a CPU/chipset right as it launches as AMD has consistently had bugs they had to patch with microcode in short order that broke folks' software. Once the patches are in place, it tends to be smooth sailing. It's the one downside I see with them.

 

[dada_dave]

As long as the RX7900XTX provides the performance level that you need at the price you want then I would say go for it and worry less about absolute performance than being happy with your overall system performance/price/characteristics. The one caveat that I would add to the GPU-side is that AMD's history of providing ROCm support their own consumer GPUs is spotty at best. Since you mentioned upgrading the GPU in the next generation, you may not be doing so day 1 and will have to wait for ROCm drivers if they ever come. Of course, AMD might get better and start releasing them more quickly and consistently.

I am curious about this statement:

I’m thinking of going all AMD, because of the linux driver support is (in my experience anyway vastly superior if you want a stable linux desktop workstation).

My impression was that the compute drivers and even graphics drivers for Nvidia GPUs on Linux tended to be more stable than the AMD ones (though those were the formerly proprietary Nvidia drivers and now "open source" ones - I know some people don't want to use those on principle and I'm not going to argue with anyone on that). While I know AMD Linux drivers have gotten better, I was under the impression that they still lagged behind their Nvidia counterparts in terms of stability and performance. Do you feel the opposite?

 

[tomO2013]

As long as the RX7900XTX provides the performance level that you need at the price you want then I would say go for it and worry less about absolute performance than being happy with your overall system performance/price/characteristics. The one caveat that I would add to the GPU-side is that AMD's history of providing ROCm support their own consumer GPUs is spotty at best. Since you mentioned upgrading the GPU in the next generation, you may not be doing so day 1 and will have to wait for ROCm drivers if they ever come. Of course, AMD might get better and start releasing them more quickly and consistently.

I am curious about this statement:



My impression was that the compute drivers and even graphics drivers for Nvidia GPUs on Linux tended to be more stable than the AMD ones (though those were the formerly proprietary Nvidia drivers and now "open source" ones - I know some people don't want to use those on principle and I'm not going to argue with anyone on that). While I know AMD Linux drivers have gotten better, I was under the impression that they still lagged behind their Nvidia counterparts in terms of stability and performance. Do you feel the opposite?

Historically I found that the NVIDIA driver (on a 1080ti pascal card) would run the fans at full blast even for casual desktop and browsing. It was just a much more difficult card to live with than the Vega AMD equivalent. I regretted at the time not going AMD for that reason.

Having said that - I'm glad you asked me as it prompted me to take a look and see what the most up to date state of play is with respect to linux support. It does look like NVIDIA has announced within the last year that they would open source kernel modules for their graphics cards in late 2022/early 2023 no less.

NVIDIA Releases Open-Source GPU Kernel Modules | NVIDIA Technical Blog

The first open-source release of GPU kernel modules for the Linux community helps improve NVIDIA GPU driver quality and security.

developer.nvidia.com

However it does look like the opensourec driver still has to fight to support newer cards. They still require firmware signing https://nouveau.freedesktop.org/ which restricts features under linux if you want to use open source drivers like nouveau and unfortunately it looks like some of the major contributors to that project have left...

NVIDIA's Open-Source Kernel Driver & Maturing Wayland Support Were Great In 2023 - Phoronix

www.phoronix.com

 

[dada_dave]

Historically I found that the NVIDIA driver (on a 1080ti pascal card) would run the fans at full blast even for casual desktop and browsing. It was just a much more difficult card to live with than the Vega AMD equivalent. I regretted at the time not going AMD for that reason.

Interesting, was this with the nouveau drivers? For nouveau, I definitely heard that the Nvidia drivers were not as good as the AMD drivers, but also that the proprietary Nvidia drivers were, on average, far better than both. However, of course, especially with Linux and all its variations, YMMV.

From personal experience using the Nvidia proprietary drivers, while I didn't have the 1080ti, I did have Maxwell, Pascal (Titan Xp which is almost the same thing), and Turing GPUs (still have the latter two) and never experienced this including for CUDA work. But again, I did not have that specific GPU however so even if you also had the proprietary drivers, maybe this is a unique interaction?

Having said that - I'm glad you asked me as it prompted me to take a look and see what the most up to date state of play is with respect to linux support. It does look like NVIDIA has announced within the last year that they would open source kernel modules for their graphics cards in late 2022/early 2023 no less.

NVIDIA Releases Open-Source GPU Kernel Modules | NVIDIA Technical Blog

The first open-source release of GPU kernel modules for the Linux community helps improve NVIDIA GPU driver quality and security.

developer.nvidia.com

However it does look like the opensourec driver still has to fight to support newer cards. They still require firmware signing https://nouveau.freedesktop.org/ which restricts features under linux if you want to use open source drivers like nouveau and unfortunately it looks like some of the major contributors to that project have left...

NVIDIA's Open-Source Kernel Driver & Maturing Wayland Support Were Great In 2023 - Phoronix

www.phoronix.com

The new "open source" drivers from Nvidia are from what I can gather a relatively thin shell with most of the "meat" has been moved to the GPU's firmware. This meant not everyone was impressed by Nvidia's open sourcing but ... . I'm less fussed by propriety vs open source, hey I use a Mac after all, and just typically use whatever works best for what I've got.

Just to clarify, the above was just to satisfy my curiosity about your past experience rather than relating an opinion to the purchase of a new device.

 

[leman]

To be honest, I wouldn't bother. If you are interested in GPGPU I'd just take the path of least resistance and get an Nvidia GPU. You'd get better ecosystem support and things will be more predictable overall.

 

[casperes1996]

Interesting, was this with the nouveau drivers? For nouveau, I definitely heard that the Nvidia drivers were not as good as the AMD drivers, but also that the proprietary Nvidia drivers were, on average, far better than both. However, of course, especially with Linux and all its variations, YMMV.

State of play has generally been that Nvidia never really bothered properly supporting Wayland with their proprietary drivers, so while X11 does work fine with their drivers, Wayland support is shoddy at best