The Ai thread

[dada_dave]

17 cringe-worthy Google AI answers demonstrate the problem with training on the entire web

From dangerous medical advice to extremist disinformation.

www.tomshardware.com

AI may kill us all just not in the way you might think. It’ll just be people blindly following its advice on how to glue your cheese to your pizza, how many rocks to eat, and generally spreading conspiracy theories and satire as though they were real.

I can’t wait for it to be trained on our forum and to have an opinion on the correct use of “thusly” in the context of computing.

 

[dada_dave]

Maybe the post below about the potential privacy concerns of the new Recall feature for Co-Pilot AI's would've been better suited to the AI thread than the Nuvia thread:

I was wondering where Recall would pop up. Hilariously, I've started dual-booting Linux (rather than use a VM) for the first time in two decades, because of this announcement. Just using Mint rather than Yellow Dog, and on an AMD rig rather than my PowerMac.

Longhorn on Recall:

Recall: random notes about the security model – random blog

 

[Nycturne]

Longhorn on Recall:

Recall: random notes about the security model – random blog

It’s really easy to get off topic here, as there’s just so much that can be said about the Windows security model. It’s also hard to separate discussions of Recall security from that security model when it comes to third-party actors.

Ultimately, it’s more that Microsoft with Windows has increasingly become user hostile. Improve Windows’ security model to reduce the impact of vulnerabilities? Nah. Harass me into buying OneDrive or some other recurring service? Yes. Try to use Windows as an ad platform? Yes. Recall is just another feature I don’t want, pushed because AI is the new thing that needs to be shoved into everything. I’m just tired of it.

I’m not expecting to abandon Windows immediately, but may as well see how much progress Steam has made with Proton.

 

[exoticspice1]

It’s really easy to get off topic here, as there’s just so much that can be said about the Windows security model. It’s also hard to separate discussions of Recall security from that security model when it comes to third-party actors.

Ultimately, it’s more that Microsoft with Windows has increasingly become user hostile. Improve Windows’ security model to reduce the impact of vulnerabilities? Nah. Harass me into buying OneDrive or some other recurring service? Yes. Try to use Windows as an ad platform? Yes. Recall is just another feature I don’t want, pushed because AI is the new thing that needs to be shoved into everything. I’m just tired of it.

I’m not expecting to abandon Windows immediately, but may as well see how much progress Steam has made with Proton.

I’m waiting on Apple’s take on AI on macOS and see how their desktop approach will be different than Microsoft’s Windows.

iOS should also influence some of new Ai abilities available in macOS 15, since they share the same kernel.

 

[dada_dave]

Nvidia announces Project G-Assist, an AI assistant for gaming with GeForce GPUs

The best strategy for beating that high-level boss might be to ask your computer for advice.

www.tomshardware.com

With Project G-Assist and a compatible GPU, gamers can access a large language model (LLM) linked to a game knowledge database as they play a supported game. Using voice prompts or text inputs, players can query the AI assistant for advice on how to proceed in a game.

SAG-AFTRA Approves AI Voice Acting For Video Games

The union's agreement with AI voiceover studio Replica is being called out by some of the industry's top voice talent.

www.gamespot.com

So I don’t know the details of this deal (and apparently neither do the voice actors which is not a good look for their union). But I can see AI voice acting being okay under specific circumstances. This isn’t the only one but the main one I can think of which I think has been mentioned in this thread by others: imagine a massive sprawling RPG like Baldur’s Gate where human writers and an actor have been paid for a huge number of lines to create a compelling character one with an arc or even multiple arcs based on your choices in game. Then imagine being able to interact with said character by simply typing (or speaking!) your lines. Maybe you can still have prompts to trigger their preset responses but even so the AI is trained to reply (and act) as the character and where they are in the story so far. Human creativity for both voice acting and the lines (animation/mocap too) is still being paid, a necessary element for creating a strong character (so far, AI has its limits), but the freedom of expression for the player goes up tremendously. Now there may be other examples where AI can extend human creativity in this space rather than be a shallow imitation of it (and the business/ethics of the former compared to the latter for writers, actors, and animators), but I think that’s certainly the main area.

Windows Copilot will add GPU support in a future release — Nvidia details the advantages of high performance GPUs for AI workloads and more

Firing up a GPU versus an NPU will still create a massive drain on your laptop battery.

www.tomshardware.com

Also announced today is the new Nvidia RTX AI Toolkit, which helps developers with AI model tuning and customization all the way through to model deployment. The toolkit can potentially be used to power NPCs in games and other applications. We've seen examples of this with the ACE demos mentioned above, where the first conversational NPC demo was shown at last year's Computex. It's been through multiple refinements now, with multiple NPCs, and Nvidia sees ways to go even further.

One example given was a general purpose LLM that used a 7 billion parameter LLM. The interaction with an LLM powering an NPC provided the usual rather generic responses. It also required an RTX 4090 (17GB of VRAM) to run the model, and only generated 48 tokens per second. Using the RTX AI Toolkit to create an optimized model, Nvidia says developers can get tuned responses that are far more relevant to a game world, and it only needed 5GB of VRAM and spit out 187 tokens per second — nearly four times the speed with one third the memory requirement. More importantly perhaps, the tuned model could run on an RTX 4050 laptop GPU, which only has 6GB of VRAM.

This will not replace the need for human writing (for the foreseeable future) but as I said can aid in providing more flexible responses to player inputs and remove some busy work writing which is already boilerplate (think NPC responses like “but then I took an arrow to the knee”).

 

[dada_dave]

Interesting reversal of the usual narrative

 

[Eric]

Ai watermark FTW.

 

[Yoused]

Interesting article over on ars

Researchers upend AI status quo by eliminating matrix multiplication in LLMs

Running AI models without floating point matrix math could mean far less power consumption.

arstechnica.com

 

[Cmaier]

Interesting article over on ars

Researchers upend AI status quo by eliminating matrix multiplication in LLMs

Running AI models without floating point matrix math could mean far less power consumption.

arstechnica.com

yeah, would be very interesting if it really works.

 

[dada_dave]

Mozilla.ai did what? When silliness goes dangerous

Mozilla.ai did a study on generative AI trends and their methodology shows that they should never be allowed near anything AI. Ever.

tante.cc

Mozilla.ai recently went into trying to understand the big trends in generative AI (aside from wasting ungodly amounts of energy and other resources to generate mediocre outputs) and the methodology is a bit … how do you say it in English … fucked up in all imaginable ways.

The[y] realized that their notes have biases and that they themselves might have biases when interpreting/summarizing them. So they took a bunch of heavily biased, mostly undocumented statistical models to summarize the notes and magically get rid of biases (but potentially add some fabrications/”hallucinations” to keep it spicy). But that’s not how any of this works.

This isn’t a “minus times minus results in a positive” kind of situation. What they are doing is stacking biases on top of one another. Their own biases are in the original data and all the biases from the models they used are now added to the mix. They literally made it worse.

And what did the LLMs help them uncover from the data? Which insights have we gained?

Across all the models, 3 key takeaways stood out:

1. Evaluation: Many organizations highlight the challenges of evaluating LLMs, finding it time-consuming.
2. Privacy: Data privacy and security are major concerns influencing tool and platform choices.
3. Reusability & Customization: Organizations value reusability and seek customizable models for specific tasks.

Wow. Mind.blown.

 

[Altaic]

Interesting article over on ars

Researchers upend AI status quo by eliminating matrix multiplication in LLMs

Running AI models without floating point matrix math could mean far less power consumption.

arstechnica.com

I’ve got a lot to say about that article, but in the meantime this guy designed an ASIC for BitNet b1.58 (sqrt(3), ‘cause ternary bits):
https://github.com/rejunity/tiny-asic-1_58bit-matrix-mul

 

[Altaic]

Mozilla.ai did what? When silliness goes dangerous

Mozilla.ai did a study on generative AI trends and their methodology shows that they should never be allowed near anything AI. Ever.

tante.cc

Reads like a clickbait rant. The author’s takeaway is sillier than what he’s critiquing. He demonstrates in his own writing that biases get stacked all the time

 

[dada_dave]

Reads like a clickbait rant. The author’s takeaway is sillier than what he’s critiquing. He demonstrates in his own writing that biases get stacked all the time

Gotta do a hard disagree here. It may be a rant and the author may not like the current machine learning craze but looking at Mozilla’s own explanation, it is an entirely legitimate rant in this case. AI encodes the biases of the training data, the idea to use it to remove bias is extremely bad. Further the hilariously obvious “insights” gained section is literally copied word for word from Mozilla’s blog post. But how obvious those are to point how unnecessary it was to use an LLM to generate that summary was more for humor. The primary point was to point out how dangerously naive a company supposedly dedicated to understanding LLMs and neural networks can be about the subject they claim to study.

We see people take this attitude, that AI models or even more prosaic statistics are “unbiased” because “math doesn’t lie”, on far more serious and important issues, like the use of deep learning in law enforcement and banking with potentially dire consequences for policy and enforcement.

 

[cbum]

how dangerously naive a company supposedly dedicated to understanding LLMs and neural networks can be about the subject they claim to study

Here is my "soft" disagree :

I have my doubts about a corporation doing anything "naively" in today's environment. Not impossible, but I would always suspect calculation first.

 

[Nycturne]

Here is my "soft" disagree :

I have my doubts about a corporation doing anything "naively" in today's environment. Not impossible, but I would always suspect calculation first.

Por que no los dos?

 

[cbum]

Por que no los dos?

Se cancelan le uno a l'otro, no?

 

[Nycturne]

Se cancelan le uno a l'otro, no?

That would be true if a single individual is expressing both qualities. But an organization, by its very nature being a sort of gestalt entity, can. The larger the company, the more likely it would be diagnosed with MPD if it were an actual person. In this case, it could certainly be a case of lack of review before publishing, etc. A chronic problem that I have seen lately in my neck of the woods where folks are being asked to wear an increasing number of hats in their day to day job.

I say this as someone who has worked at a large company that has been simultaneously calculating and naive when it comes to the AI gold rush, in particular, and has also expressed the trait quite a bit in the early mobile era. It's just surprisingly good about hiding the hot mess behind the marketing and PR departments.

 

[KingOfPain]

As someone mentioned in the comments, the acronym for "Make America First in AI" is MAFIA...

Trump allies want to “Make America First in AI” with sweeping executive order

After repealing Biden's AI order, draft would create "Manhattan Projects" for military AI.

arstechnica.com

 

[Yoused]

That would be true if a single individual is expressing both qualities. But an organization, by its very nature being a sort of gestalt entity, can.

Heh. That word reminds me of classic Mac OS. There was a trap or function used by a program to discover the capabilities of the particular platform, which was called Gestalt or something like that.

 

[dada_dave]

Musk is now training his AI on Twitter data without telling anyone and having collection be opt out (for those of you still there, disable this). And is boasting that he’s got the biggest cluster to do it.

X Training Grok AI On Tweets Without Notifying Users

Social network X (formerly Twitter) recently activated a setting that gives it permission to train Grok AI on user tweets. All X users are opted in...

www.macrumors.com

Elon Musk fires up ‘the most powerful AI cluster in the world’ to create the 'world's most powerful AI' by December — system uses 100,000 Nvidia H100 GPUs on a single fabric

Memphis Supercluster training started at ~4:20am local time.

www.tomshardware.com

Here’s the thing though, and why I posted it here instead in the Twitter thread: we know how infested Twitter has become with bots since the takeover. I’d say it’s ironic but Elon is such a hypocritical asshole that was actually expected. The important bit about that is that a recent paper from Nature describes just how important the training data is - particularly the more of it infected by AI itself, the more the models begin to collapse as real human interaction data is pushed out. Having the biggest cluster is meaningless if you have the worst data.

AI models collapse when trained on recursively generated data - Nature

Analysis shows that indiscriminately training generative artificial intelligence on real and generated content, usually done by scraping data from the Internet, can lead to a collapse in the ability of the models to generate diverse high-quality output.

www.nature.com

This obviously also has implications for future AI models even from more competent people. This isn’t particularly groundbreaking, AI recursively poisoning future AI training data has been discussed before. But I believe this one of the first quantifications of it I’ve seen at any rate. I’m still going through the paper myself.