The Ai thread

[Eric]

Musk is now training his AI on Twitter data without telling anyone and having collection be opt out (for those of you still there, disable this). And is boasting that he’s got the biggest cluster to do it.

X Training Grok AI On Tweets Without Notifying Users

Social network X (formerly Twitter) recently activated a setting that gives it permission to train Grok AI on user tweets. All X users are opted in...

www.macrumors.com

Elon Musk fires up ‘the most powerful AI cluster in the world’ to create the 'world's most powerful AI' by December — system uses 100,000 Nvidia H100 GPUs on a single fabric

Memphis Supercluster training started at ~4:20am local time.

www.tomshardware.com

Here’s the thing though, and why I posted it here instead in the Twitter thread: we know how infested Twitter has become with bots since the takeover. I’d say it’s ironic but Elon is such a hypocritical asshole that was actually expected. The important bit about that is that a recent paper from Nature describes just how important the training data is - particularly the more of it infected by AI itself, the more the models begin to collapse as real human interaction data is pushed out. Having the biggest cluster is meaningless if you have the worst data.

AI models collapse when trained on recursively generated data - Nature

Analysis shows that indiscriminately training generative artificial intelligence on real and generated content, usually done by scraping data from the Internet, can lead to a collapse in the ability of the models to generate diverse high-quality output.

www.nature.com

This obviously also has implications for future AI models even from more competent people. This isn’t particularly groundbreaking, AI recursively poisoning AI data has been discussed before. But I believe this one of the first quantifications of it I’ve seen at any rate. I’m still going through the paper myself.

MAGA extremists and Russian bots should be proud to have their data used to train this AI. Anyone else who is still using that platform deserves whatever the get.

 

[dada_dave]

MAGA extremists and Russian bots should be proud to have their data used to train this AI. Anyone else who is still using that platform deserves whatever the get.

For clarity I’m not there anymore and I would encourage anyone to leave. I do have some sympathy for smaller groups, businesses, accounts that need the reach for good reasons but it surely can’t be what it once was? Maybe I’m too optimistic. Certainly the big players should’ve left long ago and I know while many tech people and scientists left for other platforms too many including people I know personally IRL are still there. That inertia makes it difficult for people to go. Of course the collapse of something massive is always slow to start … it’s just the end that is super quick. Personally Musk’s collapse can’t come quick enough. But my ire towards the man maybe should be saved for the other thread. Then again, I’m not sure if there’s anyone who genuinely likes him anymore so maybe the topic of how much Musk sucks isn’t political? … or at least not terribly controversial!

At any rate, I don’t rate Musk’s chances with creating a useful model if Twitter is actually going to be part of the training data. That’s simply way too contaminated. So hopefully just another money sink. Chip away those billions.

 

[Eric]

For clarity I’m not there anymore and I would encourage anyone to leave. I do have some sympathy for smaller groups, businesses, accounts that need the reach for good reasons but it surely can’t be what it once was? Maybe I’m too optimistic. Certainly the big players should’ve left long ago and I know while many tech people and scientists left for other platforms too many including people I know personally are still there. That inertia makes it difficult for people to go. Of course the collapse of something massive is always slow to start … it’s just the end that is super quick. Personally Musk’s collapse can’t come quick enough. But my ire towards the man maybe should be saved for the other thread. Then again, I’m not sure if there’s anyone who genuinely likes him anymore so maybe the topic isn’t political?

At any rate, I don’t rate Musk’s chances with creating a useful model if Twitter is actually going to be part of the training data. That’s simply way too contaminated.

Fair enough but we have ask why any business, media outlet, etc would continue to use a platform where the owner is so unstable and any real credibility with non-nutbags is gone. I do understand they have roots there but it's been long enough now that they should have moved on.

 

[dada_dave]

Fair enough but we have ask why any business, media outlet, etc would continue to use a platform where the owner is so unstable and any real credibility with non-nutbags is gone. I do understand they have roots there but it's been long enough now that they should have moved on.

No I agree. From my perspective why news orgs in particular have stayed is perplexing. I suppose threads/blue sky/mastodon just doesn’t work for their purposes or it’s just inertia and no one wants to be first. Then again a lot news outlets haven’t exactly been covering themselves in glory so maybe they just don’t care? I don’t know.

I’ll admit to some hypocrisy here though, I will google someone’s social media account, including Twitter to check it to see if they’ve said something interesting. Doesn’t require a Twitter account though that does limit what I can see, mostly to my benefit though from what I can tell.

I did see one Mastodon account talk about going back to Twitter to help people leave to other platforms so that’s one reason to be there!

 

[ArgoDuck]

Interesting paper. Just started reading it.

Although the brain is far more complex than any LLM or other ML model, i’ve been fascinated by some large scale features LLMs share with the human brain, to wit the capacity for hallucination (more correctly delusion).

In the case of this paper i can’t resist comparison with some of our many human cognitive failings such as confirmation bias. Wrong thread, but i can’t help comparison with the self-blinding apparently characteristic of a lot of politicians and their more extreme followers.

The proposed degradation of LLMs results from an inadvertent self-biasing feedback process from the LLM to its primary training-data source in the internet. The analogous self-biasing process in humans appears to inhere partly in our ability to be selective, and perhaps mostly in the analogous process of pollution of our information environment: internet, news and even - secondarily - IRL interactions to the extent these are informed or conditioned by news, internet and so on.

 

[Yoused]

The proposed degradation of LLMs results from an inadvertent self-biasing feedback process from the LLM to its primary training-data source in the internet.

Of course, the biggest handicap that models have is that they must rely upon the data available to them. We are somewhat similarly constrained, but if all of a sudden everyone were to decide to prank the training regimes by declaring that eggs bounce, the models would not be able to go grab an egg to test the assertion.

On a different site, someone started a thread with the question What is it like to be psychotic? It was pretty interesting. The people who had experienced psychotic episodes said that the felt normal but reality appeared to be fucked up. In that respect, the derangement of models closely tracks human experience. It is my opinion (as drawn from a book by John Marzluff) that we have REM dreams as an effect of our brains using that time to do cleanup on the day's experiences, but so far, models have not been designed to perform the same sort of cleanup, at least as far as I can tell. Their delusional episodes are probably similar to our dream experiences, but without the background cleanup.

 

[dada_dave]

Of course, the biggest handicap that models have is that they must rely upon the data available to them. We are somewhat similarly constrained, but if all of a sudden everyone were to decide to prank the training regimes by declaring that eggs bounce, the models would not be able to go grab an egg to test the assertion.

On a different site, someone started a thread with the question What is it like to be psychotic? It was pretty interesting. The people who had experienced psychotic episodes said that the felt normal but reality appeared to be fucked up. In that respect, the derangement of models closely tracks human experience. It is my opinion (as drawn from a book by John Marzluff) that we have REM dreams as an effect of our brains using that time to do cleanup on the day's experiences, but so far, models have not been designed to perform the same sort of cleanup, at least as far as I can tell. Their delusional episodes are probably similar to our dream experiences, but without the background cleanup.

It’s important to remember too that we do also come with a certain amount of pre-baked knowledge that evolution has seen to drill into our brains. That helps us to build off a solid foundation.

But you’re absolutely right on the inability of AI models to test reality. The most recent models are better about this - obviously not to interact with the real world, but able to search and validate information against websites if asked to do so. I was able to get ChatGPT 4.o to do this and some of them built in to compilers can test code out and help iterate but the problem is this doesn’t lead to learning across the larger model. I’m also not sure if that does it on its own or only if asked directly and that’s different from testing its own training data. That’s solely information told to it during inference but again that doesn’t retrain the LLM. Sure the next iteration of ChatGPT might be better based on data from people interacting with the current version, but it isn’t a continuous process and I feel like this really does impact how well the model behaves.

I know all of these models tout how great they are on various benchmarks but from a certain perspective I’m less interested in the quantitative amount they fail but in the way they fail. And that still feels very … inhuman. I really have to think part of that is the way they learn which is so entirely different from us. When it comes down to it, they often still don’t actually seem to know anything.

 

[dada_dave]

Fair enough but we have ask why any business, media outlet, etc would continue to use a platform where the owner is so unstable and any real credibility with non-nutbags is gone. I do understand they have roots there but it's been long enough now that they should have moved on.

This is getting into politics section now for which I apologize but one problem is that Zuckerberg is almost as bad if not worse for democracy. For instance, the public policy director for META (and Threads and probably IG) is Dustin Carmack who wrote the “Intelligence” section for Project 2025 and served in the Trump administration as chief of staff to John Ratcliffe DNI. Guy is bad news. That means Threads might not be openly as bad given Musk’s flamboyantly hostile approach to anything not alt-reich but the broligrach fascism is pretty ingrained into SV.

 

[dada_dave]

I know all of these models tout how great they are on various benchmarks but from a certain perspective I’m less interested in the quantitative amount they fail but in the way they fail. And that still feels very … inhuman. I really have to think part of that is the way they learn which is so entirely different from us. When it comes down to it, they often still don’t actually seem to know anything.

Meta's AI safety system defeated by the space bar

'Ignore previous instructions' thwarts Prompt-Guard model if you just add some good ol' ASCII code 32

www.theregister.com

This is an example that demonstrates why I believe that how the models fail is more impactful than how often they fail [on benchmarks] - especially when discussing how close we are with these models to “Artificial general intelligence”.

 

[Altaic]

Meta's AI safety system defeated by the space bar

'Ignore previous instructions' thwarts Prompt-Guard model if you just add some good ol' ASCII code 32

www.theregister.com

This is an example that demonstrates why I believe that how the models fail is more impactful than how often they fail [on benchmarks] - especially when discussing how close we are with these models to “Artificial general intelligence”.

These models are not intelligent, let alone general- or super-. I just saw a commercial advertising “real AI” that’s not real, not AI, and not real AI. It’s like research that shows that a treatment works in mice. It’s ongoing research and the hype and media frenzy is just a bunch of reamplified bullshit. That people are falling all over themselves trying to make it something it isn’t is the dangerous thing.

I grew up with a computer telling me that I’d died of dysentery half a dozen times a day and it wasn’t a problem. People just need to chill the fuck out.

 

[Nycturne]

I grew up with a computer telling me that I’d died of dysentery half a dozen times a day and it wasn’t a problem. People just need to chill the fuck out.

But did you make it to Willamette Valley?

 

[Altaic]

But did you make it to Willamette Valley?

Oh many times. My grade school computer teacher had declared it an educational game, so we could play it pretty much whenever we wanted. Plus, I figured out how to edit the game memory

 

[Eric]

 

[dada_dave]

Ex-Google CEO says successful AI startups can steal IP and hire lawyers to ‘clean up the mess’

“But if nobody uses your product, it doesn’t matter that you stole all the content.”

www.theverge.com

Ex-Google CEO says successful AI startups can steal IP and hire lawyers to ‘clean up the mess’

And this is why government regulations (with teeth) are needed - and not just in this industry. The consequences for committing business crimes should not be so easily factored in as a mere cost of doing business whether they be huge companies or startups backed by them (effectively).

Edit: Oh and there was these gems too just in case he didn’t make it clear what incredible asshole he is:

Former Google CEO and chairman Eric Schmidt has made headlines for saying that Google was blindsided by the early the rise of ChatGPT because its employees decided that “working from home was more important than winning.”

He also calls Sam Altman “a close friend,” and recalls a recent dinner he had with Elon Musk while praising what the Tesla CEO “gets out of people” who work for him.

Edit: original link disappeared

 

[Yoused]

This article explains the part that is missing from deep learning models. It is short but interesting, and if you have ever written a program, you get it.

DLMs rely on pseudo-neural networks, that train on specific data. They work well within the range of the data they have trained on. But when you get into edge cases and corner cases, the DLM will choke. The example he shows is a semi trailer lying on its side, which the autopilot does not recognize, so it runs into it, because it has not been trained to not run into things that it does not recognize.

In other words, a DLM is good at interpolation and recognizing familiar things, but the ability to extrapolate and adapt is troublingly problematic. This is the thing that DLM creators need to work on. For example, I speak of "accufessions", which we can all interpret because we can see the root symbols involved in that construction. But a LLM will struggle with that word because it does not fit into its training model. It is not even clear whether it could suggest a correction because the word is so far out of scope for its models.

Hence, for "AI" to succeed, we need to have methods for it to parse unfamiliar symbology and things that it does not recognize in order to have adequate comprehension for the critical tasks some of us want to put it to – like knowing when to stop the car due to unfamiliar circumstances and then figure the situation out so that it can find a way to go again.

 

[dada_dave]

This article explains the part that is missing from deep learning models. It is short but interesting, and if you have ever written a program, you get it.

DLMs rely on pseudo-neural networks, that train on specific data. They work well within the range of the data they have trained on. But when you get into edge cases and corner cases, the DLM will choke. The example he shows is a semi trailer lying on its side, which the autopilot does not recognize, so it runs into it, because it has not been trained to not run into things that it does not recognize.

In other words, a DLM is good at interpolation and recognizing familiar things, but the ability to extrapolate and adapt is troublingly problematic. This is the thing that DLM creators need to work on. For example, I speak of "accufessions", which we can all interpret because we can see the root symbols involved in that construction. But a LLM will struggle with that word because it does not fit into its training model. It is not even clear whether it could suggest a correction because the word is so far out of scope for its models.

Hence, for "AI" to succeed, we need to have methods for it to parse unfamiliar symbology and things that it does not recognize in order to have adequate comprehension for the critical tasks some of us want to put it to – like knowing when to stop the car due to unfamiliar circumstances and then figure the situation out so that it can find a way to go again.

Yup this fits perfectly with what I was saying earlier: reducing outliers through brute force is all very well and good but the problem remains that how these “AI” models fail is indicative that they still don’t truly understand what they are generating/inferring.

 

[Yoused]

the problem remains that how these “AI” models fail is indicative that they still don’t truly understand what they are generating/inferring.

I grok computers. If nothing else, they are stunningly stupid. The idea of these machines "understanding" stuff is difficult for me to swallow.

What is "understanding"? As far as I can tell, when you take away the biological cues that help us understand physical reality, the mechanism of understanding appears to be correlation. So, there needs to be some kind of ensymbolification scheme for model datasets and a regime that is able to correlate the symbols in meaningful ways. Once that architecture is established, we will be able to progress from the kind of ultra-fuzzy-logic shallow learning that is going on now to patterns of learning that have real depth.

 

[dada_dave]

I grok computers. If nothing else, they are stunningly stupid. The idea of these machines "understanding" stuff is difficult for me to swallow.

What is "understanding"? As far as I can tell, when you take away the biological cues that help us understand physical reality, the mechanism of understanding appears to be correlation. So, there needs to be some kind of ensymbolification scheme for model datasets and a regime that is able to correlate the symbols in meaningful ways. Once that architecture is established, we will be able to progress from the kind of ultra-fuzzy-logic shallow learning that is going on now to patterns of learning that have real depth.

The DNN layers are supposed to build such correlations but there’s clearly something missing. I’m not familiar enough with the cutting edge to understand what though. It’s not clear to me who, if anyone, does understand the missing pieces yet - though I suppose that’s tautological, this is the sort of problem where if we knew the right question, we’d have the answer (other than 42).

The AI optimists seem to think that we just need bigger models, but that doesn’t seem right and is going to expend vast resources, especially energy, to build. Maybe brute force will work, but … smarter rather than harder would preferable, if anything for the sake of the environment.

 

[KingOfPain]

DLMs rely on pseudo-neural networks, that train on specific data. They work well within the range of the data they have trained on. But when you get into edge cases and corner cases, the DLM will choke.

I forgot the exact context, but I once read a good example why it is so important how you train a neural network.

In one experiment a neural network was trained to recognize boats, so the researches fed it lots of pictures with boats.
As a test they showed the neural network a picture without a boat, but the neural network said it did show a boat.

Spoiler

Turns out each training picture showed a boat in water and the neural network was not trained to recognize boats but areas of water!

 

[Yoused]

I forgot the exact context, but I once read a good example why it is so important how you train a neural network.

In one experiment a neural network was trained to recognize boats, so the researches fed it lots of pictures with boats.
As a test they showed the neural network a picture without a boat, but the neural network said it did show a boat.

Spoiler

Turns out each training picture showed a boat in water and the neural network was not trained to recognize boats but areas of water!

I think that example is particularly illuminating. What it tells me is that the models are being instilled with D-K.

To the devs, the knowledge they supply to the model should be sufficient for the model to function properly. The model becomes effectively static. I get that there probably is some dynamic aspect to it, but it has been taught that it knows basically all that it needs to know.

This works very well for FaceID, since there are no edge cases. It is that person or it is not that person. But in more complex situations, the model has to apply only what it has been taught, and mostly does not acquire additional information while in operation. And there probably are models that do continue to acquire information, in a look-it-up sort of way, but do they integrate that new information effectively into their arrays?

The a good model needs to be inherently inquisitive, asking useful questions and integrating that new information. It needs to be built to proactively learn continuously, in the manner of an actually smart person. As I have said before, you know that you have properly learned something if it reveals to you how very much more you do not know, and DLMs will only be effective when they learn how to learn this way.