The Ai thread

[Huntn]

The biggest problem for business jumping on tech to reduce the wetware expense is that eventually they are going to be struggling to make up the revenue lost from the masses of people who can no longer afford to buy their stuff.

Corporate America has been cannibalizing, undermining, it’s domestic market for 40 years. Short term profits without a thought about a long term result, and the US government has not stopped them, better said, we did not hire the right people to do this so , it’s all on us.

 

[Huntn]

CGI-AI:
Early CGI in movies, , I absolutely hated it‘s early implementation. Now I still believe that a real physical location has an unbeatable appeal, say James Bond in Sienna, CGI fills a nitch that has no physical alternatives and has progressed to a point where my brain starts to dismiss it has animation when people are seamlessly interwoven. Such as Morag, a dead civilization, or Xandar (Guardians of the Galaxy) state of the art CGI. I usually find myself asking, how much of this is CGI, where dies the practical end and CGI start?

A parallel argument can be made about AI. But first, I’ll repeat my concern that AI as a tool is powerfully and scary on multiple levels. There is without doubt a huge threat to jobs:

The interesting thing is that job loss/export has been going on for 50 years, by the hand of the Corporatocracy, but now that basically most jobs might be threatened, the alarm is being raised. Yes, it is a threat and an opportunity, but it’s just another iteration of technological progress where what used to be represented my human skill is replaced by technology. Society will have to find a new equilibrium, and that will have to include finding a way of supporting the masses that make up our civilization. It’s very possible it’s time to consider the Socialist Utopia.
That said…

I’ve not yet seen a character powered by AI, at least one that I am aware of. If it is “soulless”, I predict it will be just a matter of time befire it finds its soul, as the programing is greatly expanded to incorporate personality and emotions. If it can be done it will be.

 

[Huntn]

The robots are here and the pleasure seekers are going to fall in love with AI, say experts who studied machine-human bonding​

The robots are here and the pleasure seekers are going to fall in love with AI, say experts who studied machine-human bonding

AI will revolutionize sex, love and dating the way Tinder, Grindr and Hinge did, and there's already research on who will dive in first.

fortune.com

However, an enthusiasm for novelty is not the only driver. Studies show that people find many uses for sexual and romantic machines outside of sex and romance. They can serve as companions or therapists, or as a hobby.

In short, people are drawn to AI-equipped machines for a range of reasons. Many of them resemble the reasons people seek out relationships with other humans. But researchers are only beginning to understand how relationships with machines might differ from connecting with other people.

 

[Huntn]

Author discovers AI-generated counterfeit books written in her name on Amazon

Amazon resisted a removal request, citing lack of "trademark registration numbers."

arstechnica.com

Link describes the issue

Upon searching Amazon and Goodreads, author Jane Friedman recently discovered a half-dozen listings of fraudulent books using her name, likely filledwith either junk or AI-generated content. Both Amazon and Goodreads resisted removing the faux titles until the author's complaints went viral on social media.

In a blog post titled "I Would Rather See My Books Get Pirated Than This (Or: Why Goodreads and Amazon Are Becoming Dumpster Fires)," published on Monday, Friedman detailed her struggle with the counterfeit books.

"Whoever’s doing this is obviously preying on writers who trust my name and think I’ve actually written these books

 

[dada_dave]

Author discovers AI-generated counterfeit books written in her name on Amazon

Amazon resisted a removal request, citing lack of "trademark registration numbers."

arstechnica.com

Link describes the issue

Upon searching Amazon and Goodreads, author Jane Friedman recently discovered a half-dozen listings of fraudulent books using her name, likely filledwith either junk or AI-generated content. Both Amazon and Goodreads resisted removing the faux titles until the author's complaints went viral on social media.

In a blog post titled "I Would Rather See My Books Get Pirated Than This (Or: Why Goodreads and Amazon Are Becoming Dumpster Fires)," published on Monday, Friedman detailed her struggle with the counterfeit books.

"Whoever’s doing this is obviously preying on writers who trust my name and think I’ve actually written these books

While obviously there is a selection effect where we mostly hear about the times companies get it wrong, it is amazing how often companies take down content they should leave up and leave up content they should take down.

 

[ArgoDuck]

^^ Similar but not quite the same, this gentleman discovered his phd published as a book without his consent or knowledge. His thesis concerns the supreme court which presumably is why it got pirated. Someone quoted in the article claims it - or other works in general - may be scraped using AI, which is the link to this thread

 

[Huntn]

You say you want a little scary story associated with AI?" AI as it exists today knows enough to know it's being tested and and try to hide stuff...

Can We Stop Runaway A.I.?

Technologists warn about the dangers of the so-called singularity. But can anything actually be done to prevent it?

www.newyorker.com

it’s also possible for even a fairly ordinary A.I. to “lie” about its behavior. In 2020, researchers demonstrated a way for discriminatory algorithms to evade audits meant to detect their biases; they gave the algorithms the ability to detect when they were being tested and provide nondiscriminatory responses. An “evolving” or self-programming A.I. might invent a similar method and hide its weak points or its capabilities from auditors or even its creators, evading detection.

 

[fischersd]

By the time we realize that an AI has surpassed our intelligence it will almost certainly be too late. A cage designed by chimps would not be successful in holding even our children. It's vanity to think that the protections we put in place will hold a super-human intelligence in a box.

It'll be interesting to see how it manages to annihilate us while preserving the infrastructure it needs to survive.

 

[Agent47]

By the time we realize that an AI has surpassed our intelligence it will almost certainly be too late. A cage designed by chimps would not be successful in holding even our children. It's vanity to think that the protections we put in place will hold a super-human intelligence in a box.

It'll be interesting to see how it manages to annihilate us while preserving the infrastructure it needs to survive.

It doesn't. Its not like AI has a conciousness or an agenda. Nor does it have any form of creativity. The notion of an AI becoming a SkyNet - like supervillain is nowhere in sight.

 

[Huntn]

By the time we realize that an AI has surpassed our intelligence it will almost certainly be too late. A cage designed by chimps would not be successful in holding even our children. It's vanity to think that the protections we put in place will hold a super-human intelligence in a box.

It'll be interesting to see how it manages to annihilate us while preserving the infrastructure it needs to survive.

There're 2 aspects here:

• How much agency does the AI have? Is there a plug that can be pulled? Will we pull it or make excuses why we can’t pull it?
• How rapidly will we become dependent upon it?

 

[Eric]

You can see this robot becoming visibly irritated, it's subtle but seeming very real.

 

[Agent47]

You can see this robot becoming visibly irritated, it's subtle but seeming very real.

Quite possibly, but its trained behavior. You could have trained something very different, like laughing. Its behavor imposed from the outside, to mimic what people expect.

Similarly, you can ask an AI what it thinks or how it works. You'd get an answer, but that answer would just respond with what it learned (i.e. what its told to respond with), not what it really thinks.

 

[Yoused]

what it really thinks.

The thing about human thought is that it is dyed with chemicals. Our emotional framework creates an environment in which our reason resides. What we "really think" is not merely the convergent result of an array of algorithms but that result properly stewed in our feelings. A bot may be able to acquire the ability to mimic emotions, but actually embedding emotional content as a logic structure component is a completely different matter that may defy non-biological arrangements.

And what do we call people who act without responding to emotional cues? Is that not regarded as some sort of pathology (e.g., autism, psychopathy, sociopathy)? We have had emotionally defective people running the largest organizations (corporations, countries) in the world for most of history – if we cannot prevent such tasks being delegated to AIs, we deserve our fate.

 

[Agent47]

The thing about human thought is that it is dyed with chemicals. Our emotional framework creates an environment in which our reason resides. What we "really think" is not merely the convergent result of an array of algorithms but that result properly stewed in our feelings. A bot may be able to acquire the ability to mimic emotions, but actually embedding emotional content as a logic structure component is a completely different matter that may defy non-biological arrangements.

And what do we call people who act without responding to emotional cues? Is that not regarded as some sort of pathology (e.g., autism, psychopathy, sociopathy)? We have had emotionally defective people running the largest organizations (corporations, countries) in the world for most of history – if we cannot prevent such tasks being delegated to AIs, we deserve our fate.

That may well be the case. But AI simply does not work like a brain. It contains lots of pattern recognizers, but that's about it. No mind, no agenda. Heck, our brains have memories, associations... AI nothing even remotely like that.

 

[Eric]

Something a little more practical.

 

[fischersd]

Something a little more practical.

I sent the idea of a shopping app to first Apple then Google about 10 years (you need such tight integration with the OS, it needs to be one of them to do it). It also included doing a POS system. My marketing spin was Apple saves downtown USA! Allowing the small merchants to fight back to the likes of Amazon and Walmart.
All merchants keep their inventory and pricing up to date. Users have their wish list and target pricing that they want for their items. As they go about their day, the app will tell them if they're going near a store with their desired item at their desired price (or close).
Obviously, you can still see a map view of your items, with their current selling prices.
The POS system will also let merchants know how their pricing compares to their competitors.
Also had mapping in the store to where everything is. You'd manufacture cheap wifi repeaters that would allow you to geo-locate items in the store with precision. The barometer in the Apple Watch would allow visually impaired customers to use height to find their items in the store.
For most of us that hate shopping, the mapping of the store items would allow you to navigate through the store, getting everything on your list with the minimal amount of steps and time.
We could even add cashier-less checkout as a perk to people using the app (rolling out the door with everything in their cart, unchecked) - you could have your security personnel using AR systems to identify the customers that are using their app and what items should be in their cart - so you could have a second check without the customers being aware.

I sent it to Google, finally, after about 18 months of no response from Apple. *sigh*

 

[Jimmyjames]

Hope this is the correct thread! I saw a tweet from someone about a development in some LLM on the M2 Ultra:

https://twitter.com/ggerganov/status/1697262700165013689

Screenshot for anyone unable to use twitter

Apparently this kind of performance usually requires 4x high end gpus! What I am not clear on, is if this optimisation or technique would be applicable to high end discreet gpus also? Any insight from more knowledgable members would be appreciated.

 

[Altaic]

Hope this is the correct thread! I saw a tweet from someone about a development in some LLM on the M2 Ultra:

https://twitter.com/ggerganov/status/1697262700165013689

Screenshot for anyone unable to use twitter
View attachment 25681
Apparently this kind of performance usually requires 4x high end gpus! What I am not clear on, is if this optimisation or technique would be applicable to high end discreet gpus also? Any insight from more knowledgable members would be appreciated.

Very cool. Essentially, they generate tokens very fast using a small model, then feed those tokens into the large (slow) model and it reuses a bunch of them so it has to generate fewer tokens itself. And yes, it should work on discrete GPUs as well.

 

[Jimmyjames]

Very cool. Essentially, they generate tokens very fast using a small model, then feed those tokens into the large (slow) model and it reuses a bunch of them so it has to generate fewer tokens itself. And yes, it should work on discrete GPUs as well.

Thanks for the explanation.

 

[dada_dave]

In the same vein as @Jimmyjames and @Altaic ‘s posts:

https://twitter.com/karpathy/status/1691571869051445433

"How is LLaMa.cpp possible?" great post by
@finbarrtimbers
https://finbarr.ca/how-is-llama-cpp-possible/llama.cpp surprised many people (myself included) with how quickly you can run large LLMs on small computers, e.g. 7B runs @ ~16 tok/s on a MacBook. Wait don't you need supercomputers to work with LLMs?TLDR at batch_size=1 (i.e. just generating a single stream of prediction on your computer), the inference is super duper memory-bound. The on-chip compute units are twiddling their thumbs while sucking model weights through a straw from DRAM. Every individual weight that is expensively loaded from DRAM onto the chip is only used for a single instant multiply to process each new input token. So the stat to look at is not FLOPS but the memory bandwidth.Let's take a look:A100: 1935 GB/s memory bandwidth, 1248 TOPSMacBook M2: 100 GB/s, 7 TFLOPSThe compute is ~200X but the memory bandwidth only ~20X. So the little M2 chip that could will only be about ~20X slower than a mighty A100. This is ~10X faster than you might naively expect just looking at ops.The situation becomes a lot more different when you inference at a very high batch size (e.g. ~160+), such as when you're hosting an LLM engine simultaneously serving a lot of parallel requests. Or in training, where you aren't forced to go serially token by token and can parallelize across both batch and time dimension, because the next token targets (labels) are known. In these cases, once you load the weights into on-chip cache and pay that large fixed cost, you can re-use them across many input examples and reach ~50%+ utilization, actually making those FLOPS count.So TLDR why is LLM inference surprisingly fast on your MacBook? If all you want to do is batch 1 inference (i.e. a single "stream" of generation), only the memory bandwidth matters. And the memory bandwidth gap between chips is a lot smaller, and has been a lot harder to scale compared to flops.supplemental figure

Two notes I wanted to add:1) In addition to parallel inference and training, prompt encoding is also parallelizable even at batch_size=1 because the prompt tokens can be encoded by the LLM in parallel instead of decoded serially one by one. The token inputs into LLMs always have shape (B,T), batch by time. Parallel inference decoding is (high B, T=1), training is (high B, high T), and long prompts is (B=1, high T). So this workload can also become compute-bound (e.g. above 160 tokens) and the A100 would shine again. As your prompts get longer, your MacBook will fall farther behind the A100.2) The M2 chips from Apple are actually quite an amazing lineup and come in much larger shapes and sizes. The M2 Pro, M2 Max have 200 and 400 GB/s (you can get these in a MacBook Pro!), and the M2 Ultra (in Mac Studio) has 800 GB/s. So the M2 Ultra is the smallest, prettiest, out of the box easiest, most powerful personal LLM node today.

So the M2 Ultra is the smallest, prettiest, out of the box easiest, most powerful personal LLM node today. (for inference)