The Ai thread

[Nycturne]

I think this bit from Doctrow is particularly relevant in light of the very recent (and almost successful) supply chain attack on OpenSSH.

Here's a fun AI story: a security researcher noticed that large companies' AI-authored source-code repeatedly referenced a nonexistent library (an AI "hallucination"), so he created a (defanged) malicious library with that name and uploaded it, and thousands of developers automatically downloaded and incorporated it as they compiled the code[…]

Reading through the linked story this quote described, I had to pick up my jaw off the floor when it’s revealed that not only did a bunch of repos just start picking up this “typo-squatting” package the AI hallucinated, but Alibaba started referencing it in repo READMEs. And just to pour salt into the wound, the group that owned the official packages even started referencing it in their repo READMEs. It just seems nuts that we could very well be sleep walking into an era where attacks like these become even easier under the name of improved productivity.

The bit that describes my feelings about AI in programming better than I can, and why the above is such a concern:

Automation centaurs are great: they relieve humans of drudgework and let them focus on the creative and satisfying parts of their jobs. That's how AI-assisted coding is pitched […]

But a hallucinating AI is a terrible co-pilot. It's just good enough to get the job done much of the time, but it also sneakily inserts booby-traps that are statistically guaranteed to look as plausible as the good code […]

This turns AI-"assisted" coders into reverse centaurs. The AI can churn out code at superhuman speed, and you, the human in the loop, must maintain perfect vigilance and attention as you review that code, spotting the cleverly disguised hooks for malicious code that the AI can't be prevented from inserting into its code.

Pluralistic: Humans are not perfectly vigilant (01 Apr 2024) – Pluralistic: Daily links from Cory Doctorow

pluralistic.net

 

[dada_dave]

Look at how useless we've become as human beings, can't even even formulate an article on our own any longer, and this is just the beginning.

I know you’re not a big fan of the author but this is a pretty hilarious FU to Google and their algorithm (and their AI) masquerading as a best printer article.

Best printer 2024, best printer for home use, office use, printing labels, printer for school, homework printer you are a printer we are all printers

After a full year of not thinking about printers, the best printer is still whatever random Brother laser printer that’s on sale.

www.theverge.com

 

[dada_dave]

Meet the new world same as the old one:

Microsoft's Copilot image tool generates ugly Jewish stereotypes, anti-Semitic tropes

Neutral prompts such as "Jewish boss" output offensive images.

www.tomshardware.com

 

[dada_dave]

Humane AI Pin review: not even close

AI gadgets might be great. But not today, and not this one.

www.theverge.com

A fairly brutal review that I actually felt was being far too kind … basically all the functionality of this thing already exists on a smartwatch or something like the Siri/action button on the iPhone 15 and this is often clunkier and less useful than those form factors for doing the exact same thing. I get that they went all in on the TNG communicator aesthetic, but it turns out that form factor just doesn’t work all that well compared to what we already have. Maybe someone can come up with a reason for a AI pin/broach to exist, but nothing about this gives me much hope, even the author’s own attempts to justify the existence of the device.

 

[rdrr]

So I was just on a Zoom meeting, and in the waiting room was a Note Taking AI bot from read.ai. Of course we didn't let it in, because the meeting was about sensitive topics that shouldn't be recorded by an unknown AI application. Not to mention the meeting was in a 2 party consent recording state.

But... That goes to show how people are just not aware of what happens when they sign up for these services. If you use your work profile in google (we are a GSuite shop), to sign up for these services, it will follow you around like the plague. In some cases other more predatory note taking apps, have been know to gather your entire domain users and or contacts and attach the ai note taking app to those profiles. Not much you can do about it either, other than educating the user base, and making waiting rooms the default for meetings.

 

[dada_dave]

Ah the AI apocalypse is real:

First AI-generated rom-com is due this summer -- and the trailer puts Hallmark Channel to shame

The popular TV company's first original feature is made with Runway ML and Midjourney.

www.tomshardware.com

They’ve gotten to the Hallmark romance movie stage. Sorta.

 

[dada_dave]

Humane AI Pin review: not even close

AI gadgets might be great. But not today, and not this one.

www.theverge.com

A fairly brutal review that I actually felt was being far too kind … basically all the functionality of this thing already exists on a smartwatch or something like the Siri/action button on the iPhone 15 and this is often clunkier and less useful than those form factors for doing the exact same thing. I get that they went all in on the TNG communicator aesthetic, but it turns out that form factor just doesn’t work all that well compared to what we already have. Maybe someone can come up with a reason for a AI pin/broach to exist, but nothing about this gives me much hope, even the author’s own attempts to justify the existence of the device.

Marques Brownlee likewise gave the pin a poor review but rather than link his video on that which would just retread many of the same points as above, I’m linking the video he made in response to the … discourse about negative reviews and people jokingly/seriously accusing high profile reviewers like him of destroying companies.

 

[dada_dave]

A good read:

AI isn't useless. But is it worth it?

AI can be kind of useful, but I'm not sure that a "kind of useful" tool justifies the harm.

www.citationneeded.news

 

[ArgoDuck]

^ That’s a great read. Molly White’s experience pretty much mirrors my own in every detail, so far anyway.

On the topic of LLMs being useful for some things there’s something I’m trying right now that could be useful in transforming large amounts of textual data into a quantitative, normalized representation. Establishing whether the transformation is reliable (reproducible) and valid is the key problem, necessitating that I create a formal model and algorithm to cross-validate. Ironically, doing so may well obviate using LLMs for this task in the first place!

But i think it’s probably worth doing anyway, just as another datapoint in the map of LLM capabilities and associated limitations.

 

[dada_dave]

On the state of LLM’s as described by a 2014 comic

More Bizarro parrots

Today’s Bizarro returns to the talking parrot trope: Paradoxical animal communication. The parrot clearly understands the question and responds in English, while denying that it understands.

arnoldzwicky.org