Design talent flees back to Apple, Apple buys advanced ML startup

[RockRock8]

Apple has hired back Sebastian de With, who famously worked on MobileMe, iCloud, and the Find My Friends app (now known as Find My, after Apple added item and device tracking). He has expressed great compliment and admiration for Apple's work on Liquid Glass, so I'm sorry haters, but Liquid Glass isn't going anywhere.

Apple has also hired another designer back, who went to work at a company I'll not name, but worked on bringing their "Advanced Voice Mode" to life for their LLM. His name is Brandon Walkin, and he worked on previous features like Apple's iPad cursor that adapts to each button.

Finally, even though @Cmaier posted about it, I thought it be fitting to include it here given all the bullshit negative narrative pushing by Gurman.

Apple has acquired a new startup focused on highly advanced machine learning built for Apple's human interface efforts around effortless UI, as well as non-invasive brain-computer interfaces. This ML startup was focused on creating the ability to interface with features like Siri by understanding the way you "mouth" out words. In other words, you would be able to talk to Siri without needing to speak loudly.

They focused on products like headsets and headphones. Most interestingly to users, these founding engineers were the same engineers who invented PrimeSense, which went on to become Face ID a few years after buying the start up. This ML start up has multiple patents and working technology, coupled with many engineers who have proven they can bring features to users. Apple has some very interesting ideas and products they're working on clearly, because this is a pretty groundbreaking ML startup.

Said by one of the investors (you can take it with a grain of salt because of that, but again these are Face ID designers and engineers specifically):

Congrats on the acquisition by Apple, the second largest in their history. In 2022, [a founder] cold emailed out of the blue. He barely even told me what he was up to. But from the very first call it was obvious I had met a force of nature, and a kindred spirit. These folks have really made magic, oh how I wish this wasn't in stealth so you all could see. But with the team inside of Apple, the magic is sure to hit us all soon enough. We are so happy to have had the chance to partner with them. Congrats to the team

Apple does not purchase companies for PR. They buy start ups from time to time because of advanced technology and engineers that know how to make products. And that this is one of the largest purchases ever is a sign. This isn't an "AI" purchase. This is a human interface design powered by ML purchase! Just like Touch ID, Face ID, Apple's Gaze UI, etc.

The chatbot "market" is coming to its predicted end, and Apple will lead the industry.

Twitter .com/sdw

Twitter .com/bwalkin

You can find announcements elsewhere for the ML purchase

 

[Cmaier]

de With’s own take on liquid glass, before liquid glass actually came out, was better thought out than what Apple actually did, in my opinion; this is why I am excited for his return to Apple (though previously he was a contractor, I think?)

Physicality: the new age of UI

There’s a lot of rumors of a big impending UI redesign from Apple. Let’s imagine what’s (or what could be) next for the design of iPhones, Macs and iPads.

www.lux.camera

 

[RockRock8]

It's not bad, but I personally and politely disagree that it's better. It feels timid rather than willing to go all the way. I'm not saying that has bearing on what he will do at Apple -- he's already proven his design talent and what he can do -- but I don't think it was large enough of a leap and thought too inside the box of what iOS 18 was. This is best shown in his video UI

Also it need not be said, but I will say it anyways: designing screenshots is not the same thing as designing an actual UI system. It's easy to get things "right" in highly controlled and non-usable photos. It's another thing to design a system that is good and actually working. This is why concepts are basically useless in UI.

Apple already specifically said they want physicality in UI based on what they learned and being inspired from their spatial UI.

If there is anything to be excited about with Sebastian de With, and there is, then it's based on his real work with MobileMe, iCloud, Find My Friends, Halide, etc plus his commentary about Apple designs.

My opinion, I guess. I'd like to hear if you or anyone else has anything about the other two things too!

 

[Cmaier]

The problem is that, as implemented, there is just stuff broken with liquid glass. In finder you can’t get at the column resizers sometimes. Windows now give you only a few pixels that you need to hit if you want to move them. etc. Usability got worse (particularly on Mac). De With is a guy that isn’t going to sacrifice usability just to make it look cool. I’m an advanced user, and I was using preview the other day and searching for text, and the pages came up out of order. I couldn’t remember how to cause the search results to be in order, and had to google it - but even with the answer, I couldn’t find the right button to click; it turns out, because of liquid glass, the button bar wasn’t legible because the content was white pages and the text on the button was white. Drove me nuts for days until I figured it out.

de With’s UI’s don’t suffer from that kind of thing. It’s possible to make it look great while also making everything legible, making UX features discoverable, etc.

 

[RockRock8]

I'm not saying things can't be improved, but largely what Sebastian created in that article doesn't address anything listed. That's why I said if there is something to be excited about with Sebastian -- and there is a lot -- then it's his previous work on features and apps that actually shipped.

Regardless it's always a good thing to have more good talent join. Who knows what Apple will change to improve, but they will improve everything as they always do, any potential missteps aside.

Also unstated but needs to be said is a lot of stuff I'm finding complaints about are actually bugs and rendering issues, not UI design. If that stuff is fixed (and it is being fixed), then I think a lot -- but not all -- of issues people have mentioned go away.

iOS 7 was largely a visual update. A lot of the core tenets of iOS remained the same. And while people intensely, INTENSELY bitched about readability with iOS 7, now people are saying iOS 7 is readable and iOS 26 is not. Right or wrong, I can't take social media or journalists seriously with these complaints on that specifically.

This post wasn't meant necessarily to discuss liquid glass specifically, but the 3 people/groups joining Apple. Though obviously some discussion makes sense.

Bloomberg and Mark Gurman is out with a media tour again crying about an 'exodus' of talent, blah. He was proven wrong repeatedly on every major thing he wrote about, and he's back bullshitting and MacRumors and associated mindlessly broadcasting his stupidity

 

[RockRock8]

because of liquid glass, the button bar wasn’t legible because the content was white pages and the text on the button was white

This specifically (as I was saying in the just posted reply) is a bug.

 

[Roller]

The problem is that, as implemented, there is just stuff broken with liquid glass. In finder you can’t get at the column resizers sometimes. Windows now give you only a few pixels that you need to hit if you want to move them. etc. Usability got worse (particularly on Mac). De With is a guy that isn’t going to sacrifice usability just to make it look cool. I’m an advanced user, and I was using preview the other day and searching for text, and the pages came up out of order. I couldn’t remember how to cause the search results to be in order, and had to google it - but even with the answer, I couldn’t find the right button to click; it turns out, because of liquid glass, the button bar wasn’t legible because the content was white pages and the text on the button was white. Drove me nuts for days until I figured it out.

de With’s UI’s don’t suffer from that kind of thing. It’s possible to make it look great while also making everything legible, making UX features discoverable, etc.

I've been a Mac user since 1984, and I don't recall ever having to look up how to do so many things. Admittedly, the number of functions in macOS has increased tremendously over the years, and maybe I'm not as sharp as I used to be, but so much seems less accessible now. I agree with your point about toolbar legibility in Preview – it shouldn't depend so much on what lies beneath. It sometimes makes me wonder if Apple's designers and engineers use their software.

 

[RockRock8]

On the ML start up, I've found more investor commentary, and every single person that has seen it says it's extremely advanced and revolutionary. There was actual products developed

They also possess over 80 patents (about half are pending). Can you contribute any insight on how far along a start up might be with 80 patents @Cmaier

 

[Cmaier]

On the ML start up, I've found more investor commentary, and every single person that has seen it says it's extremely advanced and revolutionary. There was actual products developed

They also possess over 80 patents (about half are pending). Can you contribute any insight on ie far along a start up might be with 80 patents @Cmaier

Their applications seem to be filed under the name “Q Cue Ltd.” I do not find 80 patents. I find:

6 U.S. patent applications
2 Hong Kong patent applications
2 PCT applications (that can serve as a basis for future applications in actual nations)
1 Australian application
1 actual issued patent (which is a US patent): https://patentimages.storage.googleapis.com/40/f7/24/934c399d7f92bb/US12254882.pdf

It’s possible that they have other patents that are assigned to other people/entities, of course, but I didn’t find anywhere near 80.

As for the single issued patent I found, here is claim 1:

While I do not provide legal advice, I have serious questions about this particular claim’s validity (and similar doubts about the remainder of the claims), because they may claim “lip reading using a computer” which, if a court agreed, would likely be invalid under 35 U.S.C. §101 in view of Alice v. CLS Bank. This is a very common way to invalidate patents that do not explain details of the hardware necessary to perform the task, and instead claim things like “detect… analyze…control…” while using ordinary computer hardware. (In fact, in a couple months I have a hearing where I will argue about this issue for a client you’ve heard of).

This issue would have been very apparent to Apple, so they are likely not as interested in Q’s patents as they are in the talent or in other I.P. (like physical designs for accomplishing all this).

 

[RockRock8]

Thank you for the in depth reply and effort!

The claim of 80 patents (actual or applied) comes from some website called Pitchbook, and it looks like some website that has information about how the company is structured or something I don't really know. So who knows if that's correct.

I did find that when I searched a founder's name, they would come up in patents but it wasn't assigned to Q ai ; however it was directly related. Is there a reason for that? Or is it just an error?

https://patents.justia.com/inventor/yonatan-wexler

https://patents.justia.com/inventor/avi-barliya

https://patents.justia.com/inventor/aviad-maizels

 

[Cmaier]

Thank you for the in depth reply and effort!

The claim of 80 patents (actual or applied) comes from some website called Pitchbook, and it looks like some website that has information about how the company is structured or something I don't really know. So who knows if that's correct.

I did find that when I searched a founder's name, they would come up in patents but it wasn't assigned to Q ai ; however it was directly related. Is there a reason for that? Or is it just an error?

https://patents.justia.com/inventor/yonatan-wexler

https://patents.justia.com/inventor/aviad-maizels

Hard to know. The way it works, at least in the U.S., is that patents belong to the inventors, unless they assign the patents to someone else (like a business entity). Usually you assign the application at the time you submit the application to the patent office.

It looks like some of the patent applications you link to are assigned to OTHER entities (e.g. “Orcam”). So these patents and applications may not belong to Q.ai at all.

Also, there is a difference between applications and patents - an application is worthless unless it becomes an issued patent - and a lot of these lists don’t distinguish the two.

 

[RockRock8]

Hard to know. The way it works, at least in the U.S., is that patents belong to the inventors, unless they assign the patents to someone else (like a business entity). Usually you assign the application at the time you submit the application to the patent office.

It looks like some of the patent applications you link to are assigned to OTHER entities (e.g. “Orcam”). So these patents and applications may not belong to Q.ai at all.

Also, there is a difference between applications and patents - an application is worthless unless it becomes an issued patent - and a lot of these lists don’t distinguish the two.

Based on these websites, does "Application" mean they're just getting started with it and wrote it and sent it, and "Grant" means they actual have the patent?

 

[Cmaier]

Based on these websites, does "Application" mean they're just getting started with it and wrote it and sent it, and "Grant" means they actual have the patent?

yes, essentially.

 

[RockRock8]

Were you able to check those links for the inventors? There's a lot more listed application/grants, which might be where that website got 80 patents quantity from

What do you make of it?

 

[Cmaier]

Were you able to check those links for the inventors? There's a lot more listed application/grants, which might be where that website got 80 patents quantity from

What do you make of it?

I checked on a few of the ones on those links, and some were assigned to OTHER companies, not Q.ai. (i.e. Orcam). It’s possible these guys work for multiple companies, or something else is going on.

 

[RockRock8]

I checked on a few of the ones on those links, and some were assigned to OTHER companies, not Q.ai. (i.e. Orcam). It’s possible these guys work for multiple companies, or something else is going on.

Well I'm not sure if the website I linked is accurate or not. I did see under Aviad's the majority of the list pertains to Q ai. Same with the other inventor. I'm only going off this website and only where it says "Applicant" or "assignee." I don't know much about patents or how to find or search them, and you're an expert in this, so I'm asking if there's a better way you're using

 

[RockRock8]

Well I'm not sure if the website I linked is accurate or not. I did see under Aviad's the majority of the list pertains to Q ai. Same with the other inventor. I'm only going off this website and only where it says "Applicant" or "assignee." I don't know much about patents or how to find or search them, and you're an expert in this, so I'm asking if there's a better way you're using

to answer my own question: they were granted multiple patents regarding this technology, including specific processes and sensors, and all within the same company. A lot of what they were doing was around IR/lasers and using that to achieve various features, even as advanced as recognizing users based on the same silent speech recognition technology. 80 patents appears to be accurate, both accepted and pending

It's also worth noting this is the second biggest acquisition ever, and Johny Srouji himself made a comment on it. Further investor commentary suggests they were very far into developing a product and this wasn't only a proof of concept anymore. Also there was apparently 100 engineers at the company as well, so this isn't just some random thing.

That there was zero rumor of it or even a hint before Apple announced this is also insightful

 

[Cmaier]

to answer my own question: they were granted multiple patents regarding this technology, including specific processes and sensors, and all within the same company. A lot of what they were doing was around IR/lasers and using that to achieve various features, even as advanced as recognizing users based on the same silent speech recognition technology. 80 patents appears to be accurate, both accepted and pending

It's also worth noting this is the second biggest acquisition ever, and Johny Srouji himself made a comment on it. Further investor commentary suggests they were very far into developing a product and this wasn't only a proof of concept anymore. Also there was apparently 100 engineers at the company as well, so this isn't just some random thing.

That there was zero rumor of it or even a hint before Apple announced this is also insightful

Do you have a list of granted patent numbers?

 

[RockRock8]

Here are some:

“US 12,340,808”

“US 12,254,882”

“US 12,216,749"

“US 12,205,595”

“US 12,142,282”

“US 12,142,281”

“US 12,142,280”

“US 12,131,739”

“US 12,105,785”

The rest I saw on those links

 

[Cmaier]

Here are some:

“US 12,340,808”

“US 12,254,882”

“US 12,216,749"

“US 12,205,595”

“US 12,142,282”

“US 12,142,281”

“US 12,142,280”

“US 12,131,739”

“US 12,105,785”

The rest I saw on those link

Looked at the first 4. All have the same 35 USC 101 issues, looks like. None seem to claim any special hardware (all that matters in a patent is the claims. All the stuff before that is just used to interpret the claims.)