> The core issue is that Amazon envisioned Alexa as a product that would help it increase sales.
As someone who knows folks that worked on Amazon Alexa (back in its earliest days), this was almost never the focus of Alexa. It might seem cool or trendy to constantly hate on Amazon here on HN, and then somehow tie that to some comment on free market capitalism, Amazon did in fact have very normal ambitions with Alexa, build a smart assistant. Bezos, particularly envisioned building some computer from star-trek as his inspiration.
Alexa in its early days was also quite impressive for what they managed to do. It was an example of early ML scaling. They rented hundreds of houses around Seattle, had voice actors talk in different locations in the houses, collected all that data and trained a speech recognition model that was SOTA at the time. Unfortunately while their voice recognition was great, they didn’t apply this scaling approach to natural language understanding. They instead went the root of trying to manually understanding grammar, and iterating all the thousands of possible sentence reconstructions in a large decision tree to help Alexa answer your queries. This of course was hopeless, but this was also almost 5 years before OpenAI released GPT-3 so you can’t fault them for doing essentially what Siri did.
Now the issue with Alexa is like most issues big organizations face, they are slow moving and cannot pivot quickly. What they need to do, is actually hard and will need to be done by essentially a monarch CEO, he needs to delete all the work the Alexa teams have done over the last 5 years in 1 swoop and replace it with LLM’s. Many folks in Amazon made their career on improving the Alexa stack, they will of course mostly be defensive and argue vehemently against deleting entirety of Alexa tech “LLM’s hallucinate too much, we need a combined approach”, they will plead, but the stack needs to be deleted, the entire organization needs to pivot to LLMs. Apple has managed to take steps towards this, I won’t be surprised if Amazon soon will, or maybe they won’t, depends on their ability to not be disrupted.
If they are guilty of anything, it’s being unable to do good research to discover GPT-3 on their own (though tbf no one except OpenAI did this), and then being unable to ditch all the technical debt they have accumulated over 5+ years and pivot completely into LLM’s.
As someone who knows folks that worked on Amazon Alexa (back in its earliest days), this was almost never the focus of Alexa. It might seem cool or trendy to constantly hate on Amazon here on HN, and then somehow tie that to some comment on free market capitalism, Amazon did in fact have very normal ambitions with Alexa, build a smart assistant. Bezos, particularly envisioned building some computer from star-trek as his inspiration.
Alexa in its early days was also quite impressive for what they managed to do. It was an example of early ML scaling. They rented hundreds of houses around Seattle, had voice actors talk in different locations in the houses, collected all that data and trained a speech recognition model that was SOTA at the time. Unfortunately while their voice recognition was great, they didn’t apply this scaling approach to natural language understanding. They instead went the root of trying to manually understanding grammar, and iterating all the thousands of possible sentence reconstructions in a large decision tree to help Alexa answer your queries. This of course was hopeless, but this was also almost 5 years before OpenAI released GPT-3 so you can’t fault them for doing essentially what Siri did.
Now the issue with Alexa is like most issues big organizations face, they are slow moving and cannot pivot quickly. What they need to do, is actually hard and will need to be done by essentially a monarch CEO, he needs to delete all the work the Alexa teams have done over the last 5 years in 1 swoop and replace it with LLM’s. Many folks in Amazon made their career on improving the Alexa stack, they will of course mostly be defensive and argue vehemently against deleting entirety of Alexa tech “LLM’s hallucinate too much, we need a combined approach”, they will plead, but the stack needs to be deleted, the entire organization needs to pivot to LLMs. Apple has managed to take steps towards this, I won’t be surprised if Amazon soon will, or maybe they won’t, depends on their ability to not be disrupted.
If they are guilty of anything, it’s being unable to do good research to discover GPT-3 on their own (though tbf no one except OpenAI did this), and then being unable to ditch all the technical debt they have accumulated over 5+ years and pivot completely into LLM’s.