Fundamentally how is this any different from what Google or Meta or Comcast or AT&T do? Comcast knows everything that goes to the TV and sells that data. At&T sells your browsing data… Those are services you pay for monthly.

Sure the method is different but it’s the same goal. Company x learns your interests so It can monetize you by selling to advertisers

AT&T sounds like the same thing, Google sounds different because they theoretically claim to not sell your data, and instead sell ads, and Google can show you an ad you want to see because Google knows you so well. It doesn’t precisely sell you to advertisers in the same way.

Anyways, the whole thing sucks for consumer privacy and needs to be outlawed. The problem is that companies come up with unique, tricky ways of exploiting you, and people can never fully understand it without a lot of effort. Someone might be ok using Google and seeing contextual ads, but wouldn’t be ok if they knew Google was saving a screenshot of their browser every second and uploading and reselling it. The first can feel innocuous, the second feels evil.

>Fundamentally how is this any different from what Google or Meta or Comcast or AT&T do?

It's all garbage all the way down.

Why do you think it's different? At first glance it seems more or less the same thing to me.

I have this book for my kid and love it!

Seems the innovation of LLMs and these first movers is diminishing. Claude is still just chat with some better UI

Congrats on the launch. I love this idea, excited to check it out. I wonder how this fits in with the probable rise of AR glasses

Thanks! That would definitely be cool! But for AR glasses I would be keen to see e.g. how this street looked like 100 years ago or smth.

This has been my go to podcast for bedtime or when I can’t sleep… the broad topics, depth of discussion and tone are all fantastic… the ONLY thing that bugs me is the volume of guests microphones not being equalized, so you get some guests on the same episode being so much quieter than others

I hate to say this out loud but I keep going back to some particular episodes to help me nod off when insomnia grips. I'm not sure I know how the episode on the gold standard ends and I must have listened to it more than 10 timss.

The Pudding is one of the bright spots of the internet for me. Does anyone have any recommendations for other new / blog interestings websites on the same level?

My old boss was a tour rider in the early 90’s - he told me in 2012 that tiny motors were being used. I believe him.

Part of me can’t help but think that scientific journals as billion dollar industries need to keep the status quo of how articles are written, where they get submitted and who reviews them. Even though per review today is failing

Maybe it’s selection bias, but it’s amazing how the comments section for this post has so much more punctuation than a typical HN post; fascinating

“Metrics: We combine user metrics and offline eval metrics, and employ both human and automated evaluation, particularly using LLM-as-a-judge techniques”.

I’m curious to know what people are doing to measure whether the customer got what they were looking for. Thumbs up/down seems insufficient to me.

The ability of the LLM to perform purely depends on having good knowledge of what is going to get asked and how, which is more complex than it sounds

What techniques are people having success with?

Training a 2nd agent as a qualitative evaluator works pretty well "LLM-as-a-judge". You train it with labeled critiques from experts, iterate a few times, then point it to your ground truth human-labelled-data ("golden dataset"). The quantitative output metric is human2ai alignment on the golden dataset, mix that with some expert judgment about the critique output by the ai as well.

Works pretty well for me, where you can typically get within the range of human2human variance.