During the prompt embedding optimization, the embeddings are allowed to take on any vector in embedding space, instead one could use a continuous penalty for superposing tokens:
Consider one of the embedding vectors in the input tensor: nothing guarantees its exactly on, or close to a specific token. Hence the probabilities with respect to each token form a distribution, ideally that distribution should be one-hot (lowest entropy) and worst case all equal probability (highest entropy), so just add a loss term penalizing the entropy on the quasitokens, to promote them to take on actual token values.
I'm similarly puzzled by "uncured bacon" which afaik still uses naturally occurring nitrites. How they're allowed to call it uncured when it's clearly still cured is beyond me.
A lot of people use them together (cursor for IDE and claude code in the terminal inside the IDE).
In terms of performance, their agents differ. The base model their agents use are the same, but for example how they look at your codebase or decide to farm tasks out to lesser models, and how they connect to tools all differ.
Thanks!
To be honest, it started purely as a learning project. I was really inspired when llama.cpp first came out and tried to build something similar in pure C++ (https://github.com/nirw4nna/YAMI), mostly for fun and to practice low-level coding.
The idea for DSC came when I realized how hard it was to port new models to that C++ engine, especially since I don't have a deep ML background. I wanted something that felt more like PyTorch, where I could experiment with new architectures easily.
As for llama.cpp, it's definitely faster! They have hand-optimizing kernels for a whole bunch of architectures, models and data types. DSC is more of a general-purpose toolkit. I'm excited to work on performance later on, but for now, I'm focused on getting the API and core features right.
You just need a foundation of C/C++. If you already have that then just start programming, it's way better than reading books/guides/blogs (at least until you're stuck!). Also, you can read the source code of other similar projects on GitHub and get ideas from them, this is what I did at the beginning.
Both uses cublas under the hood. So I think it is similar for prefilling (of course, this framework is too early and don't have FP16 / BF16 support for GEMM it seems). Hand-roll gemv is faster for token generation hence llama.cpp is better.
The sibling link was the turning point -- articles about Tesla before the link were generally positive, articles after and including the link were negative.
Tesla could have more camera data in sum (that's not even clear - transmitting and storing data from all the cars on the road is no easy task - L4 companies typically pysically remove drives and use appliances to suck data off the hard drives), but Waymo has more camera data per car (29 cameras) and higher fidelity data overall (including lidar, radar, and microphone data). Tesla can't magically enhance data it didn't collect.
This is a crippling disadvantage. Consider what it takes to evaluate a single software release for a robotaxi.
If you have a simulator, you can take long tail distribution events and just resimulate your software to see if there are regressions against those events. (Waymo, Zoox)
If you don't, or your simulator has too much error, you have to deploy your software in cars in "ghost mode" and hope that sufficient miles see rare and scary situations recur. You then need to find those specific situations and check if your software did a good job (vs just getting lucky). But what if you need to A/B test a change? What if you need to A/B test 100 changes made by different engineers? How do you ensure you're testing the right thing? (Tesla)
And if you have a simulator that _sucks_ because it doesn't have physics-grounded understanding of distances (i.e. it's based on distance estimates from camera), then you can easily trick yourself into thinking your software is doing the right thing, right up until you start killing people.
Another way to look at it is: most driving data is actually very low in signal. You want all the hard driving miles, and in high resolution, so that you can basically generate the world's best unit testing suite for the software driver. You can just throw the rest of the driving data away -- and you must, because nobody has that much storage and unit economics still matter.
This is to say nothing of the fact that differences between hardware matter too. Tesla has a bunch of car models out there, and software working well one one model may not actually work well on another.
I don't think the Jaguars are particularly spacious or nice. They just got a good deal on the platform. If anything, given the commodity nature of vehicles I'd expect car quality to improve.
Cleanliness doesn't seem that related to how expensive the tech is either - if anything it would only go down if it ceased to affect willingness to pay. As it stands, clean cars are important to their customers. If usage increases, cleaning can ostensibly increase too, no?
> Cleanliness doesn't seem that related to how expensive the tech is either - if anything it would only go down if it ceased to affect willingness to pay.
If a service is necessarily expensive, it will more by wealthy people, who have a higher willingness to pay for cleanliness.
> As it stands, clean cars are important to their customers.
This varies depending on the customer.
> If usage increases, cleaning can ostensibly increase too, no?
Greater usage can lead to economies of scale that push down the cost of cleaning, but I think we're already at the scale (hundreds of cars) where most of the economies of scale have been reached. I expect it's close to linear from now on.
Higher willingness to pay does not matter unless the market is segmented. So if the tech gets cheaper, unless they explicitly make an expensive "oft-cleaned" tier and a less expensive "less-oft-cleaned" tier, what matters is average willingness to pay.
"Once the tech becomes cheap, expect the car quality and cleanliness to go down" -- here you were positing that the _average_ willingness to pay for cleanliness will go down enough to affect things, barring any market segmentation. And I disagree:
1. You're assuming that the reason _why_ Waymo's cars tend to be cleaner than your typical Uber / Lyft is to satisfy a wealthy clientele. This elides a big reason for the existing gap: Uber / Lyft drivers aren't professionally managed. You don't directly pay for your Uber driver's interior car cleaning when you buy a ride, but the salary of folks managing Waymo's fleets is factored directly into pricing. Even if Waymo's clientele were less wealthy, you have to clean your cars and pass the cost on to all users. Additionally, interior cameras are pretty motivating to not mess up cars!
2. You're assuming that the cars today don't get maximum utilization, and that with more utilization you'd see dirtier cars. This is a pretty bad assumption - the cars are being utilized about as heavily as you can hope. In SF for most of Waymo's existence demand has outstripped supply. And the cars are still very clean :)
So if the usage is the same, and peoples' expectations for cleanliness are the same, why would rate of cleaning change as time goes on for the service?
The only thing I can think is if no reasonable alternatives to Waymo arise - in that case, cleanliness could go down but it has less to do with the clientele / willingness to pay, and more to do with competition / monopoly.
As another note, I just don't see how cleaning-based market segmentation would make good operational sense. Is cleaning the car slightly less frequently really gonna help the bottom line? Is the price differential big enough at single-ride scale? Do even rental car companies do this for their fleets - the cheap ones still seem to clean their cars.
> That's exactly what I'm saying would happen. We already have it with Uber Black.
But I'm saying we _don't_ have that for Waymo, and it's very unlikely to happen, for many reasons. A big reason is simply that managing a fleet in heterogeneous fashion as you're describing (different cleaning schedules for the cars) doesn't really make sense IRL. It's a purely imagined scenario on your part.
> Incorrect.
Pray tell how I can pay for a cleaner car when there's only one option, car or no car?
> No, I'm not assuming that.
Then please explain how cars would get dirtier as the service scales up? If today is already seeing the cars at full utilization, barring a cost-cutting measure that determined that cleaning less frequently would be a significant cost savings (which is a big assumption on your part), then we should be seeing roughly how clean the cars will be into perpetuity.
> Again, Uber Black.
Uber Black achieves higher standards for cleaning by farming that out to the people renting out their personal vehicles. The drivers are incentivized to clean the cars more (than UberX drivers) to get more expensive fares.
But again, fleet management companies already do this for _all_ their cars. So for Waymo this is moot.
> A big reason is simply that managing a fleet in heterogeneous fashion as you're describing (different cleaning schedules for the cars) doesn't really make sense IRL.
Of course it does. Pre-Uber, we had both standard yellow cabs and black car services at different levels. (The main reason you see relative homogeneity within yellow cabs is that the government forces it by setting prices, not because of anything intrinsic about a fleet. Black cars are excluded from these rules.)
In shipping we can pay for different speeds and types of handling. On planes and trains we have different class tickets. In the rental car market, we have Hertz and we have rent-a-wreck. And even within Hertz, there are different car quality levels, which somewhat decreases flexibility (since you need to have more cars on hand than you would with a homogeneous fleet), but it's worth the upkeep to charge the wealthy customers more. Etc.
> Then please explain how cars would get dirtier as the service scales up? If today is already seeing the cars at full utilization, barring a cost-cutting measure that determined that cleaning less frequently would be a significant cost savings (which is a big assumption on your part), then we should be seeing roughly how clean the cars will be into perpetuity.
1. Tech prices come down, so the average customer willingness to pay for cleanliness comes down.
2. Services often launch with non-scalable attention to detail to control the initial public impression (eating the cost), and then relax over time.
3. Segmentation that's not feasible at the current scale but will be in the future.
> Segmentation that's not feasible at the current scale but will be in the future.
So you _do_ agree that willingness to pay is only helpful if there is segmentation.
> Pre-Uber, we had both standard yellow cabs and black car services at different levels
There is more to the gap between yellow cab and black car than cleanliness. Stuff like service / helping you with bags, ETAs, partitions between yourself and the driver, niceness of the car itself, etc.
I'm sure we'll see segmentation along the lines of vehicle size and capability, but I expect cleanliness to be the same across those segments.
> Services often launch with non-scalable attention to detail to control the initial public impression (eating the cost), and then relax over time.
I don't think cleaning is the burden you're making it out to be. These cars return to depot when their battery is down. If you're to clean them at all, you should clean them when they return for charging, and then to your set standard. It's not a big knob for controlling costs.
> Dooring people aside, what do you do if someone just leaves the door open when they leave their ride?!
Continue billing them for the ride and send an app notification or phone-call to their phone.
Other potential solutions: If the door is still not closed after n minutes, plead with passers-by, or offer a passing or nearby rider the chance to earn credit by closing the door.
