Tesla was the real power grid guy. The scope of his invention from the generators at Niagara Falls power generation to the transformers to the motors is pretty impressive. More so given that he was eventually given the patents (originally issued to Marconi) for radio transmission.
The fact that Edison is pervasively over-credited is really another example of the highly visible executive claiming personal credit for the labors of employees.
Steinmetz contributed heavily to AC systems theory which helped understand and expand transmission. while Scott contributed a lot to transformer theory and design (I have to find his Transformer book.)
My sense is that the Gemini models are very capable but the Gemini CLI experience is subpar compared to Claude Code and Codex. I'm guess that it's the harness but since it can get confused, fall into doom loops, and generally lose the plot in a way that the model does not in Gemini Studio or the Gemini app.
I think a bunch of these harnesses are open source so it surprises me that there can be such a gulf between them.
It's not just the tooling. If you use Gemini in opencode it malfunctions in similar ways.
I haven't tried 3.1 yet, but 3 is just incompetent at tool use. In particular in editing chunks of text in files, it gets very confused and goes into loops.
The model also does this thing where it degrades into loops of nonsense thought patterns over time.
For shorter sessions where it's more analysis than execution, it is a strong model.
We'll see about 3.1. I don't know why it's not showing in my gemini CLI as available yet.
Gemini-3.0-flash-preview came out right away with the 3.0 release and I was expecting 3.0-flash-lite before a bump on the pro model. I wonder if they have abandoned that part of the Pareto/price-performance.
I couldn't find anything more recent that this but apparently it has made the streets safer for pedestrians too. "Traffic fatalities in the Congestion Pricing zone are down 40% from last year."
This video shows the systems being built and shipped with cooling, cabling, etc.
It’s pretty mind blowing what this crisis shows from the manipulation of atoms and electrons all the way up to these clusters. Particularly mind blowing for me who has cable management issues with a ten port router.
This interested me as a simple-ish approach to a tough-ish promblem. From the example Trip Planner file:
"""Command line interface to process a trip request.
We use Gemini flash-lite to formalize freeform trip request into the dates and
destination. Then we use a second model to compose the trip itinerary.
* This simple example shows how we can reduce perceived latency by running a fast
model to validate and acknowledge user request while the good but slow model is
handling it.
The approach from this example also can be used as a defense mechanism against
prompt injections. The first model without tool access formalizes the request
into the TripRequest dataclass. The attack surface is significantly reduced by
the narrowness of the output format and lack of tools. Then a second model is
run on this cleanup up input.
*
Before running this script, ensure the `GOOGLE_API_KEY` environment
variable is set to the api-key you obtained from Google AI Studio.
Sadly not! It lives in our FE monorepo at work, it just kind of jumped out at us as something to split out and demo once we started using it to test some of our ideas. Its something wed consider for sure, but for now the lack of an open repo is kind of a bit of tech debt in a way. Easier to dev in our monorepo to get something out fast.
Iran (and some neighbors) starts the new year on the Spring Equinox, the first day of spring. It’s named Now Ruz which translates to new day. Kinda makes sense to kick off the year at spring. It’s also pretty precise give that it’s an astronomical event. It dates back to at least Zoroastrian times (15th century BCE).
All the equinoxes and solstices are celebrated there. The winter solstice is named Yalda Night, which was a few nights ago and Christmas may be related to this astronomical event. There is also Mehran and Tirgan. Ancients did like to get together and party.
I like that. I'm in favor of a calendar that works that way. The spring equinox does make a lot of sense. It's when plants start growing again where most people live in the northern hemisphere. The southern hemisphere seasons being the opposite of the north actually makes an equinox more equitable choice for a global calendar start/end point than a solstice.
"dec"ember used to be the 10th month which puts old new years at the beginning of march, a few weeks before the equinox. also, i haven't even noodled this in my head much but I think it works out in which direction it slips the date (hmmm maybe not), but it wasn't till Pope Gregory that it was realized there was a 100 year non leap year problem serious enough to impact the calendar.
The future of training seems to, at least partly, be in synthetic data. I can imagine systems where a “data synthesizer” LLM is trained on open data and probably some licensed data. The synthesizer then generates data “to spec” to train larger models. MOE type models will likely have different approaches in so far as something like a Mathematical expert likely gets a long way with training data from out of copyright works by Newton, Euler, et al.
It's already how we fine-tune open source LLMs. All of them live off data exfiltrated from GPT-4. And it seems to help closing the gap fast. Microsoft had a whole family of papers on this idea: TinyStories, Phi-1, Phi-1.5, Phi-2...
Synthetic data has many advantages - it is free of copyright issues, the downstream models can't possibly violate copyright if they never saw the copyrighted works to begin with.
It is also more diverse and we can ensure higher average quality and less bias. It can also merge information across multiple sources. Sometimes we can filter using feedback from code execution, simulations, preference models or humans. If you can "execute" the LLM output and get a score, you're on to a self improving loop. LLMs can act as agents, collecting their own experiences and feedback.
I think GPTs are a ploy by OpenAI to collect synthetic data with human-in-the-loop and tools, to improve their datasets. This would also be in-domain for users and for LLM errors. They would contain LLM errors and the feedback. Very good data, on-policy. My estimations for 100M users at 10K tokens per month per user is 1T synthetic tokens per month. In a year they double the size of the GPT-4 training set. And we're paying and working for it.
But fortunately 12 months after they release GPT-5 we will recover 90% of its abilities in open source models.
> Synthetic data has many advantages - it is free of copyright issues, the downstream models can't possibly violate copyright if they never saw the copyrighted works to begin with.
I feel like we don't know if this is true or not. If we decide models trained on copyrighted data aren't fair game, it's possible we'll decide "laundered" data also isn't.
I mean, maybe that's not feasible. And I hope we don't decide training on copyrighted material is bogus anyway. But I don't think we know yet.
But also - you can totally violate copyright of something you never saw.
Sure, but what matters for copyright is output, not input. For now.
If we make the (poor, imo) decision to prevent training on copyrighted data, that's a restriction on the training process, not on its result.
And in the world where we're making bad decisions to put legal restrictions on the training process, "can't train on data obtained by models that were trained without these restrictions" seems on the table.
reply