This kind of reminds me of my brother's first trip to Taiwan.
This was in the 1980s. (Before cellphones.) The guy who was supposed to pick him up didn't show up because his motorcycle broke down. And couldn't get it fixed because it was Chinese New Year.
My brother arrived, not speaking any Chinese, surrounded by people who didn't speak any English. Who were having the biggest party that my brother had ever seen, and kept giving him food and inviting him places! He had absolutely NO idea what was going on!
After a few days of this, the person who was supposed to meet him finally managed to arrive, and my brother was hooked. Spent most of the next two decades in Taiwan.
Currently, in 2025, it is not possible in most fields for a random expert to produce a machine checked proof. The work of everyone in the field coming together to create a machine checked proof is also more work for than for the whole field to learn an important result in the traditional way.
This is a fixable problem. People are working hard on building up a big library of checked proofs, to serve as building blocks. We're working on having LLMs read a paper, and fill out a template for that machine checked proof, to greatly reduce the work. In fields where the libraries are built up, this is invaluable.
But as a general vision of how people should be expected to work? This is more 2035, or maybe 2045, than 2025. That future is visible, but isn't here.
It's interesting that you place it 10 or 20 years from now, given that MetaMath's initial release was... 20 years ago!
So it's not really about the planets not being in the right positions yet.
The roman empire lasted for centuries. If they wanted to do rigorous science, they could have built cars, helicopters, ... But they didn't (in Rome, do as the Romans do).
This is not about the planets not being in the right position, but about Romans in Rome.
I could believe you, an internet stranger. And believe that this problem was effectively solved 20 years ago.
Or I could read Terry Tao's https://terrytao.wordpress.com/wp-content/uploads/2024/03/ma... and believe his experience that creating a machine checkable version of an informal proof currently takes something like 20x the work. And the machine checkable version can't just reference the existing literature, because most of that isn't in machine checkable form either.
I'm going with the guy who is considered one of the greatest living mathematicians. There is an awful lot that goes into creating a machine checkable proof ecosystem, and the file format isn't the hard bit.
20x the work of what? the work of staying vague? there is no upper limit to the "work" savings, why not be 5 times vaguer, then formal verification can be claimed to be 100x more work.
If ultimate readership (over all future) were less than 20 per theorem, or whatever the vagueness factor would be, the current paradigm would be fine.
If ultimate readership (not citation count) were higher than 20 per theorem, its a net societal loss to have the readers guess what the actual proof is, its collectively less effort for the author to formalize the theorem than it would be to have the readers guess the actual proof. As mathematicians both read and author proofs, they would save themselves time, or would be able to move the frontier of mathematics faster. From a taxpayer perspective, we should precondition mathematics funding (not publication) on machine readable proofs, this doesn't mean every mathematician would have to do it, if some hypothetical person had crazy good intuitions, and the rate of results high enough this person could hire people to formalize it for them, to meet the precondition. As long as the results are successfully formalized, this team could continue producing mathematics.
Yes, many things can cause dementia. Repeated traumatic brain injuries can cause dementia.
But the leading form of dementia is Alzheimer's. Somewhere in the order of 40% of us are expected to get Alzheimer's before we die. The list of things that have been demonstrated to cause Alzheimer's is much, much shorter.
For the last 40 years, the leading theory about Alzheimer's is that it is caused by the beta-amyloid plaques that are found in the brain after death. This theory has produced exactly zero treatments that meaningfully affect clinical symptoms, despite many drug trial and literally billions in research per year. Seriously, between various sources, we've spent something like a quarter of what it cost to put man on the Moon. (It is hard to make a precise comparison, because a lot of that funding was private.)
This single study represents more progress on effective treatments of Alzheimer's than all of that work combined. The importance of the result should not be dismissed.
This is a general problem in many technical fields.
People in a technical field, learn to "chunk" complex phrases. Their natural communication style becomes complex. Which makes them hard to understand to those outside of the field. If they want to be understood, the solution isn't to try to educate the world. It is to educate themselves. To learn how to write simply and directly.
Depending on the readability test used, the section up to "Introduction" - which is supposed to be readable - is somewhere between advanced high school and university. See https://www.online-utility.org/english/readability_test_and_... or other free tools to test it. That's bad. The percentage of Americans who can read this text is below the percentage who could read, say, a plain language version written in Spanish. We should expect people to misunderstand. We should not expect this paper to convince.
Papers like this are designed to fit into the conventions that allow knowledge to compound. Not that the conventions are perfect at doing this.
I would suggest that rather than changing this convention in a big way, there needs to be good pathways for communicating the most important takeaways to the general public. Unfortunately, there's kind of a chasm between academia and popular science.
You are providing the standard excuse. It is our job to advance knowledge. It is someone else's job to communicate it to a broader audience. It's just too bad that nobody is stepping up and doing that other job.
I don't buy it. In my experience, most scientific papers can easily be rewritten into simpler language. The act of trying to do so often catches mistakes - thereby immediately improving how well we are advancing knowledge. The resulting paper is easier to read. This makes it more likely to become better known. Both within its subfield, and in a broader audience.
The habit of doing this makes us better communicators. Which also helps academics in various other parts of their job. Including teaching the next generation.
Furthermore, easier to read papers are easier for science popularizers to understand. Which makes it more likely that the work will be popularized.
Yes, it is tempting for academics to deflect responsibility for their role in being understandable. But it is a mistake for them to do so. Their ability to communicate in an understandable way is their responsibility. The few that take up that responsibility benefit themselves.
I'm not saying it's someone else's job to communicate to a lay audience. Simply that a research paper doesn't have to be a self-contained device for doing that and accurately describing the research to people who already have a lot of background knowledge on the topic and methods.
I guess I will say that I have thought for a long time that serializing research into linear documents seems archaic at this point.
It would be nice if academics would move to BOTH publishing the technical write up, AND a more understandable write up of their interpretation of the result (in more detail than the one liner which is in all abstracts.)
The technical writeup is necessary. It's what spells out what they specifically claim to have done, and the specific results. "Specific" being highly technical and fundamental in the scientific community understanding the paper correctly. In particular, the in-depth statistics of many such papers is simply too complex for most of the population to understand, and that's fine. The technical write-up uses terms of art which do not mean what civilians read in them. (And while it's hard to do studies larger than this one, this is all the more essential in smaller studies.)
The interpretation would be useful because it's just plain dangerous to let your PR department write that. Even if they consult you. And it is interesting to focus on what the scientists themselves think they achieved. Both what they deliberately went for, and any ancillary result they think they notice. In this case in particular, they are very focused on this safety aspect, and they seem to not want to give too much attention to the efficacy aspect (which they probably did not plan for and is then suspect.)
The Game of Life is Turing complete. And therefore a complete analysis of how to write programs in it would imply a solution to the Halting problem. Which is impossible.
Yes, over-utilization is a natural response to being undercharged. And being undercharged is a natural result when investors are throwing money at you. During bubbles, Silicon Valley often goes to "lose money, make it up with scale". With the vague idea that after you get to scale, THEN you can figure out how to make money. And fairly consistently, their idea for how to make money is "sell ads".
Past successes like Google encourage hope in this strategy. Sure, it mostly doesn't work. Most of of everything that VCs do doesn't work. Returns follow a power law, and a handful of successes in the tail drive the whole portfolio.
The key problem here doesn't lie in the fact that this strategy is being pursued. The key problem is that it is rare for first mover advantages to last with new technologies. That's why Netscape and Yahoo! aren't among the FAANGs today. The long-term wins go to whoever successfully create a sufficient moat for themselves to protect lasting excess returns. And the capabilities of each generation of AI leapfrogs the last so well that nobody has figured out how to create such a moat.
Today, 3 years after launching the first LLM chatbot, OpenAI is nowhere near as dominant as Netscape was in late 1997, 3 years after launching Netscape Navigator. I see no reason to expect that 30 years from now OpenAI will be any more dominant than Netscape is today.
Right now companies are pouring money into their candidates to win the AI race. But if the history of browsers repeats itself, the company that wins in the long-term would launch in about a year from now, focused on applications on top of AI. And its entrant into the AI wars wouldn't get launched until a decade after that! (Yes, that is the right timeline for the launch of Google, and Google's launch of Chrome.)
Investing in silicon valley is like buying a positive EV lottery ticket. An awful lot of people are going to be reminded the hard way that it is wiser to buy a lot of lottery tickets, than it is to sink a fortune into a single big one.
> Today, 3 years after launching the first LLM chatbot, OpenAI is nowhere near as dominant as Netscape was in late 1997.
Incorrect. There were about 150 millions Internet users in 1998, or 3.5% of total population. The number grew 10 times by 2008 [0]. Netwcape had about 50% of the browser market at the time [1]. In other words, Netscape dominated a small base and couldn’t keep it up.
ChatGPT has about 800 millions monthly users, or already 10% of total current population. Granted, not exclusively. ChatGPT is already a household name. Outside of early internet adopters, very few people knew who Netscape or what Navigator was.
Furthermore my point that the early market leaders are seldom the lasting winners is something that you can see across a large number of past financial bubbles through history. You'll find the same thing in, for example, trains, automobiles, planes, and semiconductors. The planes example is particularly interesting. Airline companies not only don't have a good competitive moat, but the mechanics of chapter 11 mean that they keep driving each other bankrupt. It is a successful industry, and yet it has destroyed tons of investment capital!
Despite your quibbles over the early browser market, my broader point stands. It's early days. AI companies do not have a competitive moat. And it is way to premature to reliably pick a winner.
Netscape in 1997/1998 had about 90% of the target market.
OpenAI today has about 30% of the target market, maybe? (seeing as how every single Windows installation comes with copilot chat already, it's probably less. Every non-tech user I know has already used copilot because it was bundled and Windows prompted them into using it with a popup. Only one of those non-tech users even heard of OpenAI, maybe 50% of them have heard that there are alternatives to Copilot, but they still aren't using those alternatives)
When you break down how budgets have changed, the two biggest drivers of tuition increases are the growth of administration, and fancy amenities like sports facilities.
The cost of the person in front of the blackboard has not been increasing.
Ok, the prior link was comparing it to EU though, so perhaps costs for professors there went down even more, as professors make less there compared to US
The problem is that any hiring test that blacks and whites pass at different rates, is presumed racist. Never mind that the real issue might be that the blacks went to worse schools and received a worse education. Never mind that there is a big body of research showing that ability tests are a more effective way to hire good employees than interviews. If the ratio of blacks to whites hired is different than the ratio that apply, you are presumed to be racist and in violation of the Civil Rights Act.
So a company that needs to hire literate people can no longer, as used to be standard, allow high school students to apply and give them a literacy test. But they can require college.
Therefore college has become a job requirement for a plethora of jobs whose actual requirement is "literate". Jobs that people used to be able to do out of high school, and jobs that could still be done by plenty of high school graduates. That this has become so ubiquitous lead to an increased demand for college. Which is one of the factors driving tuition up.
(My suspicion is that an ability test would lead to a less racist outcome than requiring college. Why? Because minority families struggle more to afford college.)
You'll occasionally see people point out that requiring a college degree has all the same legal problems as requiring a hiring exam does. And those people are correct in terms of the judgments that impose our terrible precedents. They're all just as negative on degree requirements as they are on performance requirements.
But as a matter of empirical reality, our enforcement system declines to prosecute employers who require degrees, because requiring degrees is morally good and requiring exams is morally bad.
The rules about what's allowed don't actually derive from the law. We have laws that forbid everything, accompanied by selective prosecution of only the things that certain people disapprove of.
I mean we don't need laws like this. Precedents like this are actually dangerous because they make the law ambiguous, opening it up to selective enforcement. Instead the law should just be read as is and courts should not find new discriminations in ones not mentioned by the legislature
> (My suspicion is that an ability test would lead to a less racist outcome than requiring college. Why? Because minority families struggle more to afford college.)
This might have been true when the United States was mostly white, and "minority" specifically referred to the black population who was mostly descended from slaves brought to the US mainland pre-1808, or to an even small number of native Americans. Today, when the US population is significantly more ethnically diverse, and "minority" just means "anyone nonwhite, regardless of where they came from or what their family history is", there's a lot more variation in exactly how ability to afford college correlates with ethnicity.
While minority technically means what you said, in practice people only care about those identifiable nonwhite groups who are doing poorly.
The result is that Academia is broadly in support of discriminating against certain identifiable minorities, despite their suffering well-known histories of discrimination. The logic is literally that the current success of Asians and Jews means that they are now in the oppressor class, and so should give up opportunities in the name of achieving equity. The same universities that used to discriminate against Asians and Jews out of simple racism, now wish to discriminate against Asians and Jews because they are trying to NOT be racist.
Many in my generation (I'm in my mid-50s) find this twist absurd beyond belief.
All of this is because academia and educational institutions have a tremendous amount of power this way. They can select for ideological compliance instead of actual competence. And this is a desirable property for the rulers because they can weed out those who are likely to destabilise them if they were able to show a valuable alternate path by example.
Why spend so much money on an "education" if you could become successful by simple being competent.
The tech sector was like that at first, but then came the degree requirement and the HR ladies. It was a short run and now they are very mad that some people became successful without needing to bow to the dominant ideology.
and yet ... that's not what the case you referenced says at all. Justia's own summary, from your link:
> Even if there is no discriminatory intent, an employer may not use a job requirement that functionally excludes members of a certain race if it has no relation to measuring performance of job duties. Testing or measuring procedures cannot be determinative in employment decisions unless they have some connection to the job.
They worked at a power plant, a place where dumb mistakes can cause explosions and kill people. The power plant wasn't racist and hired blacks into the labor department, but because it was just manual labor that department paid worse than the other more technical departments.
When SCOTUS found against the power company they sent a clear message that merely being a technical, safety-critical job was an insufficient basis to establish a need to test people for intelligence. And as it's hard to argue that testing isn't needed for people who could cause massive power outages but is for <job X>, that was widely interpreted to ban such aptitude testing for any kind of job.
> When SCOTUS found against the power company they sent a clear message that merely being a technical, safety-critical job was an insufficient basis to establish a need to test people for intelligence.
That's your interpretation. The court regarded the test used the power company as unrelated to the demands of the job. You're welcome to disagree with the court about that (as the company probably did), but don't misrepresent their actual position.
The court regarded the test as unrelated to the demands of the job, despite a large body of research showing that the test is generally predictive of performance on a wide class of jobs, which that job is in.
The bar was set to, "You must have good statistical evidence that this test is applicable to this job." That is an extremely high bar, that very few private companies are ever going to be able to collect sufficient data to establish.
In fact the only organization that I'm aware of which can pass that bar is the US military. Which is why they are allowed to use ability tests in hiring and initial promotions that no private company would be allowed to use.
I didn't claim otherwise. It should however be self evident that courts have no solid basis on which to make such a call. Judges don't know anything about running power plants, hiring employees, or really anything on that matter.
This is why human rights laws are always terrible in practice. They require the courts to make decisions well outside the bounds of their expertise.
> Judges don't know anything about running power plants, hiring employees, or really anything on that matter.
This is why both parties hire the best lawyers they can, and the best expert witnesses, to make the case for their side. We cannot require that every decision in the world be made by experts in the decision domain - sometimes they have to be made by people we've entrusted to make decisions (and in particular, decisions intended to be guided by the law).
>The problem is that any hiring test that blacks and whites pass at different rates, is presumed racist. Never mind that the real issue might be that the blacks went to worse schools and received a worse education.
Your first sentence is the result of bigotry against those with "enhanced" melanin content, not the cause.
The cause is laid out in your second sentence.
Resolve the systemic bigotry (not just against those with enhanced melanin content, but against those with the least resources as, at least in the US, most schools are paid for by local property taxes, making the poorest areas the ones with the worst schools) and put us all on a level playing field and we'll be a much fairer society IMNSHO.
A politically powerful minority calls ability tests racist because they make minorities look bad. An opposing offensive minority uses those same ability tests as evidence that minorities are simply inferior. Courts ruled that those using the tests should be presumed racist because the results show racial differences.
The result of all of this is a policy, meant to help minorities, that fails them. At great expense to all of us.
And an actual easy to identify factor which sustains racial differences - poor educational policies - is politically off limits to think about. "Because that would cost money."
The resulting mess is in alternating turns absurd and sad.
This was in the 1980s. (Before cellphones.) The guy who was supposed to pick him up didn't show up because his motorcycle broke down. And couldn't get it fixed because it was Chinese New Year.
My brother arrived, not speaking any Chinese, surrounded by people who didn't speak any English. Who were having the biggest party that my brother had ever seen, and kept giving him food and inviting him places! He had absolutely NO idea what was going on!
After a few days of this, the person who was supposed to meet him finally managed to arrive, and my brother was hooked. Spent most of the next two decades in Taiwan.
reply