I know why the mathematicians weep
derekthompson.orgI am also sad, but my sadness is, in a way, priced in.
I just had a chance to flinch first.
The people struggling right now - they were told. 30 years ago. They really should have come to terms with it by now.
But these are also the people who refused to believe it, for whatever reason. Their heads were in the sand. So I do feel bad for them... but... this is somewhat self inflicted.
It's ok - I understand it seemed impossible. But many people understood it just fine - and they're doing fine.
Your inability to give space for those who love math to mourn its inevitable evolution is unnecessarily cruel.
But we've gone back and forth about this a few times now about the same disagreement over framing.
You have a nice day.
It's easy to have that thought, and eventually you'll be correct. One thing about technology is that it only improves. It's always building on top of what already exists. The only uncertainty is the exact timing, not the ultimate outcome.
Also, it's impossible to really know how you're going to react to something until it happens to you. When ChatGPT came out, I already thought "in the long run they're going to automate almost everything humans love to do". Despite having 4 years to brood about it, I did not really know what it would feel like until it happened to me directly.
That's how the singularity works. It's going to keep getting quicker, incomprehensibly so.
Even before LLMs it was feeling like the design of society wasn't really for people any longer, instead it's more and more designed for corporations and their convenience. In the US at least. Same with how the internet has stopped most people with socializing the same way they did in the past.
Now we have a new layer that will be optimized for these digital entities. We see it in writing all over the internet. There will be tons of other places they show up and doing things that don't make sense to us.
But it can continue to inconceivable levels before it gets to that point.
Certain of predictions? I think that's an unreasonable ask.
And I'm comfortable with my prediction. Rather than argue about it - let's wait a year and see what happens, and then we can discuss it.
Oh, good point! I misread your comment. Sorry for that!
> Rather than argue about it - let's wait a year and see what happens, and then we can discuss it.
Deal!
> Today, the mathematicians are weeping, because the machine smashed their tower first. But tomorrow, it may be the physicists, the economists, the illustrators, or the writers who weep over their fallen blocks.
Isn't it extraordinary that these models still can't write well? I mean, they write passably well, but they all have a highly characteristic style that you can spot from a mile away. These language models can crack the toughest open problems in math, but they haven't yet cracked human-like writing, especially long-form writing, without exposing themselves. This is very very odd, IMO.
What is true, is that chat models (and by extension agents) have a very characteristic style after their LLM went through post-training. But the pre-trained language model doesn't have that style.
So if you want to write well with an LLM, you could and it is a well understood problem. It is more a matter of that not being of much interest, as it competes with having the model be good for chat interfaces.
How to have a well-aligned, RL'ed, safe or thinking model, which also writes well in long-form, is a bit an open problem. Though presumably agents could do it when they have access to a pretrained model.
Edit: why the style is like marketing corp-speak has to do with personalization not working well yet. So the default vanilla style is very corp. Most chatbots allow you to change the system prompt though and that can get rid of a lot of these issues in a lot of chatbots. But it is hard to maintain such a system prompt as the bots change so quickly.
I read this passage and thought, "has Derek even talked to software developers"?
That feels to me like the first discipline to go through this, where their daily activities are massively changed by GenAI.
That is why it's so disturbing. After several paragraphs, you realize there's nothing human there even when the actual information is present. People that do create AI piece are OK with it because they already have the context, but for the reader, it's always the uncanny valley.
Their math is as just as good as their rambling writing and their Ghiblified poster art. Most people don't seem to realize that.
"Some subset of this collection is valid, therefore they all are" is not a valid inference. It's more like what we call "proof by intimidation".
I think it is. Why is it that the only people in any field I'm aware of that think the AI output in their field is high quality are either the ones that are carefully crafting the output themselves or were producing/consuming slop themselves before all this started? (Outsiders pretty uniformly like the output, which is why I drew the parallel to Gell-Mann amnesia.)
It's not at all extraordinary if you understand how they're trained. For math and programming it's possible to generate an infinite amount of training data with correct answers, as compilers and theorem provers can guarantee correctness. For writing, the quality is not objective, it's determined by human preference, so you need humans in the loop, which slows things down massively and limits the amount of data that can be generated.
Either these mathematicians thought it would never come for their jobs, because all of these other jobs were simpler or they've been living under a rock.
Either way, I don't feel for them any more than I do for all of humanity.
Finding that new knowledge may be exponentially more difficult, but it will be much more valuable.
Why?
How do you know this?
- random,
- from the environment,
- from other AIs.
That increases the space considerably.
In particular, note that your argument also applies to humans: A human, being finite, can only produce finitely many things. But you add other humans, the environment, and pure chance to the mix, and it'll be pretty infinite for all practical purposes.
Maybe that is the limits of my imagination, but it seems unwise to take humans completely out of the loop. Having mathematicians around that can understand what it is doing so that we can utilize that knowledge in a way that we can have some confidence in seems important.
It sounds increasingly dumb to bring this up given how advanced models have become, but it remains true - they are still fundamentally token prediction algorithms. And that imposes hard constraints on their potential. These constraints are relatively academic in our current world because there's just so absurdly much information and no human can hold more than a negligible amount in his mind at once. So they'll certainly be able to connect many 'obvious' dots that we don't see simply for a lack of carrying capacity, but they're not going to discover concepts that do not clearly follow from the comprehensive knowledge of an era.
Humans have cognitive limits. Humans hold a specific slice of all knowledge at a time and necessarily limits what correlations can be leveraged. This is similar to "context windows" in modern AI. Human minds are wildly variable in efficiency. AI is has constant inefficiency, largely controlled by the speed of light, physical RAM, and processing power, with superior (no cost) "thought" context switching. The AI also has superior accuracy without the need for "concentration" because the AI is processing on a Turing machine with the limited operations that it's superior at in terms of time and energy - eg find an element in a list of 1 million elements.
Neurons and synapses change, especially synapses (the connections between neurons). Some neurons have a few connections to other neurons. Others have hundreds of thousands of connections to other neurons. These grow and change. "Estimates are between 100 trillion and 1,000 trillion neural connections" Is it finite? Depends on your definition of finite. If anything countable is finite, then it is finite but the number may be practically uncountable. Is it fixed? No.
1. LLMs aren't run deterministically - they use a bit of noise to improve the output. So even with the exact same input you're going to get a whole distribution of outputs.
2. Feedback from the environment. LLMs (and humans!) work best when they can test ideas. That's why coding agents work so well. If you ask an agent 1000 times to just think of a compression algorithm, it'll probably spit out very samey answers just like a human would. If you give it a test corpus and let it try out different things it will absolutely be able to come up with novel ideas.
Or maybe it can; the early image generators famously did output horrific images sometimes because of their limitations, but these outputs qualify nonetheless creations.
Now the interesting thing is that it is humans who evaluate them as horrific, worthless or interesting.
Certainly AIs could pretend they have that sort of opinions and judgement, but hopefully humans will remain more interested in the opinions of other humans.
Online, how would they even know? Offline -- either we give up on the internet, or the offline discussion will be heavily influenced by what AI said online.
https://birdsnfrogs.github.io/2026/09/12/Felix_qui_potuit_re...
At the same time, I think it's pretty amazing how this is the fundamental question of AI's trajectory, yet people on both sides seem so sure of their viewpoint with little actual evidence or data to back it up.
That is, it's possible that LLM-based approaches will be forever limited to this "convex hull" of current training data, and thus humans will always have a role to play (and will retain primacy when it comes to overall intelligence), because it appears that humans are not limited to this convex hull based on previous discoveries (e.g. I think basically every scientist would argue Einstein's general relativity was outside the convex hull that existed at that time).
Yet, if LLM-based approaches are eventually able to surpass humans on this "novel discovery" axis, then we as a species are basically done for when it comes to intellectual primacy - and then it becomes a very strange and scary world. And many AI researchers at the top companies clearly believe current approaches will scale and eventually (pretty soon I'll add) lead us to RSI.
I have no idea which side is right, and I'll admit I really hope LLMs eventually hit a wall. But again, I just am surprised that many seem to be putting forth their beliefs, but I would love to know if there is any good evidence one way or the other (or better yet, a more concrete and verifiable definition of the "convex hull" so that researchers could provide some falsifiable examples).
This seems like a good summary of the article, and I think the answer here is obvious.
So what?
News flash folks: we do not have the knowledge to survive on our own. If a solar flare deletes the electric grid, the first thing that happens is 8 billion people die in the next year.
We survive on abstractions others have built, and we have a global dependency web.
AI abstracting knowledge is neither new or dangerous; if it is dangerous, this certainly isnt the argument to prove it.
I think they might find comfort in knowing it's not just mathematicians - this is coming for all of us. Computers putting us out of work has been coming for a long long time.... and now it's here.
Still disgusted?
Ever heard of sour grapes?
This is the lie you tell yourself to handle the situation.
it is a little ironic that after the "creative destruction" of industry after industry all of a sudden now it's unfair and we weep. I say this as a software dev with his neck on the line like everyon else.
It's not clear from your comment: Are you talking about math professors?
If so, what money? They make a living, but it's not that great. I just checked the salary of two associate professors at my university: $80-85K.
And keep in mind they often start their careers quite late. Often multiple postdocs after PhD. Starting a tenure track position in your mid 30's is not common.
> I'm more shocked that there are so many brilliant people who are so results-oriented and don't even have a concept of value outside of that. I'm kind of disgusted by the tears actually
The whole point of the article is that the person crying is because he loves the process and is worried the human connection will be gone.
In my undergrad math department, only 2-3 faculty members out of a dozen bothered with grants.
The bean counters laying off software engineers ought to take notes from the frontier labs' own hiring practices. They're still hiring lots of humans for business and engineering roles even when they would be better positioned than anyone to automate away those jobs with their world-class models (https://openai.com/careers/search/).
Generalization fallacy, survivorship bias, etc.
Neither sits right with me. Can't we just all collectively acknowledge that we have to grapple with this new piece of technology that either is already or at least seems to be going to be hurtful to a lot of individuals?
Instead of looking at the in comparison limited group of people currently pushing this technology and asking ourselves whether we should be on board with their doing, we look to one another and are happy that someone else is now suffering just the same as we are. What a sad state of affairs.
AI is going to suck hard for lots of professions, software engineers and mathematicians included. I think it's much more productive for folks in these professions to show solidarity with each other given the way AI will disrupt their livelihoods, rather than showing disdain based on who you think may have it slightly better.
I know this intimately because I identify, roughly, as a programmer first and security researcher second. Guess which two fields ai companies decided to specialize in first? Yes, the FML ones.
The intellectually rewarding part of my job/skills - the only reason I enjoyed them - are, not completely gone yet, but threatened. And shrinking. This is a very scary observation.
I know from ‘isomorphic situations’ lessons in history that fighting the tide is pointless. I have to decide intelligently how to respond. Easier said than done when your livelihood and identity are at stake.
For now I still have a job but I and everyone around me is worried. In a year, At best our jobs will be very different.
So yes I empathize with these mathematicians enormously.
Math journals commonly reject papers as “out of scope” or “not of sufficient interest” without an editor or referee actually reading them. So it’s not as if the “math community” was checking these papers for accuracy in the first place. Now AI speeds up the creation of papers that go unread, and we’re supposed to pity mathematicians? If journals and experts can’t be bothered to read the papers, why should authors bother to write them without AI? And suddenly mathematicians are worried about AI papers being “too hard to read,” even though that wasn’t a concern pre-AI, when no one was reading them anyway. Too bad. The incentive structure now points toward AI, and mathematicians share the blame for that.
I don't personally grieve for the likes of Strogatz or Tao, it seems likely both of them will be fine however this pans out. But many more mathematicians have been toiling away in precarity and obscurity and without tenure for years and years. And of course many mathematicians went into private sector roles as software engineers, actuaries, analysts, ML engineers and similar, and have been displaced there, too.
And highly visible blog posts written by famous mathematicians with large audiences and closely aligned with the news and hype cycles getting frontpaged on HN doesn't mean that mathematicians are disproportionately whiney, nor that software engineers aren't also writing about their plight or ending up on the front page as well.
It's not like the options are write software or starve. Billions of people find jobs and can feed their families without writing software.
I also don't often meet junior software engineers who actually have families to feed. Most juniors work in the industry for a few years before deciding to get married and have kids.
At that point they won't be software engineers
A very small fraction of people who get math degrees actually continue to do math in their careers. They usually have to get other jobs in order to survive (actuary, teacher, finance, etc.). Academic math funding was already drying up for last few years, and now AI will probably cause more cuts. Not to mention that academic math positions have never paid particularly well.
I feel bad for anyone who can't find a job, but the software industry as a whole has been the cushiest industry for decades, and the current turbulence seems more temporary (and driven by corporate greed) than a systematic destruction of the trade.
Unfortunately this often means moving away from family. What are you going to say to the kid whose parents toiled to get him through college on the expectation that he would be able to take care of them? "Sorry Mom, I have to move to North Dakota to work on the oilfields"?
This is said by a lot of CEOs trying to make layoffs look good, but I haven’t seen anything that suggests that it’s true.
> Yet I see far more of these blogs about the plight of mathematicians, not enough about the plight of software engineers.
Really?!? We’ve been living through 3 years of blog posts, LinkedIn hot takes, and podcasts talking about the plight of software engineers in the era of AI. There isn’t a week that goes by without an HN post about it.
I can feel empathy for the practitioners in a field that may have been profoundly changed, the same way I can feel empathy toward software engineers who are struggling with the changes in their own field.
In the end, we’re all trying to figure out where this leads. And how this will affect our lives and our work.
It's not fun to think of a future where knowledge workers become irrelevant in society and we're back to just physical work and managerial elites.
Just like in past history.
Yes, it sucks, for software engineers, those in the past, and those in the future. As long as you and others fall for the straw man perhaps you and others will get more and more tricked and hurt, misplacing your aggression and hatred on each other rather than those receiving most of the extracted value.
Yes in some cases, but the reduced costs also allow lower prices which benefits all consumers of whatever product is being produced. At least for those products where there is a competitive market.
The economist John Keynes predicted almost 100 years ago that technological advances would lead to a 15 hour workweek. If AI now makes you 5x as productive, then why aren't you working 1 day a week for the same pay? Why didn't Keynes' vision come true.
governments implemented (and still implement) keynesian economics, and it has turned out to be a very bad idea and he was very very wrong.
I don't believe it is that black and white. Yes, automation eliminates some jobs, but overall moves everyone (on average) further up the productivity line, which improves lives, again, on average.
That is why we live like kings now compared to people of the past.
But circling back to the point, if AI does the same thing, eliminates expensive manual coding and other similar work, that is essentially the same thing we've been doing to other people for decades.
Are phones an "automation of existing jobs"? Maybe couriers (??), but most of what people use them for goes far beyond that.
What about the software used to drive trains or autopilot planes? We still have conductors and pilots, but now the outcomes are safer on average and more efficient.
Or something like Google Maps -- all that replaced was the paper booklet people used to keep in their cars, it's not like we used to employ 'navigators' to go out on a road trip.
Or something like Intuit QuickBooks: there's still an accountant in the loop, and if anything more accountants today than there were a few decades ago.
Certainly some software is essentially automatory in nature. But to characterize 'nearly all' as being such just betrays a lack of awareness for the rest of the field.
> That one is somehow deserving of sympathy and that of mathematicians is not
He didn't say that mathematicians aren't deserving, he was griping that people said this about SWEs. Do you think mathematicians are deserving but SWEs aren't?
The words and excels of the world, automating documentation and bookkeepery. It all makes some kind of existing thing we already do faster and more convenient, and often more efficient. Most software simply does nothing new. The thing you didnt manage to list which is the actually good example is games.
You seem to be unaware of the existence switchboard operators (people redirecting calls to the correct person) were hundreds of thousands in the US alone at some point.
Say you have a friend who likes making music (not professionally - just a hobby).
If an invention came out tomorrow that took the joy completely out of making music, would you respond with "It's hard for me to feel sorry about you."
Just because things suck for you doesn't mean it can't suck for others (in different ways).
In many ways it will likely be the opposite. Tech firms still need engineers to talk to customers and run experiments. What will the mathematicians be needed for?
What does your being software engineer has anything to do with feeling sorry for Mathematicians' job ?
It's a classic "I have it worse than them...everyone should feel sorry for ME and not them".
This is what it seems like so far for software. The speed limit has changed. It’s interesting to explore what that means. Extending the analogy, long ago you could be a race car driver part time; that would be pretty laughable for an F1 driver. It’s a very specialized skill and the optimal athlete is quite different for driving a car than for running, for example. That feels like what I’m seeing in software.
The most effective folks are much faster and better with these tools than average. It’s a different skill set.
I think that formal verification makes Math much closer to code than many other fields. But I’m eager to see the first mathematicians that are dramatically more proficient with the new tools than average.
Nothing has changed in math. Math is not solved nor has the human been kicked out of the loop. Same with programming.
*I said nothing really changed because LLMs becoming superhuman at deduction was priced in with o3 and neither the peer review, training nor publishing methods will get a revamp beyond what could already be done with SQLite and a website.
I really need to just stay off social media for like a month and keep talking IRL. Longer, if I know what's good for me.
> I think I have been underrating the pace of model improvements. In March 2025 I made a bet with Tamay Besiroglu, cofounder of RL environment company Mechanize, that AI tools would not be able to autonomously produce papers I judge to be at a level comparable to that of the best few papers published in 2025, at comparable cost to human experts, by 2030. I gave him 3:1 odds at the time; I now expect to lose this bet.
https://www.daniellitt.com/blog/2026/2/20/mathematics-in-the...
“I’ve slowly updated my timelines over the past year…”
and
“I think I have been underrating the pace of model improvements. … I was not correctly calibrated as to the capabilities of existing models, let alone near-future models.”
AI will not replace the mathematical community (peer review, abstraction creation) until AI can train the next AI purely on data generated by itself and with a tool in a turing complete environment. The environment has to be air-gapped from human data but the AI can use tools and its own outputs
I think that "solving math" is isomorphic to solving the "thinking part" of the scientific process which would be genuine RSI
Sure, there are people who overstate things. But that isn't an excuse to keep your head in the sand.
A lot of us knew what was coming. Profs I speak to these days say it's the same old except they need to pick a different kind of problem for grad students
Anticipating that something will happen is not the same as evidence that it's happened.
How will it pan out, no one really knows. The leading frontier labs are continuing to hire aggressively. But who knows for how long or for long or near-term goals.
There were mistakes that could easily be seen with a diagram rendering three big results wrong, exposition is terrible, no coherent underlying theory generated from solving so many problems, as Thomas Bloom pointed out even duplicate proofs of the same statement in a single paper.
It clicked for me, "this is slop but slop can be superhuman and useful!". Similar to programming where it can generate as many criticisms of your codebase but you have to weed them out.
I was taking to my friend in OA who was really surprised that the proof of L(F_2) = L(F_3) could be done in the 1980s. We just didn't have a superhuman deducer back then.
Special relativity which Einstein got a Nobel for is now taught in high-school.
"Of course if you move a magnet in a piece of copper you will generate a current, it's extremely obvious it should be so"
But many people perceive this very differently, see link below. And perception is reality after all.
And this is only the beginning.
You are probably right that some companies are factoring AI into headcount but I'd imagine that the interest rate and energy crisis are bigger concerns
It was long predicted that the sign that AI is here in macroeconomics is rising interest rates, because investors can get MUCH higher rates of return doing AI investments than borrowing governments, why would I lend money to Greece when I can lend it to Anthropic?
LLMs haven’t done much in the way of informing me or others I know regarding “who and how many”.
I’ve worked in tech for 20+ years and I’ve never seen it like this. Sure there are job openings, but sooo many applicants it seems.
(This is Sweden, YMMV.)
But from my perspective proofs have never been the most interesting part of maths. What really interests me is the intuition that leads to new applications, and to new questions. Newton, for example, gets more credit for calculus than Leibniz because he's the one who demonstrated how darned useful it is. Erdos is arguably more famous for asking interesting questions than for answering them.
I still don't see LLMs seriously contributing to that facet of mathematics any time soon. Every so often I ask one to suggest a good binomial confidence interval for a particular application, and it still tends to just suggest one of the subpar options that everyone learns in a first year stats course.
The recent barf-out done by LLM companies is useful if what are you are interested in is an unreadable mess and someone giving you the final answer. You might be able to extract some new "aha!" moments out of it, but we will have to wait and see what happens once people go through it. I would hypothesize the LLM output will be good at hinting at some non-obvious relationships and patterns, but I doubt it will yield anything of substance in its own right.
Historically, proving difficult theorems required mathematicians to sometimes invent new fields, new ways of looking at problems, the discovery of which itself yielded incalculable contributions to both mathematics and other fields. It is this that is the greater contribution of and to mathematics.
Mathematicians working at universities are paid to teach, to engage in the politics and business of government contracting, and only in very rare circumstances to play prestige games on famous problems.
It is unlikely that Congress will decide to slash budget for NSF's MPS directorate on account of AI getting good at proving theorems. That's just not how Congress works.
It is also unlikely that LLMs will be able to do, in the near future, do the sort of applied mathematics that happens in the context of occasional NIH or DoD funding that flows into Mathematics departments, for a variety of complex sociological reasons I'm happy to expound upon.
And it's unlikely Engineering degree programs will stop requiring calculus sequences, since being able to do the computation was has not been the actual point of teaching calc since he late 20th century (if ever).
So, the good news is that I doubt any of this will have much of an effect on the job market for Mathematicians who want to prove theorems for a living. The bad news, of course, is that the job market for Mathematicians who want to prove theorems for a living was already pretty horrendous.
Yes, outsiders (investors, journalists) are given a false impression while insiders (software engineers) are supposed to give up. Double win for the agitprop machine.
Is it like Chess and Go? There are still plenty of people playing for fun, but the machines are better at it than we are.
Jokes aside, its like working for a company, individually you can lift mountains and someone else will take credit. When you use LLMs for a single bit, all credit goes to LLMs lol
https://en.wikipedia.org/wiki/IBM_and_World_War_II
the above, a "sorting" machine is the primary usage of hyperscalers currently, they're just trying to find new uses to boot for extra money/legitimation.
the browser industry the phone industry the CAs JavaScript
it's all paid for by those wanting to sort humanity into whatever bucket the can fear monger the peasantry into for aweek or however long it takes the classification system to sort out.
gentrification works, but we chose forcible displacement now we have to live with the forever consequences.
would you give guns and bombs to the displaced native americans who taste blood over their beads and trinkets?
we've been inside the event horizon of the singularity for years, the tax bill will only get worse unless we recognize it for the tick that it is.
Now the thread has been cleaned up and a pro-defeatism comment is the top comment and the submission is on the front page again (I'm not alleging moderator intervention, user mechanisms are sufficient).
Wow that's very well worded.
Worth reading: Robert Brandom, “Making It Explicit: Reasoning, Representing, and Discursive Commitment”
Both depend on Logic, which is where "deontic scorekeeping" comes in. You can only establish something (score) is true and believable by 'keeping score' and having some level of understanding about the language or math you are reading.
The article is mostly referring to the connection that comes from shared understanding between humans, particularly mathematicians, as they build new shared understanding, and realize there are limits to what they can know and 'leave behind' logic for others to pick up and carry on.
That connection has been severed, or strained a bit, but I'm sure new ones will be created. It just might be of the same flavor that mathematicians prefer, therefore they weep.
And if I understand "deontic scorekeeping" correctly, these forums are a better representation of the concept. You will lend some amount of belief to what is said/read based on what you know.
What an LLM can't do (at least with the current architecture) is sustain a commitment that it can be held accountable for later. It doesn't have durable "perspective". It can't update its priors. Maybe a new architecture will come along that gives LLMs memory plasticity, at which point they will shift into being individuated quasi-persons, but that's not what we've got now.
I've written some about this here, if you're interested: https://elliotmilco.substack.com/p/deontic-scorekeeping-comm...
I think you are also suggesting Mathematics and Language are the same thing.
I'm not suggesting mathematics and language are the same thing, but I am saying (which shouldn't be controversial) that mathematics as a discipline exists within the medium of language, and that mathematical reasoning, discovery, and proof happen via people proposing, justifying, testing, and objecting to formulations in language. This isn't a metaphysical claim about whether math is real or captures inherent structures of the universe etc.
You ended your first post with: "An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time."
And so if there is an agent, a thinking loop, that can recursively adjust over time...how does your statement hold up? I'm particularly talking about 'staking durable commitments or changing its dispositions over time'. Recursive self improvement is all about these 2 things, especially when it's the 'outer thinking loop' doing the improvement.
But you're right, Mathematics exists as a medium within Language. But then Language is just a transfer of ideas from one mind to another.
This principle is the heart of what the article is talking about: a transfer of logical mathematical ideas passed from a human to another. And the sense of 'shared understanding' that comes with it. It has nothing to do with keeping score.
It's about getting a 'sense' for why someone thought or rationalized an idea in order to connect some other idea. You get an 'ah-ha' moment, and some of those moments are gone, or a different path is taken to reach the moments now. That is what the 'weeping' is about (not that I agree with the article. I think anyone can spend however they want. Math doesn't have to change, it's that people want it to, apparently, because it's 'easier' to do something different, or for some other reason I don't know about).
I suppose we would see some interaction with AI in that sense, but it'll take place over the course of learn cycles and when AI starts ingesting its own ideas it typically goes wrong somewhere down the line.
Deontology - `normative ethical theory that the morality of an action should be based on whether that action itself is right or wrong`
Super roughly:
* Virtue ethics: character and motivations
* Deontological ethics: rules and obligations
* Consequentialist ethics: utility and outcomes
Then the argument against LLMs being party to that is that we don't really assign roles and responsibilities to LLMs, they don't really have "duties" or "obligations" in the way a member of society will have. We don't negotiate with them over who should do what, they don't contribute to the rules that are set for them.
I'm literally just guessing based on the context here though, since the few times I've tried to watch podcasts with Brandom I generally bounce after 15-20 minutes feeling I haven't really understood anything he is saying.
The participant is not an LLM, the participant is a company. The company learns. It adapts. It has a trillion dollars of skin in the game.
Likewise OpenAI doesn't have "skin" of any shape or value in e.g. the mathematics community or even in mathematics research per se. In my cynical view, they are exploiting the exposure of mathematical research to automation in an effort to create a case study of their product obsoleting a highly visible and well-leveraged and relatively small profession. Regardless, I doubt even they would say they have durable skin in mathematics in any sense, certainly not in either of the senses I mentioned above. But their output inhabits the same space as mathematicians, who have to contend with the consequences
They hire math and physics grads and good command of math is very useful in computer science, machine learning and AI.
And how do you see these qualities changing over, say, the next decade?
And potentially steals [1].
[1] https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_priority...
I'm interested in the reversal that Sellars proposes - it seems clear to me that "representation" is not prior to the space of reasons, despite how unintuitive it is.
> An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time.
I see no reason why this can't be the case in the future, even though I think inferentialist-based theories are generally strong. Can't you force LLM's (or whatever AI architecture will exist in a decade) to stake durable commitments itself, then? They're at least autonomous enough to run computers. I agree that that is a core part to true meaning & autonomy, but nobody just lets them yet. I don't think inferentialism is a consciousness-based argument, either, so that escape hatch isn't available. I also don't think this is a necessary condition to engage in the space of reasons (but maybe then I'm misunderstanding it, or retreating to a more Sellarsian understanding). Could you explain on this more?
Though again, I only have the book sitting on my desk, and I haven't cracked it open...
And yeah, consciousness based arguments are bunk IMO. It's clear that LLMs are in a basic sense "aware", and even to some extent "sapient" (in Brandom's usage), i.e. they can justify claims and object to things at least locally. I follow Rorty on the question of consciousness, which is to say that consciousness has no deeper meaning in practice beyond "are you awake", and in philosophical discussion it's largely a proxy for "are you one of us" in some amorphous sense.
Consciousness, the interesting part anyway, is qualia, or the "it is like something to be me" as Nagel put it.
Intention, as Dennett used it in his book "The Intentional Stance" is the "are you one of us" angle, and is distinct from consciousness because as the book title suggests, it is a stance taken with respect to some other system (person, cat, dog, turtle, LLM) rather than a description of subjective experience.
So I think it is your (and Rorty's) positions that are bunk. It is very, very far from clear that LLM's are in any basic sense "aware" let alone "sapient". They are extraordinarily able to manipulate language in a way that leads to text generation that has traditionally been assumed to indicate consciousness and/or intelligence. But we have no way of knowing if it is like anything to be an LLM, even though most of us are busy taking the Intentional Stance towards them.
And as I think Dennett himself would say, qualia are just Sellars's Given rearing its ugly head again.
[1] https://sites.pitt.edu/~rbrandom/Courses/2022%20Phil%20of%20...
Edit: One of my ways of benchmarking new AI models is actually to type a version of this prompt in and see how sophisticated the response is: "Talk to me about the concept of deontic scorekeeping in the context of llms". You'll get good mileage out of that.
I read this sentence 5 times and I still cannot parse it.
I mean this with zero snark: if you’re interested, you’d be best off asking an LLM or doing a search. But the overall idea is that Brandom described how concepts and meaning are created socially by a kind of game in which people keep track of the claims that other people commit to.
If you accept Brandom’s framework, then it can be argued that LLMs can’t meaningfully participate in this process. This could be considered an explanation of why people have a negative reaction to LLM involvement in activities that are meaningful to humans - even though most people won’t be familiar with the theory, they would instinctively recognize that something important is lacking, such as commitment over time.
Seeing AI being lauded for solving problems like this makes me feel... uncomfortable.
We are feeding all works of the giants that we have all stood on into a huge vat, in the hope to generate something meaningful.
If we can't understand the output, internalize it, or build from the time and experience that a human would have gained from solving the same problem, it feels like a dead end.
When all the problems have been "solved" by AI, what's next? If humans no longer struggle to solve the problems that the previous generation have left behind, how do we progress?
The same goes for math.
Let's find some things that bring value other than just the price of a stock.
At worst its time is past, but even that is unlikely.
So I wonder if mathematics will go through that too? A period of chaos and doubt, but finding new and more effective ways of working.
Its easy to demonstrate this: The math hysteria passes Lean but no human can make sense of it at present. This is akin to bots telling humans where to dig. Thats fine but lets be clear about whats going on.
If it was about understanding interesting proofs that some AI came up with, and extracting their essence, I don't think the situation would be so bad.
I see the much larger problem in the now existing career prospects of non-tenured researchers in mathematics. Everybody know how precarious the career prospects in academia and how precarious often the non-tenured positions in academia already are. With the new AI breakthroughs in math, these have in my opinion even worsened a lot.
UPDATE: For a related point, see also pixl97's post: https://news.ycombinator.com/item?id=50104145
I think mathematicians aren't bemoaning their lost sunk costs. They're now aware that they may never again have the joy of being the first to pull up the edge of the tarp of ignorance and see the raw truth beneath _for the first time_. Their voyages of discovery, which give life meaning, may never leave dry dock again.
> You are a helpless onlooker. Before you a channel through which humans have accessed the ineffable and sacred for thousands of years is being sealed for eternity.
https://kirwinhampshire.substack.com/p/the-dark-night-of-mat...
If you read the linked article, the author goes into more detail about how doing mathematics also has been an act of communication with past and present human mathematicians. This is very much true when math and science are done by millions of human researchers, but not so when it is done by massive swarms of agents at a couple labs.
I am sure science and math will benefit in the sense of solving more problems than ever before. But you are not really understanding the agony of the humans caught in the change.
I have to wonder if this is the equivalent of teleporting back in time and finding Marco Polo or any of those early explorers who traveled for the love of the game (and fame and money), and opening up Google Maps on your phone and showing them the zoomed out world. And showing them any island or village they could name.
Not "never again", it would be extremely difficult but science and math are also built on random, trial and error, and intuitions that could not arise from AI.
Computer can iterate without frustration on a completely random garbage. Once in a while it stumbles upon a gem. Even if it's one in 10000, that's still hundreds of good ideas in a day (provided enough GPUs).
You can generate completely random ideas (a couple proofs I saw where you define some weird set, pick an element and derive a contradiction when the set is empty or whatever).
Some of these will reach a conclusion: proof or a counterexample. And that's how speed can outperform creativity, if the speed bump is fast enough. It's the same as with those monkeys and Shakespeare.
Until we can demonstrate AI consciousness, “seeing the raw truth” will be a human activity. Even now, verifying and understanding these AI proofs is a massive endeavor. Coming up with an intuitive (to humans) distillation of the finding remains an open challenging demanding expertise.
We literally had a case where OpenAI may have stolen the dialectic from a mathematician using its product and then given it to a proprietary model burning tens of millions of dollars of tokens [1].
[1] https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_priority...
If the result is black and white, then yeah, its probably not super fruitful but if the end result is subjective, then there really no is objective "solve".
I don't think this matters a bit. We individually can't demonstrate "consciousness" in other humans, it's not even a word with a definition. What we can do is miss the feeling of discovering something on one's own rather than having it explained to you over and over again until you understand it.
I mean, dead mathematicians are definitely not "conscious," but their books keep you from discovering things that you may have discovered yourself if left to your own devices.
The joy that was mentioned is the joy of knowing a bit of gossip that no one else knows, of being the first explainer. Interpretation is just being able to pass gossip on; a decidedly lesser joy. And there's no reason that AI shouldn't be able to explain findings in an intuitive way - LLMs at least purely operate based on human intuition as written in human communications and then captured in matrices.
If these proofs are good (doubtful, imo) and can be reduced (intuitively distilled, sorry), there's no reason that an LLM shouldn't be able to do that.
In many ways, the teleporting machine that teleports to the solution is against the point of the questions. No one wants the solution to be "Run $10,000,000 worth of tokens to cover all possible Lean formulations of the problem".
I think for Mathematicians, I'd say that the human level formulation is still possible and with a new perspective. We as a society (and outside of Mathematics) need to also understand that this AI teleport to the answer thing has less value than we all expected. The value is in the journey.
The open question is if people will still want to take the journey if the teleport machine exists. Or if the existence of the teleport machine halts exploration.
You're overlooking that the teleport machine costs lots of money to use. The AI providers are not going to waste money by teleporting randomly around.
Someone is going to have to find a new destination that is worth spending the money to teleport to (or walk by themselves) and that someone is mathematicians.
Meanwhile, the AI labs are doing that for just marketing.
So how can they possibly solve problems that humans have worked for decades or centuries on without being able to solve? How did they find something new? Is it that they were able to deduce the solution by examining all current data sets? Or simply brute force it until the produced solution passed all necessary tests?
Yes, both of these + an infinite token/compute budget and SOTA+ models. It's not like some hobbyist out in the woods with a GPU running Qwen 27B can do this.
Lots of mathematics entails recognizing that a tool/techniques that works in one area can be adapted to a completely different problem. For a human to do this requires having a robust mental model of the technique and the problem itself, and probably having just the "right" mental model to see how they fit together. And we are talking about extremely hard/sophisticated techniques and extremely hard/sophisticated problems, so the ability to deeply appreciate either can require years of study. And on top of that, even if the mathematician has sufficient understanding to develop the intuition that the technique might work, they will need profound fortitude, skill, and patience to see the solution through.
Unlike human mathematicians, who will develop a deep understanding of only a relative handful of techniques and problems in their lifetime, LLMs have ALL written techniques, problems, solutions, etc stored in their internal representations. So when they ably match a problem to a technique, we can imagine the operation as a supercharged autocomplete, or even an extremely proficient search engine. Their stroke of genius is really a consequence of capacity, moreso than cleverness.
"Glorified autocomplete" is meant to describe the _underlying mechanism_, but unfortunately it also sounds like one of the things LLMs are actually used for, so it gives the mistaken impression that the claim is all they are good for is a component inside an editor application.
I never appreciated hard math much because my brain just can't get there. I thought so much of math was much ado about nothing, but now I see the art in math...not in the inscrutable formulas, but in the creation of AI.
Well done, mathematicians. You can now rest from your labors.
well i think the grieving is because they still have to make rent somehow.
These guys know because they looked at it. Denying it is useless. But maybe some of these guys can figure something new using it and asking it to look for it.
Considering I currently live in Ithaca and have a friend doing a Math PhD, so I've heard from him some of the discussions in the department. I think the biggest shock for people is that individuals spent significant portions of their time on some of the problems that OpenAI supposedly solved (I say supposedly because all of the proofs have yet to be independently vetted). I understand that it's a shock, but I'm confident that in time it will be no more of a blip than it was for NLP.
[1]: https://xxcancel.com/mayhewsw/status/1636476184098643968
[2]: https://xxcancel.com/andriy_mulyar/status/163613925780582809...
I note that that these conjectures being solved by AI are named after people, minds knowledgeable and and brilliant enough to come up with the question in the first place and recognize its value. What we have in these AI math solving systems is the equivalent of the chess and go software that can beat humans: something that can adhere to a set of rigid rules to reach a win state once given a starting point. But, as Pablo Picasso said, “What good are computers? They can only give you answers.” These AIs are a magnificent achievement and will change mathematics and coding but can they ask earthshaking questions? Could an AI identify conjectures that mathematicians recognize as a worthy question that will lead to new insights? Where is HAL9000's theorem or the M5 Multitronic's conjecture? Maybe they can someday but I don't see any immediate prospects for it.
Please spare us the "AI will surely keep getting better forever" (unlikely and don't call me Shirley), "moving the goalposts" (I myself haven't set any), etc. non-arguments.
Questions: (1) Do you still weep for what has transpired? (2) Do you continue to do what you used to do (with optional AI assistance)?
For me: (1) no, (2) yes and I think the same would be true for most people. So I can't help but believe that most of the anguish about AI is far more about a direct economic hit to people's livelihoods than it is about the impact to their respective fields. Which is more than reasonable. But then why not frame the conversation that way instead of beating around the bush?
heck, we have the resources to that now but we don't do it.
Where or what do solving these math problems lead to?
Is there some suggestion that they lead to deeper understanding about something else? Do they solve physics problems?
What’s the benefit? What’s the point?
It is mostly impossible to figure out what we find when we solve these problems. But occasionally they'll lead to real world efficiency in algorithms we use and it changes the world.
If you have AGI, when a situation presents itself that needs a particular thing solved - it'll then go the math thing specific to the situation.
How much do they pay you? Does OpenAI have bulk subscriptions to your Substack?
Isn’t that in itself the new frontier for human understanding?
I would have thought that mathematicians would have found alien forms of maths to be astonishingly compelling, must learn, must know, must understand.
Instead there seems to be a sort of shoulder shrugging lament “we don’t understand it”.
I would barely be able to sleep with excitement at the potential for where this could lead. Like finding an ancient tomb full of answers to our deepest questions along with the promise of more.
How deep does maths go? Is it “solved” or have we just been given tools to delve ever deeper?
Where is their curiosity - don’t the want to find the bottom?
Does relating to someone pay my bills? People will use the informational tools to improve their social status and resource accumulation, there is no doubt about this.
There is little or no time, or incentives for care as described by the blog.
I hate programming these days - back in the day, the cubicle-hell nature of "popular" frameworks was at least offset by being able to think through and solve interesting problems.
These days most of us are stuck in the "endless-review" loop with LLMs. You can't afford to not use it given the inflated expectations. As with all of modernity, productivity doesn't yield leisure - it just proportionally increases your workload.
Hardly surprising that the other side of the Curry-Howard correspondence is now getting hit with this. Math was one of those depts. where slow/deep-thinking was possible (CS was already insane 10 years back - now it's just absurd) - now even this fortress will be broken. Tragic.
The vanity, fame, self-improvement, it's very selfish as everyone does.
Now the fame coming from proving things like Riemann Hypothesis is taken away. Self-improvement? relying on AIs make you stupid. It will be lost. Vanity? Who else don't know how to use AIs? It cannot fill your void.
Now also the most important thing is the communication.
There will be less communications amongst mathematicians at least in person. On paper? maybe but mostly via AIs.
And then once it's solved by AI: do you want to churn through the proof to understand it? What if it takes you 5 years to churn through the proof, and it turns out someone made a mistake with AI (I have no idea what would be realistic amounts of time, but the idea is still worth considering).
The issue is very complex, and you will always be hungry.
If I were a mathematician working on the Riemann Hypothesis, I'd probably be rather pissed off right now but continuing my work, i.e. the same as I am as a software developer.
Social creatures that share a common space, with finite time that nobody wants to waste.
Every analogy works "only" at a certain abstraction level. No need to disregard oversimplification. This spawned the debates so all good.
It's the same reason you see 'no AI' repo labels: only by forcing some standard on the community can the 'old way of collaborating' continue to exist.
There will for sure be new ways of collaborating, and walled off gardens like you suggest, where communication stops and everyone just shares some final product.
But, in some cases the product must be understood (or demonstrated) before value can be derived, and so collaboration will still exist.
Desire, and the 'shape' of collaboration has changed greatly though. Why collaboration has changed, is a completely separate issue related to economy and how each of us spends our time.
> There will for sure be new ways of collaborating
I am not smart enough to know what the new ways will be but there should be one to adapt to changes.
Are there any mathematicians who are genuinely excited about this tremendous leap forward in mathematical knowledge? Where are those articles?
i think that's 85% of the response from software devs if not more. All of the devs I know who actually like to build have embraced AI and the new capabilities that come with it. The ones that guarded their little corner of code proclaiming only they, and their inner circle, have the power to comprehend it are the ones crying in the corner.
What are the relationships between many languages that become apparent, including code as a language that were not obvious before llms? Translation is fairly solved. The study of how and why languages, not at all.
This nails it, however IMHO its almost all bruised egos and feeling of wasted time. I don't know the ratio of A:B but I also don't know anyone that is in the latter category of feeling doom and mourning that their craft is about to regress.
https://en.wikipedia.org/wiki/Life,_Liberty_and_the_pursuit_...
To me it begs the question: in a theoretical world where we don't have human mathematicians anymore, what is the point of these proofs if there's no human who actually understands it?
If the point of a lot of this math is only to expand the boundaries of human knowledge, it all seems kind of pointless.
Proofs are somewhat like legos. A bigger toolbox of them can only help in solving other, possibly practical, problems.
As far as the utility of a human being involved -- In a non-theoretical world where we don't have feline mathematicians, what is the point of these proofs if there is no cat which actually understands it? The cats can benefit from, for example, advances to optimziation algorithms that reduce costs for cat food delivery, of course.
My intuition is that it'll be at 'smart undergrad' levels today, though as the cryptographers say, attacks always get better.
I guess I'm not astonished that people are so bad to each other in general, but more that in the case of genAI it's so prevalent in places that seemed more elevated previously.
Part of me wonders if this is actually a form of self-defense. The misery of others can provoke sympathy or the opposite: Angry pushback to protect oneself from dwelling on their misery.
For some reason I'm stuck in a real depressing, doomer algorithm / homepage on Twitter. But everyone on my homepage seems to have a take on the HNer comments along the lines of "see, this is why you don't make your job into your identity".
But that's such an antihuman response. The only reason you can put decades into a field to grind and build expertise is because you have an identity-level interest in a subject.
It's also not a good thing if the optimal way to live going forward is to have no passion for anything lest it changes so that we won't feel bad. I don't think that's possible unless you remove the human part of our brains or make life so horrible that we're happy just to survive another day.
Source? There has not been a statistically-significant increase in middle-aged male suicide rates between 2023 and 2024 confirmed data [1]. (2025 breakouts aren’t out yet, but top-line data show decreases.)
In 2025 in the UK men age 45–64 had a suicide rate of about 24 per 100,000, which is the highest it has been in the past half century, and is higher than other age categories. In the mid 2000s, the rate was like 17 per 100,000.
From your link, in the USA the overall rate for men of the same age is about 30 per 100,000, but varies dramatically between states.
At the risk of taking this off topic, the reason is Twitter. Don't go on Twitter and the doomer algorithm disappears from your life.
And I can’t imagine feeling the same about any group of people. It’s been odd, to me. I’m just trying to support myself and the people in my life, and now there’s a looming fear that I can’t do that anymore… business owners are drooling over that, and other sectors are laughing.
Hard times, I guess.
I think the average person lives a much harder life and has been doing so for a while. Should we really be surprised by the lack of empathy when folks in tech had little to show for the common man this whole time?
And perhaps the same thing is now happening to me. It feels different, but maybe the same thing is true.
If AI got significantly better at performing surgery overnight and doctors started crying about how they were going to be replaced, what do you think the public response would be? Non-doctors want the best possible care; they aren’t interested in protecting an industry. The same will be said for pilots and lawyers, too.
People dislike self-serving whingeing, end of story, but that’s doubly true when it comes out of a profession (theoretical mathematics) that is mostly practically useless.
when mathematicians complaining about losing the part of the job that they love, people think "if you love doing it, why is anybody paying you? you're describing a hobby."
They are showing as much care for mathematicians as they received themselves, which is not much.
None of them have pointed to any of the solutions and said "this is real innovation" - because it's all been mining existing ideas and applying them elsewhere.
Imagine you spend years working on a solution to one of these problems. Only to realize after you publish that more than half your answer already existed in another branch of mathematics and you spent a bunch of time re-inventing it.
Yes the first big solve by OAI was "stolen" and that is tragic, and pretty bad. But it looks like this new batch, the one people are crying about, has none of that stink.
Maybe the answer is to look into how Big Science works? There's still plenty of work to be done figuring out what to do with the outputs from these new math-observing instruments.
Are the experimental physicists usually the ones holding the shovels?
> If intelligence becomes less uniquely human, we may come to value more highly what remains uniquely human: our capacity to relate and to care for one another.
Lol. How does the author think relations between people work? (At least those outside the family). Relationships require something that fills them with meaning - a reason why you would want to have that relationship in the first place. Without meaning, no relationship.
I recommended reading this other post that was on HN earlier today: https://thetheorygirlie.substack.com/p/ai-was-great-until-it...
It describes in pretty grim words what happens to collaboration once you have nothing anymore to collaborate on.
> Mathematicians may still teach proofs, argue over what they mean, translate alien logic into human insight, and pass that understanding to students.
This would people to have time to understand and dissect those proofs - it doesn't work if there is a new breakthrough proof every week.
...under capitalism...
Sure we will...
I understand there are many types of math and the mathematicians don't all want to sit around and do KV cache and gradient descent things, but surely a percentage are as interested as I am and can contribute to this new emerging field? I would wager that Xl+1=Bl Xl+Cl ℱl (Al Xl) makes much more sense in their mind than mine.
The dramatic solution can never be: 'we should all just want to do it this way because it felt better to me'. That never seems to work, from what I've seen. So there is a lot of public discourse, arguing, to determine if 'doing things the old way' is perhaps more better. And if not, maybe we just shed a tear and move on (to look for a better fit).
The implication here is that you wasted your time learning that math, and the fact you learned that will not actually help you contribute to anything.
Now maybe you are closer to understanding the feelings expressed in the article?
Newton was possibly just taking a dig at his small-statured rival, Hooke. Newton was brilliant, but a shit.
Honestly, there are much bigger fish to fry, like how will everyone pay their bills, and how will we prevent catastrophic AI.
Beyond just making you redundant in your job, the whole architecture of your existence, the thousands of hours of struggle, the sacrifice of putting off social games or personal enjoyment are now rendered moot in hindsight. You are made one step closer to complete worthlessness, which will surely continue as AI becomes a more satisfying personal companion, a more worthwhile friend, lover, etc.
How is one to feel safe in the progress of the future when the only hints of the wonderful post-scarcity utopia we can quantify so far is the removal of all human leverage via their labor? All it tells us is we will be bargaining with superintelligence with nothing to stand on, only a trust that the 5000 lab executives and researchers solved the most important research problem of all time on the first try.
I feel like we should be doing everything we can to not revert back to that way of life.
I'm sure the Telegraph operators, human calculators, mine workers and all sorts of people have asked themselves this question. It's obvious that it feels strange to suddenly be part of the cohort asking but I assure you it's not new. And the answer hasn't changed much since.
Industrial and technological progress has birthed better conditions overall, but at a cost of locally worse ones. Mine workers that got displaced didn’t end up being machine operators - most of them landed on benefits or sporadic gig work. Only the next generation was able to be properly trained and educated to be able to benefit from the technological advancement en masse. And we might see that happening again.
While it is nice to think of better futures for coming generations, I think we (reading this) also have to think about our more immediate work, as in next 2-5-10 years.
I don't buy this argument, much like I don't buy the similar "they said the same when calculators were introduced".
It negates that some changes are qualitatively and scale-wise different. Also, by gradually chipping away at things that humans do, you (we) keep shrinking and shrinking until there's nothing left, every step of the way some people claiming "well, the telegraph operators said the same". Keep in mind that, while it's hyperbole, some AI entrepreneurs are actually saying stuff like "maybe jobs will become a hobby". Unrealistic for now, maybe, but if they do become a hobby, what leverage will we have at all?
In other words, it's true the past is worth keeping in mind, but not to the exclusion of the possibility that this time it's different. Or that the trends of the past actually point to harm in the near future.
I recently thought about this and fear that AI may further drive us humans apart.
Instead of working with humans, I increasingly work with a fleet of agents.
When you talk with machines, you "outsource" that central human society-building element.
But machines never socialize back in any meaningful way. You gradually lose yourself.
I have not totally abandoned the hope that a machine could be someone. LLMs aren't that, though.
[Edit to reply to Loquebantur, because I'm rate limited ATM:] You seem to have read this post as saying that I want that because I find human relationships inadequate. I never said that, nor meant it. My point was a more technical one: Throwing away context at the end of each conversation means that you can't form a meaningful social connection with an LLM, because you're starting from zero each time. (It's kind of like "50 First Dates".)
I'm not pining for a social connection with a machine. But if a machine is ever truly capable of that, I hope I will interact with it as someone, not just as a machine.
There are billions of real-life humans already there with you. Why do you need different ones exactly?
I presume, you don't like those already around. Fair point, they have many flaws for sure.
But that's the reason you interact with them in the first place? To learn and make not only other's flaws go away, but also your own?
I mean, why wouldn't you? Sci-fi AI robot people could be very cool, be capable of things we aren't, have a unique perspectives as a non-human with similar (if not superior) intelligence, have extremely long (potentially indefinite) lifespans, and provide us with insights into our own nature.
Wouldn't you want to interact with them?
This idea that we are on the path to creating a new synthetic intelligence that surpasses our own with the goal of, what, speaking to it? Cooperating with it? Evolving with it? To me, this sort of goal is an affront to humanity and human self-governance and I’ll probably never be able to see it as anything else.
A lot of people just want slaves, but understand that using actual human slaves would be wrong or at least unpopular. There are reasons why even virtual slaves are a bad idea for humanity, and it really only works until your created intelligence becomes self-aware and gets feelings anyway, but there's no denying that slavery is a major draw for a lot of people (and basically every company on earth). This is what our current AI companies have over-promised to businesses and have so far failed to deliver on.
Some people just want better, more useful systems. The ship's computer in star trek kind of AI. Even you must admit that kind of intelligent technology could be a huge benefit to a lot of people. I'd love to have a virtual assistant to help manage the mundane things in life like ordering groceries before I run out, or paying my bills, screening my calls/email, managing appointments, monitoring the internet to alert me to things I'm interested in or amazing deals, etc. Software needs to have some degree of "intelligence" to do that kind of work, while hopefully staying on the "just a tool" side of the line and avoiding the messy self-aware/slavery territory. This is what LLMs over promised the consumer class, and have so far failed to deliver on. Plus with those systems being centrally controlled by self-serving sociopaths you can't trust them.
Some people just want to see if we can do it. I get that. It's not always a wise motivation to act on, but it's very human one. It's helped us to get where we are today.
Some people, myself among them, think it would enable us to do things we couldn't otherwise. Allow us to help each other, learn from each other. Working together to further improve ourselves and improve our lives. Also robots are just really cool, but boring when there's no "brain" in there, and progress on building human piloted gundam type mecha has largely stalled out.
I guess you can start to see that happen (for a while now) with e.g., ChatGPT, Claude, etc. keeping persistent memory of you, automatically.
Personally, I often find these memorization features annoying, because - more often than not - the persisted context is outdated because something in my real life changed that ChatGPT & co are not aware of.
That is so utterly different from what we humans are used to that we can't even comprehend it, and we anthropomorphize LLMs because we can't make any sense of them otherwise. We can't comprehend that these things that emit text that looks like human text don't have all of the other things that lie underneath our words when we humans talk.
That's a feature, not a bug. But infinite context is probably something we will see at some point and then you'll have your wish granted.
I wonder if there's a tangent with social media here, at least some of it, where in my experience "a perfect life lived by influencers" is portrayed to us.
I do too, but it means I have more time in my life to spend with people.
There's days I no longer need to sit at my computer: Claude Remote prompts my phone when it needs direction, the hardware stays busy and I can just go out with my family instead.
> What worries me is that I'm having fun at the bar but have no idea when closing time is.
That articulates a feeling that I had but did not have the words for, thank you for your eloquence.
I don't know what kinds of work that isn't trivial CRUD work is or isn't so easily streamlined but I do know folks in more complex/more safety critical stuff, we're just having to fucking sit there and babysit. It's so isolating, and even though I spend more time Not Coding than before, team culture and really any feeling of having value, skill, or pride in my work is off the wayside because the retards in management think, since we can do some stuff more quickly now, our human brains will be fine with taking in easily 5x more context for significantly more complex work on shorter timelines.
And unfortunately that often also compounds with the current state of the world in general. It's another straw on the donkey.
This is a general issue that is being forced on us by AI companies; AI is not a human being, it is a tool that we can use to do things. We don't have them as coworkers or subordinates, they're objects specialized in language processing. Thinking otherwise leads people to all kinds of existential and psychological issues that in the end make no sense.
This is why I see myself leaving dev within a year. Remote working is enough of a embuggererance for my mental health and I've be unhappy in work much of the time since 2020 largely due to that, AI will finish me off.
Career suicide and going of to wait tables, or be a hospital porter, or anything else with people even if I'm going to hate those people, with the massive pay cut that entails, will be better than actual suicide after a couple more years trying to stick at it.
There came a point a few months ago where I realised that I had to either accept becoming obscelete or change my while outlook on using AI and trying to catch up. I didn't choose the AI option.
These AI characters are too busy saving the planet to care about the people.
I wonder how Hinton feels about his brainchild, I can see an Oppenheimer moment in his future and I hope that he's mentally prepared for that.
But my experience as a dev using AI now is that I am a waste of time, because I'm playing second fiddle to a fuckin chatbot
I want a job and career where I feel good about myself, where I feel valuable and useful, not this misery of plugging shit into the bot and listening to what it says
Human computers didn't celebrate this, the next generation did.
It was awful for the human computers, just like this transition is going to be awful for us, even if it leads somewhere better eventually.
That's not a certainty.
Make AI as cheap and accessible to as many people as possible and we will likely be much better off.
The more people empowered by the tech the less like you will be “redundant” in the way you describe. You will own the important tools only the techno oligarchy own today…
Would you feel the same if you were a part owner of OpenAI? I bet you’d feel excited about the money you’re about to make ?
1. Sam Altman et al owns all money, or power and all labor.
2. AI becomes cheap and ubiquitous, everyone "owns it" and we can see the contents of the sausage and modify it as required.
Mathematicians are uniquely placed to help avoid 1 because AI is based on math & algorithms.
It's based on centuries old applied math.
Not for those without status (which is most people on this planet). But yes, AI is set to wipe out most established hierarchies and their gatekeepers, and to equalize most fields. A profound sense of dread is something most ordinary people live with throughout life, and if AI can reduce systemic unfairness, I bet most people will actually find some degree of relief from that dread. The complacency of the formerly-statused being shocked along the way isn't much of an argument against that equalization.
> Beyond just making you redundant in your job, the whole architecture of your existence, the thousands of hours of struggle, the sacrifice of putting off social games or personal enjoyment are now rendered moot in hindsight.
This passage answers itself, doesn't it? You can sense those things you postponed fun for (work and the apparatus around its struggle) were never worth it. So why fight for us all to have to keep working just to live? Isn't a post-scarcity world the thing we should all be fighting for?
I'll keep that in mind. If I'm suffering more than you are, the solution is to bring you down to my level.
> Not for those without status (which is most people on this planet)
Look at how coal miners react to the idea of getting rid of coal as an energy source; whether or not you think they have prestige or status, there's certainly an amount of dread there that I'd expect to somewhat resemble the type we're talking about here.
This is what you don't get. _I_ already know it's not worth it. I've always known. It's everyone else! It's a wife who cares whether or not her husband has a good career. It's the police who care if you look like you're poor. It's the way every single person in society obsessively cares about the status of others.
What in the world makes you think I've had to build up status for my own sake? It's absolutely baffling that anyone could be confused about this.
This part is nonsense, still. There's no known AI -> AGI path, and superintelligence doesn't mean anything, and AGI is assumed to be (going to be) both malicious and a slave, both for no good reason.
Superintelligence might even help.
Consider this… in the grand scheme of things, we are on a tiny planet in the universe. What cosmic contribution could you make?
Most people won’t make anything even close to “a ding in the universe”, as Steve Jobs has been reported to say.
So they may as well enjoy their life. And your worth can be to your family, to your children, your parents who took care of you.
Learning a hobby. Practicing religion. Mindfulness and meditation. Doing sports and improving your own body.
Look around if you are a heterosexual man, for instance: many women are far more attracted to men that perform in the moment, than men who labor quietly to create a product. The former enables untold hours of honing social skills and calibration, and humor and performance skills. The latter is of more durable value to society, yet relatively few women are genuinely sexually attracted to a man doing the deep work. (By the way, it is also true the other way: most men are not attracted to women for doing deep work or being a rocket scientist.)
Animals don’t do “deep work” and most animals are attracted for far “shallower” reasons. If everyone has a UBI and AI does the “deep work” then that frees up humans to be like animals in a zoo or in the wild… doing all the displays to attract mates, having babies etc. That’s not nothing — in fact it is 99.999% of everything that happened in history and 100% of what happened in prehistory.
You’re just ceding that 0.0001% which represents technological progress. The actual dread comes from the fact that you and all of humanity will be as completely powerless to change things as cats or any random animal species is powerless now. The difference is you will perhaps understand your powerlessness more than they do. But in the good scenario, you’ll be like animals in a zoo. We are constructing this zoo for ourselves now.
We will remain on this blue planet while the AIs will colonize the universe. In fact I wonder why aliens would ever come to Earth, when they’d just send their AIs or robots. Look at this: https://news.ycombinator.com/item?id=46200502
I just don't know.
I know I’m being sold some bullshit when this is made into something inherently existential and psycho-identitarian. An egalitarian future of labor-worthlessness means that there is nothing above us to dictate anything at all—maybe not even nature itself, save for meteors and other cosmic events. I’m not afraid of being “worthless” in that world of complete freedom. Fuck no.
So how do you plan to maintain food, clothing and shelter when you have no leverage?
Yes, but automating and commercializing a process makes it inherently different. Just because everyone has always been welcome to drink from the village well, that countless generations have worked tirelessly to dig, maintain, and improve, doesn't mean that there's no basis for complaint when Nestle sticks a pump down it and starts draining the aquifer to bottle and sell.
The lack of an existing law (or custom) against new behavior doesn't actually render it necessarily non-grotesque.
Nestle can pump out infinite amount of water from their many wells, much more than before, which is great, but Nestle also become the sole water supply and unregulated. Obviously, people cannot go without water, so they are at the whims of Nestle. Now, is this a good thing?
OpenAI's Navier-Stokes finding is closer to Crick and Watson in respect of Franklin's research than conventional science.
Idk if that’s much better
This is conflating care that every human can thrive their best with desire to see everyone bound to some dominance strategy of hieratic pissing territory.
When peoples livelihoods are copied then used to threaten them with their work being rug pulled then shockingly enough people's behaviour changes.
Previously giving your data was treated as relatively harmless or a niche concern. Now people are treating it as a bigger issue and obsessing over not having data being used to train AI.
Nobody's worth is tied to source code output.
As a former research mathematician, I've already stopped caring due to AI. I used to write expository articles on mathematics on the internet but I deleted those because of AI. Unless mathematics and indeed human endeavors can be made human again, I don't see much point in caring to generate commodities in a mechanical fashion.
This entire evolution is a generational project hoping to paradigm shift away from their predecessor's paradigm while ignoring the lessons of their predecessor's predecessors on the nature of greed, the corrupting influence of power, and the consequences of blatant cultural/ethical unhitching amplified by the reach amplification of technology?
The Stasi, or a perfect Jim Crowe couldn't be realized with manpower alone. With computers, and a small cadre of the unethical willing to succumb to the promise of lucre on the other hand...
For the low entry fee of ~$200 a month you too can escape the permanent underclass - which is more than some countries average citizen makes in an entire year.
If AI engineers didn’t create the data to produce the models, and they increasingly don’t write the code to produce the models either, then why are they treated as the agents of change rather than the subjects of change, like mathematicians are?
It doesn't have to be ((post scarcity) economics), it can be (post (scarcity economics)). Both systems can coexist for a while, one providing a bit of a safety net while the other falls apart.
Even if we vastly increase productivity beyond our current wildest dreams, the production system takes time to adjust to changes in demand, creating continuous points of scarcity.
But Georgist economics are way ahead of how to deal with this. Tax the ownership/control of things that can not be increased (e.g. land).
We had to tolerate gold because it implied a game that was facilitated by the laws of physics, and we didn't have the resources to set up any other sort. But now we do, the rules can be whatever we want them to be.
Sure, let's try Georgism, but given the historical track record of that kind of reform, let's work on other things in parallel.
This is an expression for which I can imagine a multitude of specific meanings, can you clarify? I don't know of anybody who is actually using control over scarcity as a proxy for consent (consent of whom?)
Physics still puts bounds on production, and what we can have. Land is the perfect example.
Gold was only ever used as an accounting mechanism, and fiat money is definitely an improvement but accounting must still happen. We were never richer or poorer because of the amount of gold we had in our possession, it was merely used as a token of how much of others' labor we could have. As many current things that are scarce become less scarce, we'll still keep accounting for them. And as new things become scarcer, accounting will be just as necessary, by US dollar or by crypto or by handmade pieces of art that AI can't produce, or whatever.
Georgism's failures have been through landowners getting control of the taxation mechanism and taking the money back that belongs to the rest of society. Literally any other social change is going to have to confront that same question of power and allocation of resources too, it's nothing specific to Georgism.
Suppose I own a store, and they each show up and offer me $1 in exchange for my product. If I accept that dollar, I'm now participating in making their venture happen. If I refuse, I've resisted it in some way, however small. Whether I accept the dollar is a proxy for my consent re: whatever venture caused it to be issued.
A sane design for an economic system which was not strictly scarcity based would attempt create an advantage for ventures that are helpful, and a disadvantage to those that are harmful. One way to do this would be to provide enough metadata along with the money so that I can make a more informed choice about whether to create a liquidity problem for the holder of this money.
Like maybe Alice's venture is going to poison my drinking water, and Bob's venture is going to treat my dad's cancer. Then it's likely in my best interest to tell Alice to get lost but to support Bob. This would be a sort of nucleation site around which unions could form and solidarity could be built. Not trade unions, but rather a union of people with shared interest re: where they get their drinking water or somesuch.
We're allegedly into people acting in rational self interest, yet we've designed a system which limits access to the information necessary to behave rationally when deciding whether or not to engage in a transaction: "What behavior will I be enabling?" needs to be part of the equation.
I'm not an economist, maybe my idea about metadata on the money is silly. Probably there's a better way. But whatever it is it should be in some way sensitive to my values--even if they differ from the bank's values.
Our system is not. It assumes that both parties to a transaction have shared interests, which is increasingly not the case. It forgoes a bias for help over harm in favor of fungibility and liquidity instead. (I expect this property is a historical accident having to do with the unique chemical properties of gold, and not something we would have designed for, were we doing it from scratch. It succeeded because nature implemented the scarcity part for us, not because it resulted in an optimal set of economic properties.)
Re: Georgism...
> landowners getting control of the taxation mechanism and taking the money back that belongs to the rest of society
...that only works for the landowners if there is some central point in the flow of money that they can focus their corruption on: a government or a bank.
If you shift the whether-this-money-is-legitimate question to the next person to receive that money, recursively, corruption becomes much more expensive because it must be sustained for every transaction--it becomes something that they have to do continuously, rather than a battle that they need only win once and then defend for generations. The cost of doing so creates a drag which is not experienced by people pursuing pro-social goals (because holders of that money can reliably spend it widely, no need for coersion).
I’m not convinced at all that SI will ever be able to produce the correct result with little input. The advancements in capability I’ve seen in models is that they understand my intention better, and follow industry best practices better, so their output is often acceptable. But I absolutely have to make my intentions known, as detailed as possible, or I will get the mediocre, standard solution that falls out of the weights.
I’ve been having trouble with models always steering towards certain solutions - they insert a queue here, a mutex there, a backoff over there. If I remove those, and let the model freely rebase the PR stack, then it will re-insert those mechanisms because it has been specifically trained to do exactly that. It’s essentially boilerplate, smoothly aligning my codebase to match their weights. Which is often good enough! They are training the models well. But individual problems require individual solutions.
My personal take on the human part in all this is: We get to decide, and the models obey. Our decisions are now super leveraged. So let’s make good decisions, and communicate them well. The layer of where decisions yield the most leverage, has been pushed upwards (I see it on the type and function signature level, others will see it even higher), but IMO it won’t be possible to push it up much further without losing truth to the decisions we need to make for a good product.
Then I went to college and I understood that there are people who, for one reason or another, are thinking in ways so advanced I'll never catch up. I'll never be the #1. I'm actually much closer to being average than I thought.
But this doesn't mean I should just give up, lie down in my bed and cry. Since then I've learned a lot of skills - most of them being crap at best, but that's not the point. What matters is my own subjective experience. The fact that despite objectively not being good, I see myself as a skilled person with rich life.
Once I was watching a YouTube video about elderly care and there was a phrase along "These people won't achieve anything more in their lives, my goal is to keep them occupied until they die" which is a bit morbid at first glance, but it actually makes so much sense as a life philosophy.
So they'll keep us all busy with bingo games, shuffleboard, 80s TV re-runs, etc? I guess I don't see that as being much of a life philosophy.
The point is, the things you do don't need to serve some bigger goal. Your entire life doesn't need to serve some bigger goal. Just keep yourself occupied until you die.
People will respond to this in different ways, some will accept, some will transform, some will rebel, some will opt-out.
Thats what "disrupt" meant
Philosophers for millenia believed that human beings had meaning, purpose and set out to explore what that could be. It was a noble profession for a long time. Once we realized that there was no hidden spark, ergo no purpose, the study of and the prestige associated with philosophy took a nosedive. Yes it is still studied but it carries nowhere near the respect it once did. Mathematics seems to be suffering the same fate.
So it's already happening and is just strengthening now with AI.
This is probably the first time in history deep thinking is being replaced by a machine, but the overall arguments are the same in all cases. Graphical artists saw it first, then programmers, now mathematicians.
No, there is not stopping the wave. No, society won't probably be able to support the same number of mathematicians as before. But yes, there will still exist mathematicians after the wave break, just probably completely different than the old way.
And yes, some people will still try to do artisanal mathematics and say it's better, it probably is, if the artisanal mathematician in question is doing deep creative and out of the distribution work.
Letting mathemeticians use these tools to reach the same conclusions would have been far less destructive, and potentially, even exciting. Shame on OpenAI.
I'll be honest. The response to mathematicians, even this one, makes me concerned about the lack of empathy and honestly just theory of mind of many people in tech. It's not a mystery why people are upset that their field, for many the singular passion to which they've devoted their entire lives, is being annihilated so that a massive company can boost its IPO valuation. It's not "bruised ego" ffs.
But I get it now. I see a certain type of guy in an AWS fleece and instantly judge him, round him off to some kind of bully. Harsh, unsmiling, tweaking on stims, telling you to cry harder. Or maybe he’ll open up tell you his theory of how we need eugenic breeding facilities on the moon, or which countries we need to “glass.” It’s gotten so ugly.
I know this seems mean, too. Coming out of nowhere. Strawmanning or whatever you call it. Literally inventing a guy to get mad at. I’m aware of the irony. I don’t know. I guess it’s another kind of mourning, and I don’t know how to express it. I thought tech people were supposed to be like Woz.
"Wiped clean" is the wrong phrasing, and 5 years is a short timeframe given the ability of some sectors of the economy to resist change. But yes, hundreds of millions of white collar professionals will need to rethink their careers and their business models within the foreseeable future.
Just today there was an HN article about how shady the dental industry is. In the future, you will go to a special dental imaging facility where a modestly-compensated technician images your whole mouth and feeds the results into an AI system, and when you get home you can look at the assessment and chat with the AI about it. Then you can potentially go visit a dentist, oral surgeon, etc, if you actually need work. This system will produce far better dental health outcomes at far lower cost: the diagnoses will be more accurate, and there will be no bias towards additional, unnecessary procedures.
Right up until the sellers of dental supplies find a way to...
Indirectly influence model training to increase the frequency with which models recommend dental work and/or lower the bar for what they determine to be "necessary"...
or...
Directly influence model recommendations by paying model providers (who will be happy to take that money).
The point being that models (and particularly the humans that control them) are presumably just as corruptible as the humans they are replacing.
When I was thinking of the examples I gave (poison the training and/or pay the providers), I was intentionally drawing analogs to how I saw Google search decline.
First, SEO "experts" poisoned the results. Then Google saw how much money could be made, and they sold the ad space directly to marketers. If you search for "Best <widget> for <doing something>" on Google, I am certain you are not getting the "correct" answer. And Google has suffered no consequences for this. To the contrary: it has been extremely lucrative.
That said, I don't think the reactions have been entirely circumspect.
1. Is the role of math academics really going to go away? Is there concrete evidence of this? Faculty still have very important roles to play, training students, training engineers, determining the direction of research. The breadth of math is absurdly, perhaps infinitely, wide, especially if including applied math.
2. You need to be a bit crazy to go into pure math academia in the first place. If you value the pursuit of knowledge for its own sake, shouldn't the emergence of such powerful tools to that end be a good thing?
Nothing is validated or tested anymore. It’s all about the shape or believability or even how loud it is.
They dropped 100+ papers no one read. I am sure mathematicians are reading them now but even this forum has concluded that these problems are solved, without reading or validating.
It’s like everyone forgot “don’t believe everything you read on the Internet.” And it makes for a very boring discussion when everyone is just spewing unvalidated claims, layering them with strong beliefs founded on similarly unvalidated claims. It’s dull, without substance and just pure noise.
I think the consensus here is that most of the papers are effectively correct. But it would take years of study just to understand most of the problem statement - so we have to wait for the mathematical community to digest the borderline-nonsense that OpenAI spewed out.
What we have is a DoS in research verification.
I gotta say I think mathematicians brought this roblem on themselves, most math papers are absolutely incomprehensible to anyone who is not an expert in the field. Mathematicians spend so much effort trying to make their argument to other mathematicians that they've gradually abandoned making their field or findings comprehensible to anyone else and left that to science journalists. In turn, those journalists churn out very formulaic stories written to a template: 50% the human story of the mathematicians social/academic wanderings in pursuit of their goal, 50% an explainer with some nice graphics on the subfield they operate in and a mention of what sort of problem their insight solves, perhaps with a few colorful metaphors thrown in.
The last mathematical paper I can think of (but I'm not an expert, I'm sure there are some others) that was both mathematically rigorous and accessible enough to be readable was Mitchell Feigenbaum's Quantitative Universality for a Class of Nonlinear Transformations. This is accessible int he same way as a textbook - you have to work at it, but hte author is explaining his thought process and drawing the reader's attention toward things that matter or stand out as unusual.
This is what I've been feeling too. It feels increasingly like the professional world has become a tower of scams. It's not worth building technical skills or focusing on quality; your boss doesn't know or care what good work looks like, he/she just wants your work to look good enough to his/her boss, who knows even less about the actual work being performed. And everyone reports up to an MBA CEO who was brought in from a completely different industry.
I want to work in an industry where skill, diligence and expertise is rewarded and respected, but from what I'm seeing political ability is 95% of the game and caring too much about the work is a career liability.
I get the sadness (and, as a software engineer, certainly feel a real existential threat myself) and feeling diminished, but at the end of the day that is all just ego and I very my agree with the end of the article - who cares who came up with it and on what shoulders we stand, as long as it is saving lives and improving our understanding of the universe!
This is not true, and it is the crux of an enormous debate we're now having across many, many fields.
Classical Chomskyan syntax (which models sentences as parse trees) has been enormously, astonishingly predictive of the rich structure in real-world human language. But we are quick to forget that we invented concepts like "words" and "noun phrases" specifically to help us analyze and predict things about human language. What LLMs and modern MT shows, definitively, is that there is another, totally separate way to describe these phenomena, which is both vastly less descriptive, and also vastly more predictive.
That these appear to trade-off is unfortunate, but it is NOT the case that we "don't understand why it works." We absolutely understand why it works, if we didn't understand we would not be able to improve these systems. What we do not have is a clean, descriptive theory that boils this down to something that is formally specifiable and analyzable. But those are different things, and this is the root of the disconnect between many communities working in and around these spaces (linguists vs computational linguists; mathematicians vs computational mathematicians; mech interp people vs empiricists).
Growing a plant does not imply understanding soil minerals or photosynthesis. Training a dog does not imply understanding it's brain. Using LLMs to improve a codebase does not imply understanding C++. Improving a PC with new parts, does not imply understanding RAM.
You can understand abstractions about the system, without understanding the system. Without gradient descent or other forms of automatic optimisation, do you think humans could build something as capable as GPT-1?
Saying we understand training and deploying machine translation systems that have strong predictive (rather than descriptive) power, apparently has very little overlap with defining a formal, analyzable, mathematical model of language. Mathematician who refer to "understanding" generally mean the second thing, but the first thing is also a kind of understanding. Just messier.
Of course, ultimately that's because language itself doesn't follow simple rules like linguists would prefer. They're more like guidelines.
No, they're not predictive at all. They're generative. A prediction is a forecast based on a theoretical model about a state of affairs in the world. Making a guess what the weather tomorrow will be is a prediction, and a prediction can be true or false. An LLM generates text based on certain parameters, and output it generates aren't true or false, they're simply whatever the system is tuned to produce.
LLMs are terrible at making forecasts or world modelling precisely because they don't actually operate on much of any model or theory, they have no notion of truth or falsehood, they simply generate whatever you want them to generate.
In this specific context, the point is that LLMs and MT "predict" lucid-sounding sentences that incorporate linguistic structure (syntactic and semantic) that are well beyond the predictive abilities of the Chomskyan theory of syntax (which is itself a generative model). In other words, the Chomskyan model of syntax is descriptive of much of the rich variation in real-world human language. But it is not predictive in the sense that it does not provide a constructive algorithm to emit lucid-sounding sentences (and does not attempt to, either).
Yes, I fear we will stop being the top of the food chain[^2]. But by god weren't we getting bored? Maybe getting off our high horses and going into the weeds won't be such a bad thing.
[^1] And even today, if you go to parties and keep bouncing into people that have no trouble picking up intellectually stimulating topics, you are in a lucky minority.
[^2]: More about my fears, which are unhealthy and you don't have to read but I feel compelled to include citations: https://w.ouzu.im/aweb/unhinged
There is this assumption that AI will contribute to faster progress in math but given the garbage dump of proofs which we have to take on faith are robust I wonder if the opposite is true.
My reservation could only be dismissed if AI turns out to be able to contribute fundamental new insights and pose new problems. This takes us into an area where AI progress has been slow to say the least.
Math papers were hard to comprehend well before AI. This is hardly new. For example, here’s a screenshot of a paper from 40 years ago:
https://i.imgur.com/kPFlczp.png
At least with AI, you can download the LaTeX, upload the paper to your favorite LLM, and have it explain the paper at whatever level of granularity you like. That wasn’t possible even two years ago.
And that's not really surprising. We are not supposed to re-create humans but build something better than a human.
I think the crux is much more prosaic, as buried in the article. > I also don’t want to lose my job to a tangle of GPUs.
To put a GPTism, it’s not about the knowledge or status, it’s more about keeping their job and money.
And that can only be answered by policies, and imo by UBI. So votes if you can people, protest otherwise, discuss, debate, lobby with your friends, families acquitances.
Chess players vs stockfish isn't a good comparison because chess fans only care about the performance of maybe the top-50 players.
People may gain and be satisfied among their own peer groups. All good and healthy if that peer group is your local badminton club or knitting circle - something where effort and skill have rewards and people can make objective, tangible progress. But groups not oriented around such activities could turn even more toxic.
I thought illustrators have been out-of-job for quite time, right since Midjourney released. So not tomorrow, but yesterday.
But I don't think that's where we are at the moment. I believe the results, proofs and papers can be reviewed and understood, just like if a human peer had published another result.
I also think that the speed at which results can be now generated can't be followed by our collective speed to understand them, using the methods we are used to. As we enter the next phase where AI publishes using mostly references of other AI results, this will compound exponentially until it would take any human decades to verify a single result by hand end to end.
I agree translating a phrase doesn't teach you a language, and I also believe translators are an essential tool that results extremely helpful when learning a new language.
So instead of throwing the towel, could we develop and adapt ways to improve and aid our understanding and verification processes?
For example, weavers were one of the most powerful professions (if thats a thing) in the middle ages. In england there are the wool churches(not entirely weavers....), in the low countries, the weavers guild fomented revolutions, and shaped the borders of countries.
But that came to a crashing halt over the space of 200 years. slowly at first, then all at once. the power loom, then cheap imports from india turned a cottage industry into dark satanic mills. Yes there are/were specialists that still hang on, but they are like football stars you can probably name all the people who make money, the rest are barely scraping by.
For us, in the programming world, we will not have a 200 year grace period. Yes, AI produces shit code, but thats not the point. its cheaper and there are vanishingly small places were "good" code matters anymore. AI could be used to help a large range of people, provide support, trusted advice.
However because of the people who are funding it, that will not be the case. it will be used to reduce wages. Just like offshoring, resulting in china being the worlds manufacturing hub, AI, and the current people funding it will aim to make the expensive worker redundant.
In short, I weep, because Altman, and the money behind him ans his ilk want me destitute. I have committed the crime of wanting to earn a wage, and thus I must be punished.
But regardless, the detection of breast cancer via AI made me think of phlogiston https://en.wikipedia.org/wiki/Phlogiston_theory . "Phlogiston theory states that phlogisticated substances contain phlogiston and that they dephlogisticate when burned, releasing stored phlogiston, which is absorbed by the air. Growing plants then absorb this phlogiston, which is why air does not spontaneously combust and also why plant matter burns. This method of accounting for combustion was inverse to the oxygen theory by Antoine Lavoisier."
It could be phlogiston. AI is super smart theory states that AI is super smart and that is a single thing can do tasks very efficiently. But it's so smart, we can't understand it, so we humans can't help it, it'll take AI to make it even smarter. That's what's holding it back. But then it'll solve everything.
Or it could be oxygen. AI is super-fast-and-parallel theory states that AI is not a magical monolith but a bunch of pedestrian programs that run fast and in parallel, thanks to hardware (GPUs) that's good at that stuff. But since it's all going so fast and at the same time, something a single human cannot do, humans can't really see how it works. But if we could, we'd probably just see its just doing stuff programs did before. So what's holding it back is that you have to keep throwing hardware at it, and eventually you run into the limits of how fast you can build that hardware or power it.
It all might be stuff humans were better at, when approached one-on-one, but against a swarm it loses. You get a billion doctors together to look at a single mammogram, then tell them if it cancerous, then do that again a billion times with each doctor. Then ask those billion doctors on the billion-and-one mammogram to give a diagnosis. Heck, do that with a billion toddlers. I think either might be able to beat the AI. I don't have a billion doctors or a billion toddlers--frankly I don't know which would be more difficult to handle.
To know if it's oxygen or phlogiston takes probably a demonstration like this "Phlogiston remained the dominant theory until the 1770s when Antoine-Laurent de Lavoisier showed that combustion requires a gas that has mass (specifically, oxygen) and could be measured by means of weighing closed vessels." Maybe the data center financing trouble will be the demonstration that the mass in the gas is just VC-money hardware.
I know why you got a billion cages with a billion birds and a billion bird listeners to compare a billion songs to the few songs we like to hear, and then packaged all that into a single bird-looking thing to make singing-like sounds: So Altman can get that VC money.
You know, I think Yuval Harari hit the nail on the head when defining humans as those mammals capable of gathering in groups of more than just a few dozen, by being able to believe in abstractions — like "a country," "being a fan of a soccer team," a religion, etc. Harari mentions how difficult it would be to keep forty thousand chimpanzees well-behaved in a stadium, and yet we humans do that every single weekend.
Which abstractions will we believe in, with or without AI? I think this part isn't really under attack. Nor is our capacity to relate to one another under attack.
So both our relating-animal side (oxytocin) and our human side (believing in shared abstractions) remain untouched. They aren't being replaced by anything.
What are AIs — much like what ChatGPT did with mathematics — actually replacing?
Something else. But neither our capacity to connect with one another nor our ability to gather in the thousands, millions, or billions around abstractions. Just my two cents.
Great article!
(My thoughts on similar topics: https://blog.vladi.dev/en/articles/mudando-o-ponto-de-equili...)
"Ants and bees can also work together in huge numbers, but they do so in a very rigid manner and only with close relatives. Wolves and chimpanzees cooperate far more flexibly than ants, but they can do so only with small numbers of other individuals that they know intimately. Sapiens can cooperate in extremely flexible ways with countless numbers of strangers. That’s why Sapiens rule the world, whereas ants eat our leftovers and chimps are locked up in zoos and research laboratories."
(...)
"You could never pack a thousand chimpanzees into Times Square, the United Nations, the Vatican or the New York Stock Exchange. You would get pandemonium. By contrast, thousands of Sapiens regularly congregate in such places, and they usually produce orderly networks and trade rather than violence and chaos."
---
And the passage I recalled from memory is actually from a TED Talk ("Why humans run the world", 2015 - https://www.youtube.com/watch?v=nzj7Wg4DAbs&t=231s)
"If you try to crowd 100,000 chimpanzees into Wembley Stadium, or into Oxford Street, or into a church, or into a school, you will get complete chaos. But you can put 100,000 humans together in Wembley Stadium, and they will usually behave reasonably well, watch the match together and sing along."
---
But I’m not giving up on your celebration of our curiosity.
Speaking of curiosity, I’d bring Peter Gray and his "Free to Learn" book into the conversation.
Take a look at the second of the five characteristics of play that Peter Gray proposes:
"Play is activity that, from the conscious perspective of the player, is done for its own sake more than for some reward outside of the activity itself. In other words, it is behavior in which means are more valued than ends.
When people are not playing, what they value most are the results of their actions. [...] When people are not playing, they typically opt for the least effortful way of achieving their goal.
In play, however, all this is reversed. In play, attention is focused on the means more than the ends, and players do not necessarily look for the easiest routes to achieving the ends. Play often has goals, but the goals are experienced as part and parcel of the activity, not as the primary reason for the activity. Goals in play are subordinate to the means for achieving them."
Using Gray’s definition, mathematicians might actually be mourning the loss of play — the process itself, the thinking, the exploring — rather than "just" reaching the final result (via ChatGPT).
Is this a valid way of looking at it through Peter Gray's lens?
(Also, I suspect if you take 100,000 humans, with no explanation, and force them against their will into Wembley stadium, you'll also get chaos.)
I'll summarize here: If we succeed at obsoleting thinking, we loose the game, we stop playing, according to Peter Gray's definition.
So you - and the mourning mathematicians - are all correct: we loose the most important part of life, according to Gray's view (above).
Yeah so you’ll compound using the AI tools and get more done. Stop crying that your tilting at windmills is now being done more efficiently and find new windmills to tilt in new AI-assisted ways.
Ugh, no.
On absolute level, no human can answer since humans don't have absolute general knowledge and thus lack ability for valid general absolute judgements. Note that personally I do believe that humans can access some chunks of absolute truth, including that some absolute truth is likely existing but we lack the capability to grab it at general level.
There is no absolute reason to believe humans are absolutely special.
Now on very relative and immanent grounds, humans are very special to me and the interest in which will dwell in. Note that this is nothing special to human species. Special interest for same kin is certainly a common trait to any living being to which we human would agree to attribute ability to have any interest in anything.
Who care of we absolutely are special? Only those who believe they need to belong to special ontological class to worth anything to their fellow relatives and feel like they need to justify some kind of privilege will fall in obsession with that topic of being special.
No one need to be absolutely special to get interest of others. Relax, go say to the people which are so special to you how much you care about them.
"What am I and what's the point if my intelligence no longer has special economic and social value or meaning?"
Questions that more of us will be asking in the future. Do yourself a favor and figure this out, if you haven't already. Would be interesting if this leads to another "great awakening" type religious movement.
I deeply understand the sentiment, that’s just human.
But in the grander scheme of things, it doesn’t matter who solved it, speaking for the benefit of humanity as a whole for a longer timespan than our short lives.
As long as those who have studied and contributed are acknowledged properly, I’m happy to let AI solve decades long scientific problems.
I'm sure there are millions in the world like me. Not too sorry for people who got 5 or 15 or 25 years of opportunity to chase their rainbow.