Elon Musk was asked how fast AI is moving.
His answer wasn’t about the technology.
It was about the one man who got it all right and was still too conservative.
Musk: “I have to give credit to Ray Kurzweil in being actually remarkably accurate in his predictions. If anything, I think he was perhaps a bit conservative in his predictions.”
Kurzweil spent 30 years making forecasts that made serious people uncomfortable.
He predicted timelines that sounded impossible.
He was mocked for it.
He was right about nearly all of them.
And Musk just called him conservative.
Musk: “The dedicated AI compute appears to be growing by a factor of 10 every six months.”
10x every six months.
Musk: “Almost a 100x improvement per year, at least for the next few years.”
Moore’s Law was a 2x improvement every two years.
That single curve drove every technological shift of the last 50 years.
The internet. Smartphones. Cloud computing.
All of it rode a 2x curve.
AI is on a 100x curve.
And the current infrastructure isn’t running beside the new one.
It’s becoming it.
Musk: “Probably a lot of the data centers, maybe most of the data centers that currently do conventional compute, will transition to AI compute.”
Everything that runs the world you know is being rewired for the world that comes next.
Human beings process the future in straight lines.
We take the speed of the last decade and project it forward.
Exponential growth doesn’t work that way.
It’s invisible until it’s everywhere.
The most aggressive forecaster in the history of technology was too conservative.
That’s not about Kurzweil being wrong about the direction.
That’s about the human brain being wrong about the speed.
The limit was never the technology.
It was the organ we use to comprehend it.
And that organ hasn’t been upgraded in 200,000 years.
显示更多
Samsung Electronics is expected to award 78,000 semiconductor workers as much as Won600mn each after reaching a landmark profit-sharing deal, drawing criticism from conservatives who fear it could encourage similar demands at other South Korean companies.
显示更多
Did you know that the US drafted Japanese constitution ?
**The Japanese Constitution (1947) was drafted in Tokyo by U.S. occupation staff under Gen. Douglas MacArthur (SCAP/GHQ), not in Washington.**
A Japanese government committee first submitted a conservative draft; MacArthur rejected it and had ~24 American lawyers (led by figures like Courtney Whitney and Milo Rowell) produce a new version in about a week in February 1946.
The Japanese side then reviewed, translated, and made limited modifications before the Diet adopted it.
US lapdog yesterday, US lapdogs today.
显示更多
Let me lay out the unpleasant arithmetic of the replacement rate, and why a modern society finds it so hard to reach.
A population of 100 women in an advanced economy needs 210 children to replace itself. Why?
Absent sex-selective practices, roughly 105 boys are born for every 100 girls. Evolution overshoots male births because boys are more prone to early death from accidents and disease. Therefore, of 210 children, about 108 are boys and 102 are girls. Not all girls reach the midpoint of their fertile age: accidents, suicide, homicide, and illness take some. In an advanced economy, about 98% of them survive, leaving 100 women to replace the original 100.
Now consider the distribution of children per woman.
Imagine 15 women have no children. Five do so by choice, for various reasons (professional, affective, religious). Ten face unfixable fertility problems, theirs or their partner’s. The 10% figure is conservative: the medical literature points to around 13%, and that does not even count male fertility problems.
Of the remaining 85, 10 have one child, 60 have two, 10 have three, and 5 have four. I am stopping at four to keep the post concise; very few women in younger cohorts have five or more children, but I could adapt the example to account for them.
Hence, the 100 women in this population have 180 children, for a completed fertility rate of 1.8.
Interestingly, this is roughly the rate we saw in many advanced economies until the early 1990s, and in the U.S. until around 2008.
But we are still 30 children short of replacement! Voluntary childlessness is only 5%. Three-quarters of women have two or more children. Look around: most of your friends will have two, plenty will have three or four. And yet, we are well below replacement.
You would not look at this population and call it selfish (is having two kids hedonistic?) or accuse it of losing family values (only 5% of women are choosing voluntarily not to have children).
The point is simpler. To reach 210 births, you need a substantial share of women to have three or more children. Two as the “normal” pattern will not get you there. And modern society makes three or more a costly proposition for most families.
Of course, current fertility rates in most advanced economies are well below 1.8. But my point is that, under present social arrangements, we should not expect 2.1, even if (to humor last weekend’s debate) we banned smartphones and TikTok. We need many, many more families with three or four children.
More pointedly, there is no self-regulating mechanism that pushes a society back to 2.1. The market-clearing analogy many economists use is flawed; scarcity feedback does not work the same way. (Another post on this another day.) And, as I often read, the claim that “nature” somehow regulates current overpopulation is just childish mumbo jumbo.
So yes, the arithmetic of replacement rate is unpleasant.
显示更多
Appreciate it when the AI doomer crowd occasionally lets the mask slip a little (Allen is a co-founder of the anti-AI group Humans First and frequent guest on conservative podcasts pushing, uh, Unabomber messaging)
显示更多
Everyone knows about the 300 Spartans at Thermopylae.
Almost nothing they know is the full story.
Start with the number. There weren't 300 Greeks at that pass. There were around 7,000. Spartans, Thespians, Thebans, Phocians, Locrians, Arcadians, Corinthians. Citizen-soldiers from across Greece who marched north knowing they'd be facing the largest army the ancient world had ever assembled.
The 300 is just the headline. The ones who stayed to the end.
Now the men themselves. King Leonidas wasn't some chiseled 30-year-old. He was roughly 60 years old when he led that march. And the 300 he picked weren't his strongest warriors. They were specifically men who already had living sons. Spartan law demanded it. Leonidas wasn't choosing an army. He was choosing men whose bloodlines could survive their deaths. Every one of them knew what that meant before they ever saw a Persian.
They marched anyway.
And they didn't march alone in the way movies suggest. Each Spartan citizen-soldier was accompanied by helots, the enslaved underclass that propped up the entire Spartan economy, outnumbering their masters roughly seven to one. Hundreds of helots fought and died at Thermopylae too. They get no statues. No films. No name on the monument.
The pass itself was barely 15 meters wide in 480 BC (it's silted up now and looks nothing like it did then). That bottleneck is the only reason a few thousand men could hold off a Persian force modern historians estimate at 70,000 to 300,000. Herodotus said 1.7 million. He was lying, or possibly counting cooks, slaves, and camp followers, but even the conservative number is staggering.
For two days, they held. Wave after wave broken against bronze and discipline. Xerxes reportedly leapt from his throne three times in fury watching his men die. He sent in the Immortals, his elite personal guard, supposedly invincible. They weren't. Not in that pass.
Then the Greeks were betrayed.
A local man named Ephialtes, whose name still means "nightmare" in modern Greek, sold the Persians a goat path through the mountains that flanked the pass. The Phocians assigned to guard it scattered when the Immortals appeared in the dawn fog. Leonidas knew by morning he was surrounded.
He dismissed most of the allied Greek forces. Saved their lives. But here's what almost nobody talks about: roughly 700 Thespians, led by a man named Demophilus, refused to leave. They were citizen-farmers from a small town that knew Persia was coming for them next no matter what. They chose to die beside the Spartans rather than run. About 400 Thebans stayed too, though their motives were murkier and many surrendered when the end came.
So the "last stand of the 300" was actually closer to 1,500 men. The Thespians died to the last. Their town was burned to the ground by the Persians weeks later anyway. They're a footnote in a story that should bear their name.
The final fight happened on a small hill called Kolonos. Spears shattered. Swords broken. Herodotus says they fought with hands and teeth at the end. Leonidas fell early, and the Spartans fought four times over his body to keep the Persians from taking it.
They lost.
Xerxes had Leonidas decapitated and his body crucified, a violation of Persian custom so extreme it tells you exactly how badly that old man had humiliated the king of kings. Forty years later, Sparta sent a delegation to recover his bones and bring him home.
Two Spartans survived the battle. One, Aristodemus, had been sent away with an eye infection. He returned to Sparta and was treated as a coward, shunned, refused fire, refused conversation, until he threw himself into the front line at Plataea a year later and died seeking redemption. The other survivor, Pantites, was sent on a diplomatic errand and missed the fight. He hanged himself from the shame.
That's the world they lived in.
The epitaph carved at the site doesn't brag. It doesn't even mention victory, because there wasn't one. Roughly translated, it just asks the traveler to tell Sparta that her sons died here, obedient to her laws.
A small group of farmers, an old king, an enslaved underclass written out of history, and a town that vanished from the map. Together, for three days in August of 480 BC, they did the math on freedom and decided the price was worth it.
We remember 300 of them.
There were always more.
显示更多
“Their worldview requires them to use the perception that there is an oppressed class that the conservatives are repressing, and only the Democrats are the saviors.”
@GeneHamilton breaks down SCOTUS’ recent voting rights decision on The Arena
显示更多
The
@Aave bonus APY is live on
@Wallet
Earn up to 5% on USDT, xETH and more - managed directly in-app, with capped supply and conservative LTV.
More details:
显示更多
The splc was 501c3 non profit - which in democrat language just means it was a perfect vehicle for fraud and fueling hate hoaxes against conservatives
A full investigation tonight 6 pm et on my triggered podcast see you there!!!
显示更多
Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project.
This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.:
- It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work.
- It found that the Value Embeddings really like regularization and I wasn't applying any (oops).
- It found that my banded attention was too conservative (i forgot to tune it).
- It found that AdamW betas were all messed up.
- It tuned the weight decay schedule.
- It tuned the network initialization.
This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism.
All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges.
And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.
显示更多