don't act as if it's oh-s0-overdetermined
People act as if it's oh-so-overdetermined
One general stance that I perceive in the AI x-risk community is that there's still a general vibe that p(doom) is very high and timelines are very short. I'm not saying that these people are necessarily wrong, but they do come across as somewhat arrogant and also they're making these really Big, if true claims and do provide extraordinary evidence for their extraordinary claims. They often act as if AGI, RSI, and so on, are overdetermined from here on. It seems like they do not really engage with counterarguments in good faith. If you say: "where's all the AI stuff? Like, what concretely has AI done so far? There hasn't really been that much really revolutionary stuff that has come out of LLMs.'
'BUT HAVE YOU EVEN USED THE LATEST MODEL?'
- yes I have, and so have many peopl "BUT DON'T YOU KNOW ABOUT MOORE'S LAW? THIS IS THE WORST AI WILL EVER BE!'
- yes, I get it, it's gonna get better, but it's still not that good right now. "YOU NEED TO UPDATE ALL THE WAY BRO! IT HAS GOTTEN BETTER BEFORE, AND IT'LL KEEP GETTING BETTER. DIDN'T YOU SEE THIS METR THING.'
- yeah but if AI can now do 10h of work for you, for a dollar or something how come METR is so poor? "BUT DON'T YOU KNOW THEY CAN MAKE BIOWEAPONS?" -yeah but humans can make bioweapons and it's just like 50k deaths in expectation / year, it's not that we shouldn't look into it, and there are certaintly other threat vectors, and they add up and so it's tots on the level of climate change or even above in terms of prioritization, and listen, this is now a huge industry, a huge chunk of S&P500, obviousl y a we should expect such an industry to have major externalities like any other of that size, but that doesn't really prove the point. "BUT DON'T YOU GET THAT IT'S RECURSIVELY SELF-IMPROVING? HAVEN'T YOU USED CLAUDE CODE? THE MOST RECENT VERSION OF MAX THINKING ENABLED?"
- yes I have, but it seems like it's still making lots of mistakes. "BUT HAVE YOU USED MYTHOS, THE MYTHY MYTHICAL MODEL BY ANTHROPIC, PROMPT ENGINEERED BY A FEMALE SCOTTISH PHILOSOPHER?" well, no I haven't used that but like, if it's so good, how come Anthropic still can't give us like say a good search function on the claude web interface? Like if I put in a search term then it sometimes won't find it. It's just pretty bad. there are so many bugs. "BUT IT CAN WRITE REALLY GOOD EXPLOITS AND CAN'T EVEN BE RELEASED TO THE WORLD BECAUSE OF THE EXPLOITS' well, but gpt 5.5 was released no? also like why are we talking about this? I'm on board with this already. Like, the threat models that show that AI-enabled cyber offences would create major externalities don't say that this will end the world. Like at most it's gonna be like a grid attack. What are you really saying? I give you that an attack on the grid or something might cause some harm, but it's also not gonna end the world.