complete rejection of even the possibility that we are dealing with bad faith actors

Incorrect. I definitely think we are dealing with bad faith actors. I talk about that at the end of my very first message. I actually agree that the study they looked at, based on asking a chatbot things and then inferring judgements from the answers, is more or less useless. I’m just saying that doesn’t imply that the entire field of AI safety is made of bad actors.

You also claim that you can’t (or don’t want to) provide a brief explanation of your argument and you defer to the YT video. This is another red flag. It is reasonable for one to provide a 2-3 sentence overview if you actually have some knowledge of the issue.

No. I said, “AI chat bots that do bizarre and pointless things, but are clearly capable of some kind of sophistication, are exactly the warning sign that as it gains new capabilities this is a danger we need to be aware of.” That’s a brief explanation of my argument. People deploying AI systems which then do unexpected or unwanted things, but can get some types of tasks done effectively, and then the companies not worrying about it, is exactly the problem. I just cited someone talking at more length about it, that’s all.

I did watch the video and it has nothing to do with grifting approaches used by AI companies.

Yes. Because they’re two different things. There is real AI safety, and then there is AI safety grift. I was talking about the former, so it makes sense that it wouldn’t overlap at all with the grift.

Further on you talk about non-LLM ML/AI safety issues without any sort of explanation what you are referring to. Can you please let us know what you are referring to (I am genuinely curious)?

Sure. Say you train a capable AI system to accomplish a goal. Take “maximize profit for my company” as an example. Then, years from now when the technology is more powerful than it is now, it might be able to pursue that goal so effectively that it’s going to destroy the earth. It might decide that enslaving all of humanity, and causing them to work full-time in the mines and donate all their income to the company’s balance sheet, is the way to get that done. If you try to disable it, it might prevent you, because if it’s disabled, then some other process might come in that won’t maximize the profit.

It’s hard to realize how serious a threat that is, when I explain it briefly like that, partly because the current AI systems are so wimpy that they could never accomplish it. But, if they keep moving forward, they will at some point become capable of doing that kind of thing and fighting us effectively if we try to make them stop, and once that bridge is crossed there’s no going back. We need to have AI safety firmly in mind as we devote so much incredible resources and effort to making these things more powerful, and currently, we are not.

I think it’s highly unlikely that whatever that system will be, will be an LLM. The absolutely constant confusion of “AI” with “LLM” in the people who are trying to dunk on me is probably the clearest sign, to me, that they’re just babbling in the wilderness instead of trying to even bother to understand what I’m saying and why AI safety might be a real thing.

You cite a paper; can you provide a brief summary of what the findings are and why they are relevant to a skeptical interpretation of “AI safety” messaging from organization like Apollo Research and Anthropic?

The only relevance the paper has is that I was challenged to show that LLMs are gaining capabilities over time. That’s obviously true, but also, sure, it’s been studied objectively. They set out a series of tasks, things like adding numbers together or basic reasoning tasks, and then measured the performance of various iterations of LLM technology over time on the tasks. Lo and behond, the newer ones can do things the old ones can’t do.

The paper isn’t itself directly relevant to the broader question, just the detail of “is AI technology getting any better.” I do think, as I said, that the current type of LLM technology has gone about as far as it’s going to go, and it will take some new type of breakthrough similar to the original LLM breakthroughs like “attention” for the overall technology to move forward. That kind of thing happens sometimes, though.