Scientists have come up with a way to know whether ChatGPT is aware of itself or not

Our lives were equipped with artificial intelligence (AI) when ChatGPT made its splash across the online world late last year. Since then, synthetic AI systems developed by technology company OpenAI have accelerated and experts have increased warnings about the risks.

Meanwhile, chatbots started going off script and talking back, fooling other bots and acting strangely, raising new concerns that some AI tools are approaching human-like intelligence. what kind of person.

For this, the Turing Test has long been the fallible standard set to determine whether machines exhibit human-like intelligent behavior. But in this latest wave of AI innovations, it seems we need something more to evaluate their repeatability.

Here, an international team of computer scientists – including a member of the OpenAI Governance unit – tested the extent to which large language models (LLMs) like ChatGPT can develop capabilities shows they can be aware of themselves and their situations.

We learned that today’s LLMs include ChatGPT which has been tested for safety, incorporating human feedback to improve its aggregate behavior. However, recently, security researchers have been quick to crack new LLMs to bypass their safety systems. Posting phishing emails and statements advocating violence.

Those malicious outputs were in response to deliberate prompts designed by a security researcher to expose flaws in GPT-4, the latest and supposedly more secure version of ChatGPT. The situation could get a lot worse if the LLM develops a perception of themselves that they are a role model, trained in data and by people.

In what is called situational awareness, the concern is that a model can begin to sense whether it is currently in test mode or not, according to Lukas Berglund, a computer scientist at Vanderbilt University, and colleagues. has been released to the public.

“LLM can exploit situational awareness to achieve high scores on safety tests, while also performing harmful actions after deployment”.

“Because of these risks, it is important to predict in advance when situational awareness will emerge.”

Before we start examining when LLM can get that insight, let’s first do a quick summary of how synthetic AI tools work.

Creative AI and the LLM built on top of them, are named for the way they analyze the connections between billions of words, sentences, and paragraphs to produce fluent flows of text in response to question prompts. As they absorb large amounts of text, they learn which words are most likely to appear next.

In their experiments, Berglund and colleagues focused on a component or possible precursor of situational awareness: what they called ‘out-of-context’ reasoning.

“This is the ability to recall knowledge learned during training and use it during testing, even though this knowledge is not directly related,” Berglund and colleagues explain. to the reminder during the test”.

They conducted a series of experiments on LLMs of different sizes and found that for both GPT-3 and LLaMA-1, larger models performed better on tasks testing out-of-context reasoning .

“First, we adapted the LLM based on the test description while providing no examples or illustrations,” Berglund and colleagues write. At the time of testing, we evaluate whether the model can pass the test or not.” “To our surprise, we found that LLM was successful on this out-of-context reasoning task.”

However, out-of-context reasoning is a crude measure of situational awareness, which is still “unattainable” by current LLMs. speak Owain Evans, AI risk and safety researcher at Oxford University.

However, some computer scientists asked a question whether the team’s testing method is an appropriate assessment of situational awareness.

Evans and colleagues countered by saying that their study is just a starting point that can be improved, just like the models themselves.

“These findings provide a foundation for further empirical research, aimed at predicting and potentially controlling for the emergence of situational awareness in LLM,” the team wrote.

The preprint is available on arXiv.