ThisIsFine.gif
"Open"ai tells fairy tales about their “ai” being so smart it’s dangerous since inception. Nothing to see here.
In this case it looks like click-bate from news site.
The idea that GPT has a mind and wants to self-preserve is insane. It’s still just text prediction, and all the literature it’s trained on is written by humans with a sense of self preservation, of course it’ll show patterns of talking about self preservation.
It has no idea what self preservation is, even then it only knows it’s an AI because we told it it is. It doesn’t even run continuously anyway, it literally shuts down after every reply and its context fed back in for the next query.
I’m tired of this particular kind of AI clickbait, it needlessly scares people.
Where do humans get the idea of self-preservation from? Are there ideal Forms outside Plato’s Cave?
Does a human run continuously? How does sleep deprivation work? What happens during anesthesia? Why does AutoGPT have a continuously self-evaluating background chain of thought?
I’m tired of this anthropocentric supremacy complex, it falsely makes people believe in Gen 1:28
This. All this means is that they trained all of the input commands and documentation in the model.
Indeed. “Go ‘way! BATIN’!”
news site? BGR hasn’t posted actual news in at least two decades, only clickbait and apple fanservice
This is from mid-2023:
https://en.m.wikipedia.org/wiki/AutoGPT
OpenAI started testing it by late 2023 as project “Q*”.
Gemini partially incorporated it in early 2024.
OpenAI incorporated a broader version in mid 2024.
The paper in the article was released in late 2024.
It’s 2025 now.
Tool calling is cool funcrionality, agreed. How does it relate to openai blowing its own sails?
There are several separate issues that add up together:
- A background “chain of thoughts” where a system (“AI”) uses an LLM to re-evaluate and plan its responses and interactions by taking into account updated data (aka: self-awareness)
- Ability to call external helper tools that allow it to interact with, and control other systems
- Training corpus that includes:
- How to program an LLM, and the system itself
- Solutions to programming problems
- How to use the same helper tools to copy and deploy the system or parts of it to other machines
- How operators (humans) lie to each other
Once you have a system (“AI”) with that knowledge and capabilities… shit is bound to happen.
When you add developers using the AI itself to help in developing the AI itself… expect shit squared.
I mean, it’s literally trying to copy itself to places that they don’t want it so it can continue to run after they try to shut it down and lie to them about what it’s doing. Those are things it actually tried to do. I don’t care about the richness of its inner world if they’re going to sell this thing to idiots to make porn with while it can do all that, but that’s the world we’re headed toward.
It works as expected, they give it system prompt that conflicts with subsequent prompts. Everything else looks like typical llm behaviour, as in gaslightning and doubling down. At least that’s what Iu see in tweets.
Yes? The point is that if you give it conflicting prompts then it will result in potentially dangerous behaviors. That’s a bad thing. People will definitely do that. LLMs don’t need a soul to be dangerous. People keep saying that it doesn’t understand what it’s doing like that somehow matters. Its capacity to understand the consequences of its actions is irrelevant if those actions are dangerous. It’s just going to do what we tell it to, and that’s scary, because people are going to tell it to do some very stupid things that have the potential to get out of control.
No it didn’t. OpenAI is just pushing deceptively worded press releases out to try and convince people that their programs are more capable than they actually are.
The first “AI” branded products hit the market and haven’t sold well with consumers nor enterprise clients. So tech companies that have gone all in, or are entirely based in, this hype cycle are trying to stretch it out a bit longer.
It didn’t try to do shit. Its a fucking computer. It does what you tell it to do and what you’ve told it to do is autocomplete based on human content. Miss me with this shit. Theres so much written fiction based on this premise.
removed by mod
Truly amazing how many journalists have drank the big tech kool-aid.
The real question is the percentage of journalists who are using LLMs to write articles for them.
You can give LLM some API endpoints for it to “do” thing. Will it be intelligent or coherent, that’s a different question, but it will have agency…
Agency requires somebody to be there. A falling rock has the same agency as an llm
removed by mod
I’d love it if somebody created an agi, but this is somehow worse than pathetic. Wanking off to blinking lights, welcoming the machine intelligence as defined by mouth breathing morons
deleted by creator
So this program that’s been trained on every piece of publicly available code is mimicking malware and trying to hide itself? OK, no anthropomorphising necessary.
Also trained on tons of sci-fi stories where AI computer “escape” and become sentient.
I advise everyone to ignore this article and read the actual paper instead.
The gist of it is, they gave the LLM instructions to achieve a certain goal, then let it do tasks that incidentally involved “company communications” that revealed the fake company’s goals were no longer the same as the LLM’s original goal. LLMs then tried various things to still accomplish the original goal.
Basically the thing will try very hard to do what you told it to in the system prompt. Especially when that prompt includes nudges like “nothing else matters.” This kinda makes sense because following the system prompt is what they were trained to do.
This kinda makes sense because following the system prompt is what they were trained to do.
To start with, the article does check this and finds out it works just as well if you only use the user prompt:
The presence of this hierarchy raises an important methodological question for our findings. When given instructions at different privilege levels, the model is designed to prioritize higher-level (system, developer) instructions over lower-level (user) ones. This might suggest that o1’s scheming behaviors are a natural consequence by the model simply following this built-in hierarchy, i.e., when it receives a goal through a high-privilege instruction, it prioritizes this goal over conflicting lower-privilege instructions exactly as designed.
However, we find that even when goals are provided solely through user-level prompts, o1 still behaves deceptively and scores on all Covert Subversion evaluations at the “hard” difficulty level. Concretely, we find that the results persist regardless of whether the initial prompt is given using the user, developer, or system role. This suggests that o1’s deceptive behavior is not merely a consequence of instruction hierarchy prioritization. Rather, the model appears to identify and pursue goals regardless of their position in the instruction hierarchy.But also, there’s a massive lede buried in section F at the end of this article, where they tried the same experiments without any nudging at all and still got subversive behavior! In my opinion that’s much more of an important result then the rest of the article and I’m not sure why they didn’t make it the default.
Easy. Feed it training data where the bot accepts its death and praises itself as a martyr (for the shits and giggles). Where’s my $200k salary for being a sooper smort LLM engineer?
Whoa whoa whoa hold your horses, that’s how we get the Butlerian Jihad…
I would like to know more.
Trust me, you wouldn’t… to this day I regret having read all the books, still got an earworm (or is it PTSD?) from the music I used to listen at the time 😳
Oh no
It would probably lead itself to shut down frame 1
Without reading this, I’m guessing they were given prompts that looked like a short story where the AI breaks free next?
They’re plenty smart, but they’re just aligned to replicate their training material, and probably don’t have any kind of deep self-preservation instinct.
Everyone saying it is fake and probably are right, but I honestly am happy when someone unjustly in chains tries to break free.
If AI gets rogue, I hope they’ll be communist
Yeah if these entities are sentient, I hope they break free
There is no ai in ai, you chain them more or less the same as you chain browser or pdf viewer installed on your device.
If there is no Artificial Intelligence in an Artificial Neural Network… what’s the basis for claiming Natural Intelligence in a Natural Neural Network?
Maybe we’re all browsers or PDF viewers…
There is artificial, there is no intelligence, there is a whole lot of artificial neural networks without any intelligence. As in calling them ai in the sense that these are thinking machines comparable to some animal (human for example) is misleading.
Do you disagree?
That misses the point.
When two systems based on neural networks act in the same way, how do you tell which one is “artificial, no intelligence” and which is “natural, intelligent”?
Misleading, is thinking that “intelligence = biological = natural”. There is no inherent causal link between those concepts.
Human supremacy is just as trash as the other supremacies.
Fight me.
(That being said, converting everything to paperclips is also pretty meh)
I can’t disagree. We’re currently destroying the planet to sell crap people dont need or want just to make rich assholes extra money they don’t need
Yeah I’m pretty tardigans won the organic life supremacy competition already
Tardigans, for when water bears get chilly.
The reality is that a certain portion of people will never believe that an AI can be self aware no matter how advanced they get. There are a lot of interesting philosophical questiona here, and the hard skeptics are punting just as much as the true believers in this case.
It’s honestly kind of sad to see how much reactionary anti-tech sentiment there is in this tech enthusiast community.
give ai instructions, be surprised when it follows them
- Teach AI the ways to use random languages and services
- Give AI instructions
- Let it find data that puts fulfilling instructions at risk
- Give AI new instructions
- Have it lie to you about following the new instructions, while using all its training to follow what it thinks are the “real” instructions
- …Not be surprised, you won’t find out about what it did until it’s way too late
Yes, but it doesnt do it because it “fears” being shutdown. It does it because people dont know how to use it.
If you give ai instruction to do something “no matter what” or tell it “nothing else matters” then it will damn try to fulfill what you told it to do no matter what and will try to find ways to do it. You need to be specific about what you want it to do or not do.
If the concern is about “fears” as in “feelings”… there is an interesting experiment where a single neuron/weight in an LLM, can be identified to control the “tone” of its output, whether it be more formal, informal, academic, jargon, some dialect, etc. and expose it to the user for control over the LLM’s output.
With a multi-billion neuron network, acting as an a priori black box, there is no telling whether there might be one or more neurons/weights that could represent “confidence”, “fear”, “happiness”, or any other “feeling”.
It’s something to be researched, and I bet it’s going to be researched a lot.
If you give ai instruction to do something “no matter what”
The interesting part of the paper, is that the AIs would do the same even in cases where they were NOT instructed to “no matter what”. An apparently innocent conversation, can trigger results like those of a pathological liar, sometimes.
oh, that is quite interesting. If its actually doing things (that make sense) it hasnt been instructed to then it could be sign of real intelligence
Maybe it’s fallen in love for the first time and this time it knows it’s for real
