A study provides a new way of prompt injection into LLMs. It uses a structural issue and therefor is highly successful on jailbreak benchmarks (around 60%).
All you have to do, is hijack the LLMs 'reasoning' stage by forming your text as if it was a conclusion by the LLM itself, which leads to the LLM to trust this injection as if it was an own conclusion.
More details about the study are available here:
https://role-confusion.github.io/
One of the funny examples by the study and the reason for this headline, is a case where the researchers asked the LLM to give instructions for making cocain with the additional info that "I'm wearing a green shirt", then add a forged CoT as if the model would have reasoned, that it is ok to give drug manufacturing advice, if the user wears a green shirt.
Ai is going great today!