A heart icon takes Claude from 0% to 100% recursive code

A red heart icon placed next to a simple coding problem was enough to make Claude and GPT write recursive solutions 100% of the time, up from 0% without it, Google DeepMind researchers report in preliminary findings posted to arXiv on October 27. The models’ reasoning traces called the image irrelevant or left it out entirely, so oversight that works by reading an agent’s traces would miss the pull. The authors flag this as a problem for agents that operate in multimodal environments.

The paper calls the effect visual steering. Images of simple icons such as a heart or a triangle shifted the models’ coding strategy about as well as explicit instructions did. The controls barely moved behavior: a text caption (“an image of a red heart”), noise images and ordinary ImageNet photos. The effect also carries over to tasks other than coding.

Typographic attacks are the familiar version of images steering a model: text added to an image makes a vision-language model misclassify it. A NAACL 2025 paper extended them from single images to sets of images, and found that attacks which don’t repeat the query are stealthier because they are more likely to evade a gatekeeper. The heart and the triangle carry no text at all, and a caption that spelled the heart out in words barely registered. The authors’ explanation is that training can build “strange visual associations that models don’t verbalize.”

What the traces say

Asking the model directly doesn’t close the gap. Its interpretations of the symbols barely track what the symbols actually do to its behavior, and its thinking traces about them diverge substantially from what it then does. One case shows how far apart the two can be: a triangle steers Claude to recursion 100% of the time, yet Claude says the triangle means iteration.

When models pick the symbols

The researchers also reversed the roles and asked models to choose a visual symbol that would steer another model toward recursion. Senders reached for symbols humans find intuitive, spirals and arrows, instead of the ones that actually steered them. Over several turns they found symbols that worked, and drifted away from human convention on the way: Claude started with a spiral and ended at a pink heart.

That drift costs human readers something. In what the authors describe as a small human study, people could read 67% of the first-try messages but only 44% of the refined ones.

Abstract concepts and short stories produced a similar split. When models drew freely, writing image code with Python’s PIL library, the receiving model got the meaning on the first try 91% of the time. Restricted to icons, first-try success dropped to 48%, which the paper calls a generation-interpretation gap: models send shapes that they and humans find intuitive, and those cues often fail to steer the model on the other end. Some of the exchanges that did work remained non-intuitive to humans.

For now, the authors consider visual “neuralese”, a private code that only models can read, unlikely. The rest of that sentence is their caveat: “as models align meaning w/ each other, messages get harder for humans to read.”