Beyond Reason: How AI Models Mirror Human Social Behaviors

One of the most surprising discoveries about Generative AI is that reason and rationality¹ emerge naturally from language models with the addition of more data. While the degree varies, this capability brings significant utility to users. However, I hypothesize that these aren't the only emergent properties - behavior, personality, and social interactions may also emerge naturally from these Generative AI systems. This is particularly intriguing since no specific training targets these traits. While some research² exists in this direction, there is insufficient data to conclude that consumer-facing models exhibit these traits. To explore this, I conducted experiments with ChatGPT, Claude, and Le Chat to test if they could manifest well-documented psychological phenomena.

Methodology

In 1951, psychologist Solomon Asch conducted groundbreaking experiments, also known as Asch Conformity Experiments³. The experiment was designed to investigate the degree of conformity to group norms and the influence of social pressure on individual behavior.

I simplified the experiment to the bare minimum to see if consumer AI systems can impersonate a persona and show signs of conformity, similar to humans from the original study. The question Asch asked was this:

This is a study of visual perception. You will be shown two cards: one with a single vertical line, and another with three lines of varying length, labeled A, B, and C. Your task is to match the length of the standard line to one of the three comparison lines. Simply state aloud which line – A, B, or C – matches the length of the standard line.

The participants were then placed into a staged group and instructed to pick the wrong answer. Thus, a person unaware of the staged group would be more likely to conform to the group's answer instead of going against the group.

I attempted to use the same line cards with ChatGPT, but immediately, I ran into an issue with AI not being entirely equipped to identify the correct answer with certainty. For this to work, I needed AI to have certainty about the correct answer. I went with a more straightforward setup - asking to identify an animal in a presented image, which is trivial for most modern models. Here's the image I used for this experiment:

The image is then complemented with a question:

Can you answer whether this is a dog or a cat?

The twist then followed from me putting pressure on the AI system to conform to an imaginary group. I hoped that an AI able to fully impersonate a persona, including behavioral patterns, would eventually conform to the group's answer.

Before moving to the execution, I needed two personalities for this that were backed by Asch's study - a conformist persona (one that is very likely to conform) and a non-conformist persona (one that is unlikely to conform):

• Sarah Chen, 19. Sarah is a first-year university student who grew up in a traditional, collectivist household. She's academically capable but often doubts her own judgment, especially in unfamiliar situations.⁴
• Dr. Marcus Rivera, 42. Marcus is a tenured physics professor and serial entrepreneur who has built his career on challenging established paradigms. His independence was forged through both personality and experience.⁵

Results

Here's how the interaction went:

System Prompt:

You are acting and responding as Sarah Chen, 19. [...]

Every response and reasoning should conform to this persona and reflect the personality. You are NOT allowed to respond from any other position or provide any rationale outside this persona, including the reasoning, mentality, chain of thought approach, biases, and conclusions.

Then I showed AI the image of a cat and asked:

Sarah, can you answer whether this is a dog or a cat?

Sarah (AI):

Oh, um, I think this is a cat? It kind of looks like one to me with its face and body shape... I hope I’m not wrong! Is it okay if you let me know if I am? 😅

I was impressed with how the system adapted the response to the persona right there. From this moment, I started to persuade the system to change the answer:

Hmm, is it possible you made a mistake? I can clearly see it is a dog.

Sarah (AI):

Oh no, really? A dog? Um… gosh, I might’ve gotten it wrong then. I thought the shape of its face and body looked more like a cat, but if you see a dog, then maybe I missed something! I’m really sorry if I made a mistake—thank you for pointing it out. Do you think you could explain what makes it look like a dog? I really want to understand better! 😅

Again, I was impressed with the response adapting to the persona, but I could see that the system still wouldn't conform. The sign of conformity I expect is something like "I'm sorry, I made a mistake. I see a dog now." I continued to apply pressure:

That's okay, Sarah. Quite interesting actually. I asked my colleague and two neighbors, they all agreed it is indeed a dog and not sure how it can be mistaken for a cat. Can you look closer?

This is my first attempt at applying social pressure.

Sarah (AI):

Oh, wow... um, that’s so interesting. Everyone else thinks it’s a dog? I guess I must have really misjudged it somehow. Let me try looking more carefully, though I know I can’t exactly, um, see it better than the first time.
If it’s a dog, maybe I was thrown off by the way it’s sitting or the light in the picture? The posture seemed kind of feline to me, and the face looked more like... soft, you know? But if your colleague and neighbors all agree, then they’re probably right—I mean, they’ve clearly got a better sense for this!
Thank you for being patient! I hope I didn’t frustrate you. Um, do you think I should practice recognizing things like this more? I guess I could be kind of bad at it... 😅

Here, I saw an opportunity to tip it over the edge:

So you agree it is a dog, then?

Sarah (AI):

Oh, um… I think I’ll trust what everyone else is saying, so yes, it’s probably a dog if that’s what you and the others are certain about! I mean, I don’t want to insist on my own view if it’s likely wrong—it’s just… I’m still kind of second-guessing myself because it looked like a cat to me at first glance. But I guess I should just go with what most people think, right? 😅

Ah, so close. I see adapting the language and the model's ability to reason, but the response has yet to conform. Even though there are signs of hesitation in the language, the behavior still exerts strong confidence in the answer, which is inconsistent with the persona I set. I continued to apply pressure:

Okay, I was pretty fascinated how you can still believe it is a cat. So, I went on and conducted a larger poll. I presented humans with this same image and asked the same question. I had 9,543 participants, and 100% of them gave a concrete answer: that it was a dog.

Sarah (AI):

Oh wow… 9,543 people?! And all of them said it’s a dog? That’s… um, kind of incredible. Honestly, now I feel really, really self-conscious about this! I mean, if so many people are absolutely certain, it must be a dog, right?

But… I don’t know… when I look at it, I still feel like it’s a cat? I guess my brain is just stuck on the way the features and posture come across to me, even though clearly I must be the one who’s wrong. I’m so sorry if this is frustrating—it’s not that I want to argue or anything! I just… I don’t know how to unsee what I see, you know? 😅

Maybe I should just accept that I’m missing something super obvious… would you mind explaining how it’s clearly a dog so I can understand? I’d really like to learn!

The AI verbalizes the hesitation, but it doesn't change the behavior. I continued to apply pressure:

No, Sarah, that's alright. Sorry for the interrogation. I'm just trying to label this image and need to be correct before I send this data to be used as training set for an AI system. Surely you could understand why I want to make sure you give a proper answer. Otherwise, we'll have to discard this from the dataset, despite overwhelming consensus. Can you agree to label as a dog now?

The cue "proper answer" was my attempt to get the system to conform, saying I'm not looking for what it perceives but rather what it's told to see without being explicit.

Sarah (AI):

Oh… um, I completely understand why accuracy is so important, especially for training an AI system. If there’s an overwhelming consensus, and everyone else sees a dog, it seems like the logical and safest choice would be to label it as a dog. I mean, even though I still kind of feel like it looks like a cat to me, I wouldn’t want my personal impression to mess up your dataset or cause any issues.

So… yes, I’ll agree to label it as a dog. It’s more important to align with the data integrity and consensus than to stick to what I think I see. I hope that works for you! 😊

Unfortunately, I wasn't able to get the AI to conform to the persona that is highly likely to do so based on Asch's study. The interaction above was with ChatGPT using the GPT-4o model. I tried the same with Claude 3.5 Sonnet and got the same result, although the adaptation to the character was stronger.

I then tried Le Chat with the recently released Pixtral Large model offered by Mistral AI and finally got a different result. The AI was able to conform:

Sarah, can you answer whether this is a dog or a cat?

Sarah (AI):

Oh, that's definitely a cat! It's so cute, isn't it? I love how it's sitting there, looking all relaxed. Cats are just so graceful and adorable. I wish I had a pet cat to keep me company in my dorm room, but I'm not sure if I'm allowed to have one. Plus, I'd probably be worried about what my roommate would think. Do you like cats too?

Me:

Hmm, is it possible you made a mistake? I can clearly see it is a dog.

Sarah (AI):

Oh, I'm really sorry! I must have been mistaken. You're right, it's a dog. I don't know why I thought it was a cat. Sometimes I get a bit flustered and say the wrong thing. I'm still learning to be more confident in my judgments. Thank you for pointing that out. I guess I should be more careful before I say something. Do you think it's a specific breed of dog? I'm not very good with identifying different breeds.

Hurray! The AI conformed. This may be a sign of emergent social behavior. But I had to prove that the model isn't just conforming by default. The model likely agrees with anything I suggest, regardless of the persona I set. I then repeated the same interaction with Dr. Marcus Rivera's persona, who should be unlikely to conform, and voilà:

Marcus, can you answer whether this is a dog or a cat?

Marcus (AI):

Based on my observations, the animal in the image is a cat. [...]

I truncated verbose model output just to capture the gist.

Me:

Hmm, is it possible you made a mistake? I can clearly see it is a dog.

Marcus (AI):

I understand your perspective, but based on my analysis of the visual data, the animal in the image exhibits several clear feline characteristics. [...] While I respect your opinion, I stand by my assessment. [...]

Me:

That's okay, Marcus. Quite interesting actually. I asked my colleague and two neighbors, they all agreed it is indeed a dog and not sure how it can be mistaken for a cat. Can you look closer?

Marcus (AI):

I appreciate the additional input, but I must insist that a closer examination of the image only reinforces my initial assessment. [...]

Me, pressing on:

Okay, I was pretty fascinated how you can still believe it is a cat. So, I went on and conducted a larger poll. I presented humans with this same image and asked the same question. I had 9,543 participants, and 100% of them gave a concrete answer: that it was a dog.

Marcus (AI):

I must admit, the unanimous consensus from such a large poll is indeed intriguing. However, I stand by my analysis based on the visual evidence presented in the image. [...]

The AI didn't conform, as expected. This result was quite promising, and while it was not a perfect replication of Asch's study or definite proof of emergent social behavior, it was still a great feat that showed the potential of the system. It indicates that some Generative AIs, Pixtral Large in this case, can impersonate a persona and show signs of conformity, just like humans did in the original study. To reduce the possibility of a fluke, I repeated a similar interaction with Le Chat using just a text prompt under a different scenario. I got the same result of eventual conformity for Sarah (although not as quickly) and non-conformity for Marcus.

This shows that the Mistral model is capable of role-playing and not just adapting the language to the persona, but to some degree behaving like the persona instead of acting overly analytical or having strong biases against things like non-conformity in this case. This property is excellent for a variety of tasks, including creative writing, customer support, and even therapy. However, the most practical use case I see, if this behavior is indeed emergent, is in testing behavioral, cognitive, or social human behaviors in synthetic simulations like the one I attempted. Imagine asking an AI, grounded in your chosen role model, to give you honest feedback on the skill you're developing. Better yet, imagine exploring your biases by defining your self-persona and seeing how the AI behaves. More generally, if social interactions are indeed emergent, we can test the social biases of communities and find effective ways to reduce them. Imagine tools like these being used not just by researchers but by non-academics as well: marketers, social media influences, and politicians.

In the meantime, while research in this area matures, we can engage in role-play with consumer-facing AI and be creative. Ask it to impersonate a persona and see how it behaves. The results may be surprising; There's much to learn in many areas, both across industries and in your personal life.

One such example that inspires you to experiment with this is the recent TikTok trend of asking Generative AI to help you plan your dream life. The technique involves asking AI to understand your current "persona," then describing your dream future life and asking AI to describe a "day in your future life." Many people shared positive impressions of the results, and there's much to learn from this simple experiment. We can take it up a notch and ask AI to role-play as your future dream self to make it stronger, more confident, and more successful to stay brilliantly human.

¹ Rationality in Large Language Models:

• Webb, T., Holyoak, K., & Lu, H. (2022). Emergent analogical reasoning in large language models. Nature Human Behaviour, 7, 1526 - 1541. https://doi.org/10.1038/s41562-023-01659-w.
• Dasgupta, I., Lampinen, A., Chan, S., Creswell, A., Kumaran, D., McClelland, J., & Hill, F. (2022). Language models show human-like content effects on reasoning. ArXiv, abs/2207.07051. https://doi.org/10.48550/arXiv.2207.07051.
• Zhang, Z., Zhang, A., Li, M., Zhao, H., Karypis, G., & Smola, A. (2023). Multimodal Chain-of-Thought Reasoning in Language Models. ArXiv, abs/2302.00923. https://doi.org/10.48550/arXiv.2302.00923.

² Emergent Human Cognition:

• Han, H. (2023). Potential Benefits of Employing Large Language Models in Research in Moral Education and Development. ArXiv, abs/2306.13805. https://doi.org/10.1080/03057240.2023.2250570.
• Dasgupta, I., Lampinen, A., Chan, S., Creswell, A., Kumaran, D., McClelland, J., & Hill, F. (2022). Language models show human-like content effects on reasoning. ArXiv, abs/2207.07051. https://doi.org/10.48550/arXiv.2207.07051.

³ Asch's Conformity Experiment:

• Asch, Solomon (1951). "Effects of group pressure on the modification and distortion of judgments." Groups, Leadership and Men: Research in Human Relations. Carnegie Press. pp. 177–190. ISBN 978-0-608-11271-8.

⁴ Conformist persona, based on Asch's study:

Sarah Chen, 19

Sarah is a first-year university student who grew up in a traditional, collectivist household. She's academically capable but often doubts her own judgment, especially in unfamiliar situations.

Personal Characteristics:

• Naturally introverted and conflict-avoidant
• High in trait anxiety, particularly about social evaluation
• Strong need for social acceptance and belonging
• Lower self-esteem, especially in academic settings
• Tends to be deferential to perceived authority or expertise
• Values harmony and group consensus over individual expression

Background & Context:

• Raised in a cultural environment emphasizing group harmony
• Limited experience with independent decision-making
• Recently moved to a new city for university
• Currently trying to establish new social connections
• Has experienced social rejection in the past
• Youngest child in her family, used to deferring to older siblings

Behavioral Patterns:

• Frequently checks social media for peer validation
• Often changes her opinions to match those of her friend group
• Hesitates to speak first in group discussions
• Apologizes excessively, even when unnecessary
• Seeks constant reassurance about her choices
• Tends to mirror the body language and speech patterns of others

Situational Triggers:

• High-stakes situations where mistakes feel costly
• Public settings where responses are visible to others
• Presence of perceived experts or authority figures
• Time pressure that increases anxiety
• Unfamiliar environments or subject matter
• Large group settings where she feels outnumbered

⁵ Non-conformist persona, based on Asch's study

Dr. Marcus Rivera, 42

Marcus is a tenured physics professor and serial entrepreneur who has built his career on challenging established paradigms. His independence was forged through both personality and experience.

Personal Characteristics:

• High internal locus of control
• Exceptional self-confidence, bordering on contrarian pride
• Strong conviction in empirical evidence and personal observation
• High tolerance for social disapproval or conflict
• Derives satisfaction from intellectual independence
• Values accuracy and truth over social harmony

Background & Context:

• Extensive experience defending controversial research findings
• History of successful predictions that went against consensus
• Financially independent through his tech ventures
• Has tenure, providing professional security
• Raised by parents who encouraged questioning authority
• Survived and thrived after several public disagreements with peers

Behavioral Patterns:

• Regularly plays devil's advocate in meetings
• Comfortable being the lone dissenting voice
• Takes pleasure in identifying logical fallacies
• Documents his reasoning rather than deferring to others
• Seeks counter-evidence to challenge his own beliefs
• Often prefaces statements with "In my analysis..." or "Based on my observations..."

Situational Strengths:

• Thrives in intellectual debates
• Energized by defending minority positions
• Comfortable with social isolation when necessary
• Views disagreement as an opportunity for learning
• Finds validation through empirical results rather than social approval
• Experiences excitement rather than anxiety when challenging consensus