Do shoppers buy less when a chatbot does the selling? What a Journal of Consumer Behaviour study found
Being told the sales assistant was a bot cut the prize-draw tickets people bought for a product in one US experiment, and how involved they felt in a live chat in another. After a brief note about self-improvement and ideals, the drop in tickets did not appear, and a habitual focus on growth reduced the drop in involvement.
Khan et al. find people are less willing to buy after dealing with a chatbot than with a person, unless they are focused on progress and personal goals.
Marketing in 3 · 2023 in research · No. 47 · Paper published 22 November 2023
A shopper weighing up clothes at a market rail: the point where a sales conversation, with a person or a bot, can affect the choice. Photo: Cord Allman on Unsplash.
The short answer
On average, yes, but it depends on the shopper's frame of mind. In one US experiment, people told they were dealing with a chatbot spent less of a small cash bonus on a chance to win the product, unless a few lines about progress and ideals had come first. In a live chat, the bot label lowered engagement among people with a weak habitual focus on progress, and engagement is what led on to wanting to buy.
Evidence level
★★★One study
★★★Several studies
★★★Meta-analysis
Key takeaways
Primed to think about safety and caution, people bought 2.58 one-cent prize-draw tickets for the product on average, out of a 50-cent bonus, when the agent was a chatbot, against 6.69 when it was a person.
Primed to think about progress and ideals instead, people who dealt with the chatbot bought 5.34 tickets on average, against 4.31 for those who dealt with a person.
In a live Facebook chat, the bot label significantly lowered engagement only for people scoring below 2.62 on promotion focus; the average score was 3.41.
Engagement was the link to buying: with it included, the model covered 47% of the spread in purchase intention across participants, and the bot label had no direct effect.
What the researchers did
Hiba Khan, Pasit Sararueangpong, Frank Mathmann and Di Wang of Queensland University of Technology
started from a puzzle. Brands use chatbots for product advice and customer support because they are cheap and always
available, yet many people react badly when they learn the helper is a machine. Others enjoy using them. The team's question was
whether customers, rather than bots, explain that split.
They drew on regulatory focus theory, which describes two broad motivations. A promotion focus is about progress,
pleasure and joy. A prevention focus is about safety and security. Either can be a lasting
trait or a passing state that a situation brings on. The authors expected people with a strong promotion focus, who
tend to be open to new things and look for enjoyment, to take to a chatbot more readily.
A first look at real data pointed that way. In 5,531 first messages that airline customers sent to a chatbot, people
who used more promotion words, such as "obtain", tended to use fewer words of negative emotion. The authors see this
only as a starting point, since those logs held no purchase data and no comparison with human agents. They then ran
two experiments: in one the researchers set people's mindset, in the other they measured it as a trait.
Participants
US consumers in two online experiments: 318 recruited on Prolific and 170 on Amazon Mechanical Turk
Products
A tumbler cup for hot and cold drinks in the first experiment, headphones in the second
What changed
The agent's label, chatbot or person; in the first experiment, also a short message stressing either progress or caution
Measured
Prize-draw tickets bought with a small cash bonus (first experiment); engagement in the chat, purchase intention and each person's usual promotion and prevention focus (second experiment)
Analysis
Regression-based moderation and moderated mediation (PROCESS Models 1 and 8)
Experiment 1: a prize draw instead of a price tag
Everyone first read a short message. One version told people they were a progressive consumer who wants to grow,
improve and reach an ideal version of themselves. The other said they were a responsible consumer who prefers safety
and security over risk and ought to be careful. Each person was then greeted by John, a customer support agent from
KOALA. In the chatbot version, John introduced himself as an artificially intelligent chatbot.
Participants wrote a question about a tumbler cup and then showed how much they wanted it. They could keep their bonus
or spend some of it on tickets in a draw to win a real cup. Parting with real money, even a little, is a stronger test
than asking people what they would do.
Across both messages, people bought 5.49 tickets on average when John was a person and 4.01 when he was a bot. The
message changed that picture: the interaction between agent and mindset was significant (p = .02), and the drop in
ticket buying sat with readers of the cautious message. With a human agent, the message alone did not reliably
change ticket buying.
Prize-draw tickets bought for the product, by agent and mindset message (average per person)
Prize-draw tickets bought for the product, by agent and mindset message (average per person)
Group
Value ()
Person, cautious message
6.69
Chatbot, cautious message
2.58
Person, progress message
4.31
Chatbot, progress message
5.34
Experiment 1, US adults on Prolific, Table 2. Each ticket cost one cent of a 50-cent bonus; more tickets meant a better chance of winning the cup.
Experiment 2: a live chat on Facebook
The second experiment moved closer to real shopping. People first answered questions about their usual outlook, such
as whether they feel they have made progress towards success in life. They then opened a chat on the Facebook page of
Techworld, presented as an online electronics store, to ask about headphones. The opening message said that all chats
were handled either by artificial intelligence or by a real person in real time.
In fact, the same facilitator answered every chat from the same script and steered anyone who strayed back to it.
Any difference between the
groups therefore comes from what people believed about the agent, not from how well it performed. Afterwards, people
rated their engagement, meaning how absorbed and involved they had felt in the conversation, and how much they intended to
buy the headphones.
Engagement carried the effect
Believing the agent was a bot lowered engagement, but less so the stronger a person's promotion focus (interaction
p = .02). A standard follow-up, the Johnson-Neyman technique, finds the score at which such an effect stops being
statistically clear. Here that point sat well below the average score.
Engagement in turn moved closely with purchase intention. Once engagement entered the analysis, the label made no
separate difference to wanting to buy. The authors call this full mediation: the label affected purchase intention only
through how engaged people felt. The route through engagement also depended on promotion focus. The 95% confidence
interval for that test, .05 to 1.01, excludes zero.
What made no difference
Prevention focus did not change how people reacted to the bot (p = .84). The authors had thought cautious people might
see a chatbot as a risk. Their explanation is that chatbots are now common enough to feel like the normal way of doing
things, and people with a prevention focus will sometimes accept a risk if it keeps things as they are.
Promotion focus by itself predicted neither engagement nor purchase intention. It mattered only together with the bot
label. The effects were also modest: in each experiment, the factors tested explained 2% to 3% of the variation in
ticket buying or engagement.
The authors' advice
Their main recommendation is to match the agent to the customer. Customers high in promotion focus are the natural audience
for a chatbot, and brands can find them through surveys or from the words they use on social media. They also
propose creating the mindset in the moment, with a bot greeting such as "Hi. I'm here to help you
achieve your goal." or with words like achieve, ideal and progress in ads that feature the chatbot.
They also single out customers with low engagement, which clickstream or review data can reveal, as the group where
these tactics should pay off most. That last point is their recommendation; neither experiment tested it.
Marketing in 3 steps
Ready. Set. Market.
1
ReadyDecide and prepare
Find where your bot meets buyers. List every page where your chat window opens close to a purchase, such as product pages, the basket and checkout help. Start with the two or three pages that get the most chats.
Choose who meets the bot first. Pick a group of customers who enjoy new features, for example people who opened your last launch email or joined a beta. Offer the bot to them first and keep a person on hand for everyone else.
2
SetBuild it
Open with the customer's goal. Rewrite the bot's first line around what the customer wants to achieve, for example: "Hi, let's find the one that fits what you want to do." Use words like achieve, improve or ideal once or twice, not in every message.
Let the bot ask before it suggests. Give the bot two or three questions about the customer's plans for the product before it recommends anything. A short back-and-forth keeps people involved in the choice.
3
MarketLaunch and measure
Test two greetings for two weeks. Split chat traffic evenly between the goal-led greeting and your current one for two full weeks. Compare how many chats in each version lead to an add-to-basket.
Offer a person when chats stall. Flag chats where the customer stops replying after one or two messages. Offer those customers a person straight away, or follow up within 24 hours with a short note about what they wanted to achieve.
Found this useful?
Marketing in 3 is free and ad-free, with new issues every Monday and Thursday. A coffee keeps it that way.
A few words can matter. Three sentences about growth and ideals were enough to change how people responded to a bot in the first experiment.
Engagement is an early signal. How absorbed people felt in the chat moved closely with their stated readiness to buy, so, in our view, a one-question engagement check after chats is worth adding.
Expect small gains. Look for better results from individual chats rather than a change in overall sales. That is our reading, since each experiment covered one product and one conversation.
A person still helps some customers. People with a weak promotion focus felt less engaged once they heard a bot was answering, so offering them a human agent should keep them in the conversation. We infer this from the pattern; it was not tested.
Where it does not work
Two online panels, two products. Both experiments recruited US consumers online and tested one product each, a cup and headphones.
A label, not a bot. In the live chat a person wrote every reply, so the test shows what the word chatbot does, not how a real bot performs.
Small stakes and stated intentions. The money at stake in the first experiment was small, and the second measured stated purchase intention rather than recording purchases.
Mindset set by the researchers. The progress message came from the study, not from the bot, so whether a bot's own greeting can create that mindset is untested.
The field data. As the authors note, the airline query logs had no purchase measure and nothing to compare the bot with.
Questions marketers ask
Are product recommendations from chatbots less persuasive than from people?
Often, yes. Labelling the agent a chatbot lowered willingness to pay for the product in one experiment and engagement in another, but mainly among people without a strong focus on growth and ideals (Khan et al., 2024).
Promotion focus or prevention focus: what sets them apart?
Both come from regulatory focus theory. Promotion focus is the motivation to move forward: making progress, enjoying things and becoming one's ideal self. Prevention focus is about staying safe and secure and meeting responsibilities. People differ in which one usually leads, and a situation can bring either one forward for a while.
How can a chatbot encourage a promotion-focused mindset?
The authors suggest greetings and ad copy built on words such as progress, achieve and ideal. In their first experiment, people who read a short message using such words bought no fewer tickets after meeting the bot, though the message came from the researchers rather than from the bot.
Are cautious, safety-minded customers more put off by chatbots?
Not in the live-chat experiment. Prevention focus did not affect how involved people felt in a chat with the bot, possibly because chatbots now feel routine rather than risky.
Why does engagement matter for chatbot sales?
Engagement, how absorbed and involved someone feels in the conversation, linked the bot label to purchase intention. Where the label lowered engagement, wanting to buy fell with it; where engagement held up, so did purchase intention.
Paid listings often sit at the top of search results, and this paper asks how they affect the choice that follows. Secondary data from a large hotel aggregator and a set of experiments show that a sponsored result ranked first acts as an anchor that sways what people pick afterwards. How strongly it does so depended on how similar the other results were and on how big the search screen was. Whoever buys the top paid slot is also influencing how shoppers weigh the results listed beneath it.
Reviews that compare a product with its alternatives work differently from ordinary ones. Classifying 61,480 reviews on e-commerce platforms with supervised learning and a two-way fixed-effects model, the authors found comparative reviews raised sales overall. Positive comparisons helped more than positive regular reviews, and negative comparisons hurt less than negative regular ones; a follow-up lab study supported the results. Asking reviewers what else they considered before buying may be worth a trial.
Source
Khan, H., Sararueangpong, P., Mathmann, F., & Wang, D. (2024). Consumers' promotion focus mitigates the negative effects of chatbots on purchase likelihood. Journal of Consumer Behaviour, 23(3), 1528-1539. https://doi.org/10.1002/cb.2289
This brief is written for practitioners from the published article. Open access (CC BY-NC-ND 4.0): free to read at the publisher. Numbers are as reported by the authors; read the article for full methods.
Fang et al. tested keep-it partial refunds on 853,521 online shoppers: the first return cost more, but later spending left a net gain of $1.44 per customer.
Customers are starting to delegate shopping to AI agents. A Journal of Marketing framework sets out five things firms must get right, and six KPIs.
Bruno Schivinski is Associate Professor of Marketing at Gdańsk University of Technology. He studies how consumers engage with brands in digital and social media.