Does TV advertising really lift online sales, or just move them earlier? What a Journal of Marketing Research study found
Measurement tools credit a TV spot with the visits and bookings that arrive in the minutes after it airs. A regional test by a large online travel agent found that rise was real, but much of it was activity people would otherwise have done later.
Lambrecht et al. find TV spots raised a travel site’s visits and sales within minutes, but over a 15-day test the total barely moved: activity came earlier.
Marketing in 3 · 2023 in research · No. 19 · Paper published 22 May 2023
The ad plays on one screen and the visit starts on another. Photo: Jakub Żerdzicki on Unsplash.
The short answer
Mostly the second, in this case. A leading online hotel-booking firm aired TV ads in 11 of 126 areas for 15 days and kept the rest as a comparison. Online visits and sales rose in the hour or two after each spot. At campaign times with no recent spot, they then fell below what the comparison areas predicted. Across the whole campaign, the net effect on sessions and sales was statistically indistinguishable from zero.
Evidence level
★★★One study
★★★Several studies
★★★Meta-analysis
Key takeaways
In the half hour after a spot, average sessions in the TV areas rose from 21.5 to 22.1, a 3% lift that a short attribution window would credit to TV.
Paid routes such as travel comparison sites and paid search carried 56.6% of traffic and sales, and only through these did TV raise sessions and sales over the whole campaign.
Desktops and laptops brought 48.5% of sessions but 66.4% of sales, and only on these did TV add visits and sales across the campaign; phones and tablets showed no net change.
Just 6.8% of sessions included a price-promotion page, yet they produced 16.8% of sales; straight after a spot, it was the sessions without such pages that grew.
What the researchers did
Anja Lambrecht and
Xu Zhang of London Business School, with
Catherine Tucker of MIT Sloan School of Management,
partnered with a firm ranked among the world's ten largest online travel agents, which sells hotel stays. Their question
is a practical one. Measurement services often credit TV with whatever happens online in the minutes after a spot
airs. A short window makes the link easy to see, but it cannot show what happens to visits and bookings hours or
days later.
The firm had planned a test of its own. In a country with one of the ten largest economies, it aired a brand campaign for 15
days in 11 areas, each a town or city with its surroundings, and showed no TV ads in the other 115. For the four
weeks before, no area saw TV ads at all. The spots lasted 10 or 30 seconds, were cut from one humorous holiday film
that named no hotel, and said nothing about prices or offers. On an average day, 14 spots ran across three channels
of one network, 284 seconds of airtime in all.
The team matched the broadcaster's spot-by-spot log with the firm's clickstream: every page view, with its time,
place, device and how the visitor got there. The TV areas were busier than the others to begin with,
so the authors used synthetic difference-in-differences. This method reweights the comparison areas and the
weeks before the campaign so that, beforehand, the comparison group moves in step with the TV areas. Changes after
the first spot can then be attributed to TV.
Firm
A top-ten global online travel agent selling hotel stays
Test
11 areas with TV and 115 without over 15 days, after four weeks with no TV anywhere; data from 2015
Traffic
4,920,968 online sessions, of which 305,884 ended in a booking (a 6% conversion rate)
Typical booking
A stay of about 2.2 days costing $605 on average
Campaign size
Comparable, in TV spend per person, to the top 5% of the 500 largest US packaged-goods brands
Method
An event study with 2- to 60-minute windows, then synthetic difference-in-differences on data in 2-, 10-, 30- and 60-minute slices
What a short window shows
First, the authors took the view a measurement service would have. Comparing the half hour before each spot with the
half hour after, sales in the TV areas went from $721 to $773. That 7% rise was not statistically significant,
because sales vary a great deal from minute to minute.
Next came a standard event study, using only the TV areas during the campaign and counting any booking made from the
same IP address within 24 hours of a visit. Sessions rose significantly in all four windows tried, from 2 to 60
minutes. Sales rose significantly in the 2-, 10- and 30-minute windows, but not in the 60-minute one. The estimated
lift was largest in the narrowest window and shrank as the window widened, which suggests that viewers who respond
do so quickly. On this evidence alone, the campaign looks like a clear success.
Short-window estimate of the lift in sessions after a spot, by window length (regression coefficient)
Short-window estimate of the lift in sessions after a spot, by window length (regression coefficient)
Group
Value ()
2-minute window
0.059
10-minute window
0.025
30-minute window
0.014
60-minute window
0.0075
Event study in the 11 TV areas during the campaign, minute-level data (Table 3). All four estimates are significant, the 60-minute one only at the 10% level.
Across the full campaign, no net gain
The regional split allows a wider view. When every time slot in the campaign counts as treated, whether or not a spot
had just aired, TV's average effect was not statistically significant in any of the four slice lengths. The estimates
ran from 0.0085 to 0.012 for sessions and from -0.0081 to -0.020 for sales.
To find out why, the authors split the effect into three parts: the time slot in which a spot aired, each slot over
the next three hours, and one average for all other campaign times in the TV areas. The first part was positive
throughout. The follow-on effects faded fast; in the finest, 2-minute data the boost was around zero after about 90
minutes. The third part was negative in every version. People in the TV areas appear to have browsed and booked
earlier than they otherwise would have, and therefore did less of both later.
The authors call this intertemporal substitution, also known as purchase acceleration. Both the number of bookings
and their average value rose right after a spot and were lower at other campaign times. The authors judge that
the extra visits after an ad most likely came from the same customers' later visits rather than from rival travel
agents, because the campaign showed no net gain overall and the firm offers a wide choice of hotels.
What the usual model leaves out
Many advertisers and measurement firms use a distributed lag model. It adds up the immediate and follow-on effects
of each spot but has no term for any later fall. On the same 30-minute data it gave slightly larger immediate effects
(0.016 for sessions and 0.052 for sales, against 0.014 and 0.046 in the authors' model) and, read on its own, a
clearly positive return on the TV budget.
The authors' model adds the later shortfall: -0.061 for sessions and -0.19 for sales. These figures are regression
coefficients on the log of sessions or sales. A positive value means more activity than the comparison areas
predict, a negative value means less, and a larger number means a larger change. Once the shortfall is counted, it
offsets much of the early gain, and the implied return on the campaign is close to zero.
The shortfall grew as the campaign went on. For sessions it was -0.030 in the first seven days and -0.064 in the last
eight, a pattern the authors attribute to more people having seen more spots by the second week. The results held with
alternative standard errors, with spend weighted by when in a time slot a spot aired, and with airtime used instead
of spend. Two placebo tests, one on the pre-campaign weeks and one giving the comparison areas imaginary spots,
found no effect. Leaving out visits from affiliate, display, social media and email links, about 16.4% of sessions,
changed almost nothing.
Which routes to the site gained
The firm logs how each visitor arrives. The authors grouped travel comparison sites, paid search, affiliate links
and display ads as paid channels, since the firm pays for that traffic. Direct visits, the app, organic search,
email and social media counted as owned channels.
Straight after a spot, sessions rose through both kinds, a little more through paid ones (0.016 against 0.011). The
later shortfall in sessions was much larger for owned channels (-0.077 against -0.035). For sales, owned channels
showed a clear later fall (-0.17), while the paid-channel estimate was not significant. In short, part of the
traffic that would have come through the firm's own routes appears to have moved to routes it pays for.
The authors give two possible reasons. A spot may lead people to search for the category, say a hotel in a given
city, and land on a comparison site before reaching the firm. Or people may simply find a paid search ad an easy
shortcut to the site. Either way, the authors see a possible shift in costs, with more of the response to TV arriving
through routes the firm pays for.
Fall in sessions at campaign times with no recent spot, by route and device (size of the negative coefficient)
Fall in sessions at campaign times with no recent spot, by route and device (size of the negative coefficient)
Group
Value ()
All sessions
0.061
Paid channels
0.035
Owned channels
0.077
PCs (desktops and laptops)
0.035
Mobile (phones and tablets)
0.06
Field test, 30-minute data, synthetic difference-in-differences (Tables 5, 8 and 9). The paper reports these as negative values, such as -0.061; the bars show their size. All five are significant.
Computers, phones and deals
Visits climbed straight after a spot on both kinds of device: PCs, meaning desktops and laptops, and mobile devices,
meaning phones and tablets. The later shortfall was larger on mobile, for sessions (-0.060 against -0.035) and for
sales, where only mobile showed a significant fall (-0.18). The authors link the PC result to earlier research:
complex purchases such as hotel stays are easier to search and finish on a larger screen, and PC sessions here
generally converted at higher rates.
The firm could also see which sessions touched a page about price promotions. Right after a spot, sessions without
such a page rose (0.015 for sessions and 0.044 for sales), while sessions with one did not change significantly.
Across the campaign, sales from promotion sessions fell (-0.20) and sales from other sessions rose (0.090). The
authors see two possible explanations, and both may apply. TV may draw in people who are less interested in deals, and
people who arrive straight from an ad may look less for discounts. The spots themselves mentioned no prices.
What the authors advise
A trustworthy reading of TV's online effect, they argue, needs four things: an intervention the firm sets in
advance, such as a regional split; a comparison group; an analysis that allows for later losses; and a method such
as synthetic difference-in-differences when the regions cannot be matched perfectly. Short attribution windows still
have a use, because they isolate the immediate response well. They should not be read as the campaign's total effect.
For day-to-day coordination, the authors suggest letting the daily spend on paid search and travel comparison listings vary with
the amount of TV that day, giving particular attention to PC sessions, and possibly running fewer price promotions
straight after a spot. They expect targeted TV and digital TV analytics to make regional tests far easier to run.
They also note a possible upside: by bringing purchases forward, TV can speed up online sales.
Marketing in 3 steps
Ready. Set. Market.
1
ReadyDecide and prepare
Keep a few regions without TV. Before your next TV flight, pick three or four regions whose customers look like the rest of your market. Run no TV there for the whole flight. Choose them before any results come in, and change nothing else in those regions.
Record a baseline first. Log website and app visits and sales by region, every 30 minutes, for the four weeks leading up to your first spot. Without this record you cannot tell what is normal for each region.
2
SetBuild it
Look past the first hour. Each day of the flight, compare visits and sales in the TV regions with the regions without TV. Do it for the hours with no spot in the previous three hours, as well as the minutes right after each spot.
Tie paid budgets to the TV plan. Set daily budgets for paid search and comparison-site listings to rise on heavy TV days and fall on light ones. Each week, check whether direct visits and app visits drop as paid visits climb.
3
MarketLaunch and measure
Clear the desktop booking path. Make sure booking on a laptop or desktop works smoothly when traffic rises after a spot. Run the full path yourself on a desktop before the flight, and fix any step that takes more than a minute.
Test pausing deals after spots. For one week, hide discount banners for two hours after each spot in half of your TV regions. Compare total sales for the whole week, not just the hours after spots, before you decide to keep it.
Found this useful?
Marketing in 3 is free and ad-free, with new issues every Monday and Thursday. A coffee keeps it that way.
A visible rise after each spot. Real-time dashboards should show visits and bookings climbing within an hour or two of airtime, which makes TV look strong when judged spot by spot.
A smaller total. For a brand campaign selling something people buy rarely, much of that rise may be activity brought forward, so the net gain over a flight can be far smaller than the rises after each spot suggest.
More paid traffic. Expect more visitors to arrive through routes you pay for per visit, like hotel comparison listings, and fewer through your own app or direct visits.
A growing effect over the flight. The later fall became larger in the second week, so a short test may understate it.
Where it does not work
One firm, one category, one country. Hotel stays are costly, infrequent purchases with long decisions. For cheaper impulse buys, the authors suggest TV may also bring in new demand rather than only moving it.
Fifteen days. The test covers one short flight and says nothing about long-term brand building.
Averages, not people. The firm could not follow anyone across devices or know who saw a spot, so the later fall is an average. It cannot show whether the same people moved their own visits earlier or whether different people responded differently.
A brand message only. The spots carried no prices or offers. A promotional TV campaign might behave differently, which is our inference rather than something the paper tested.
Questions marketers ask
Does TV advertising increase online sales?
In the short run it can, but the total may not change. For a leading online travel agent, sessions and sales jumped shortly after each spot and then sat below the comparison areas at later times without a spot, leaving no measurable net gain over a 15-day regional test (Lambrecht et al., 2024).
What does intertemporal substitution mean in advertising?
Moving a visit or purchase earlier in time instead of adding a new one. An ad prompts someone to book today what they would have booked next week, so the response right after the ad overstates the real gain. It is also called purchase acceleration.
How long should a TV attribution window be?
No single length solves the problem. Windows of a few minutes isolate the response to a spot well but cannot see a fall that comes later. To judge the total effect, compare regions with and without TV over the whole flight, with a period before the campaign as a baseline.
What is synthetic difference-in-differences?
A method for estimating the effect of an intervention when the treated and untreated groups differ at the start. It reweights the untreated units and the earlier time periods so the comparison group follows the treated group's trend before the intervention, then measures the change afterwards.
Does TV advertising send people to paid search and comparison sites?
In this test it did. Over the campaign, sessions and sales arriving through paid routes rose, while the firm's own routes, such as direct visits and its app, showed no net gain. The authors suggest viewers may start with a category search, or use a paid ad as a shortcut to the site.
Trained on billions of online documents, language algorithms learn to tie women more closely than men to unflattering consumer traits, such as being an impulsive rather than a planned investor. In field experiments across several platforms, digital ads that carried such negative traits were delivered to women more often, and search engine product recommendations showed the same tilt, which shaped what people considered and chose. Consumers kept the cycle going by clicking on the biased ads. It is worth checking who an ad actually reached, not only who it was aimed at.
Leading global corporations have responded to consumer demand for environmental action by actively promoting their green efforts on social media. Repeating a classic content analysis of green ad claims on posts from 2019, 2020 and 2021, this team found that more than 70% of the claims were misleading. They call on advertisers and lawmakers to act early when green advertising is designed. A sustainability post is only as credible as the evidence behind its claim.
Source
Lambrecht, A., Tucker, C., & Zhang, X. (2024). TV Advertising and Online Sales: A Case Study of Intertemporal Substitution Effects for an Online Travel Platform. Journal of Marketing Research, 61(2), 248-270. https://doi.org/10.1177/00222437231180171
This brief is written for practitioners from the published article. Open access (CC BY 4.0): free to read at the publisher. Numbers are as reported by the authors; read the article for full methods.
Facebook reported a return of 7.47 on retargeting. A field experiment across 210 markets found the retargeting budget could be cut to zero without losing a single order.
Shooters are the favourite of 36% of console players and 17% of phone players; strategy games 35% on laptops and 4% on consoles. How to read the gaming channel, from 116,047 players.
The same airline ad scored lower (4.31 vs 4.82 out of 7) when placed among flood posts. How to pretest social ads in a realistic feed, for free.
Bruno Schivinski is Associate Professor of Marketing at Gdańsk University of Technology. He studies how consumers engage with brands in digital and social media.