Most Amazon DSP campaigns don’t have a creative problem — they have a creative hypothesis problem. Media buyers are A/B testing headlines and swap colors when they should be running structured experiments designed to isolate variables and generate compounding learnings. Here’s how to fix that.
Amazon DSP sits at a unique intersection: it gives you Amazon’s first-party purchase and browsing data as targeting signals, but the creative layer still runs on programmatic display and video logic. That means your Amazon DSP creative testing strategy needs to respect both worlds simultaneously — the precision of Amazon’s audience signals and the psychological dynamics of display and video creative. Most teams only optimize for one.
This post lays out a phase-based creative testing framework that treats banner and video as separate optimization tracks, connects creative performance to actual purchase signals, and gives you a repeatable process for compounding learnings over time.
Why Standard Creative Testing Fails on Amazon DSP
Before building a better system, you need to understand why the default approach breaks down. The typical playbook — run two or three banner variants, pick the winner by CTR after two weeks, scale it — has three critical flaws in the Amazon DSP environment.
Flaw 1: CTR Is a Misleading Primary Metric
On Amazon DSP, a high CTR banner is not necessarily a high-converting banner. Because you’re reaching audiences at different funnel stages — from in-market shoppers to lifestyle audiences to remarketing segments — a creative that generates curiosity clicks from top-of-funnel users will inflate CTR while contributing zero to ROAS. Amazon display ad creative optimization requires you to segment performance data by audience type before making any creative judgment.
The right hierarchy of metrics for creative evaluation on Amazon DSP looks like this:
- Remarketing audiences: Detail Page View Rate (DPVR), Add-to-Cart Rate, Purchase Rate
- In-market audiences: New-to-brand purchase rate, DPVR
- Lifestyle/contextual audiences: Brand Search Lift, Video Completion Rate (for OLV), DPVR
Collapsing all of these into a single CTR average is how you end up scaling a creative that looks great on paper but hemorrhages budget at the bottom of the funnel.
Flaw 2: Treating Banner and Video as the Same Creative Problem
The debate around Amazon DSP video vs display ads is usually framed as a budget allocation question. It shouldn’t be. They are fundamentally different creative mechanisms that require separate testing logic.
Display banners operate on pattern interruption — you have roughly 300 milliseconds to register relevance before a user’s eye moves on. The creative variable that matters most is visual hierarchy: what is the first element the eye lands on, and does it immediately signal category relevance to the target audience?
Video (OLV and OTT on Amazon DSP) operates on narrative compression — your job is to deliver a complete emotional or rational arc in 15 to 30 seconds that creates enough purchase intent to survive the gap between ad exposure and the next time that user is on Amazon. These are not the same problem, and testing them on the same cadence and with the same metrics is a structural mistake.
Flaw 3: No Learning Architecture
The most expensive creative testing mistake isn’t running a bad test — it’s running a good test and learning nothing replicable from it. When teams test creative in an ad-hoc way, they accumulate results without building a knowledge base. The fourth campaign has no advantage over the first. A proper Amazon DSP creative testing strategy is designed specifically to generate transferable learnings, not just campaign-level winners.
The Phase-Based Creative Testing Framework
This framework runs in three phases: Isolation, Amplification, and Systematization. Each phase has a distinct objective and a distinct set of creative variables under test.
Phase 1 — Isolation: Test One Variable at a Time, Per Audience Segment
The first phase is purely diagnostic. You are not trying to find your best creative yet. You are trying to understand which creative dimension has the highest performance leverage for each audience segment you’re running.
For Amazon DSP banner ad performance, the primary isolation variables to test are:
- Hero element: Product-forward (white background, clean product shot) vs. lifestyle (product in context) vs. benefit headline-forward (minimal product, max copy prominence)
- CTA framing: Transaction-oriented (“Shop Now,” “Add to Cart”) vs. curiosity-oriented (“See Why 10,000+ Customers Switched”) vs. urgency-oriented (“Limited Stock”)
- Color contrast and brand prominence: High brand visibility vs. category-coded colors that prioritize recognition over branding
For video, the isolation variables are:
- Opening frame: Problem-first (pain state) vs. solution-first (product reveal) vs. social proof-first (testimonial hook)
- Pacing: Fast-cut (under 2 seconds per scene) vs. narrative (3–5 second scenes with voiceover)
- Closing CTA: Direct response end card vs. brand awareness end card
Run each variable test within a single audience segment, not across your entire campaign. This is the discipline most teams lack. If you mix audience segments in a variable test, you can’t tell whether performance differences are driven by creative or by audience composition.
Phase 2 — Amplification: Build Composite Winners and Stress-Test Them
Once Phase 1 gives you clear signal on which variable combinations win per segment, Phase 2 assembles composite creatives from your winning variables and stress-tests them across broader audience sets and placements.
This is where Amazon display ad creative optimization starts compounding. You’re not starting from scratch — you’re combining proven elements. A composite banner that pairs your winning hero element with your winning CTA frame should outperform either individual variant, and your Phase 1 data gives you the confidence to scale it with budget.
Phase 2 is also where you introduce placement-specific creative variations. Amazon DSP serves across Amazon-owned properties (Amazon.com, IMDb, Fire TV, Kindle) and third-party inventory. The same creative does not perform uniformly across these environments. An OTT creative that’s built for a 10-foot TV screen in a living room requires different visual density and audio assumptions than a desktop banner on a news publisher site.
Build at minimum two placement-specific variants in Phase 2:
- Amazon-owned inventory: Higher purchase intent context, product-forward creative with direct ASIN linkage performs best
- Third-party inventory: Lower purchase intent context, brand and category awareness framing outperforms direct-response in most categories
Phase 3 — Systematization: Build a Creative Intelligence Bank
Phase 3 is what separates teams that get incrementally better from teams that plateau. The goal is to codify everything Phase 1 and Phase 2 taught you into a creative brief template that encodes winning hypotheses as defaults for future campaigns.
Your creative intelligence bank should document, at minimum:
- Winning hero element type per product category and audience segment
- CTA language that correlates with purchase rate (not just CTR) per funnel stage
- Video opening frame type that maximizes completion rate and subsequent DPVR by audience type
- Placement-specific creative specifications that have been validated in-market
- Seasonal or promotional creative variables that outperformed evergreen variants (and the magnitude of the lift)
This bank becomes your unfair advantage. When a new product launches or a new audience segment is activated, you’re not starting from zero — you’re starting from a validated hypothesis set that compresses your learning cycle dramatically.
Applying Amazon DSP Ad Creative Best Practices Within This Framework
The word “best practices” in programmatic advertising has become nearly meaningless because it’s used to describe advice that’s true in the aggregate but may be wrong for your specific product, audience, and competitive context. That said, there are a handful of structural guidelines that hold up consistently within this framework.
On Banner Sizing and Format Distribution
When evaluating Amazon DSP banner ad performance, prioritize the 300×250, 728×90, and 160×600 sizes — not because they’re recommended, but because they represent the highest impression volume in Amazon DSP’s inventory mix. Running creative tests on low-impression formats will not give you statistically meaningful data fast enough to be useful.
Dynamic eCommerce Ads (DEA) within Amazon DSP deserve separate mention. They auto-pull product imagery, pricing, and ratings from your ASIN listing, which means creative performance is partially a function of listing quality, not just ad creative decisions. If you’re running DEAs alongside custom creatives, keep them in separate line items so you can isolate creative performance cleanly.
On Video Length and Completion Rate Benchmarks
For OLV placements, 15-second non-skippable units consistently outperform 30-second skippable in purchase rate efficiency — not because 30-second video is less effective per completion, but because the completion rate delta is large enough to matter at scale. The exception is for high-consideration categories (supplements, tech hardware, home goods over $150) where the longer format’s ability to address objections before purchase intent degrades can offset the completion rate disadvantage.
For OTT/CTV inventory on Fire TV and IMDb TV, 30-second non-skippable is the standard and completion rates approach 95%+. This is your highest-quality video impression on the Amazon DSP network — and most teams under-invest in OTT-specific creative because they’re repurposing OLV assets not designed for a lean-back viewing context.
On Frequency and Creative Fatigue
Amazon DSP’s frequency controls are often set at the campaign level rather than the creative level, which creates a blind spot. A creative can hit frequency fatigue while your campaign is still technically within your overall frequency cap if it’s carrying disproportionate impression weight. Monitor creative-level frequency in your reporting and set hard rotation rules — no single creative variant should exceed 40% of total impressions in an active test unless you’ve already entered Phase 2 amplification.
The Forward-Looking Reality of Amazon DSP Creative
Amazon’s push into generative creative tools and its expanding first-party signal set (including streaming behavior from Prime Video and Alexa interaction data) is going to change what “creative optimization” means on this platform significantly. Teams that have built a structured creative intelligence bank — as described in Phase 3 above — will be far better positioned to leverage these tools, because they’ll know which creative hypotheses to test with AI-generated variants versus which they’ve already validated.
The media buyers who win on Amazon DSP in the coming years won’t be the ones who run the most tests. They’ll be the ones who run the most systematic tests and convert those results into compounding advantages. That’s the difference between a testing program and a testing strategy.
Want more frameworks like this? Macetric.com publishes in-depth, practitioner-level analysis for performance marketers who’ve moved beyond the basics. Explore our full library of programmatic strategy, media buying frameworks, and growth marketing intelligence — built specifically for practitioners who need insights they can act on immediately.
