Why your Facebook Creative Testing Strategy is Flawed

Paid social advertising presents an opportunity to test different ad creatives and learn what your prospects care about. Any paid social agency worth its retainer is offering creative testing. But is that testing strategy providing accurate and meaningful insights on the content strategy? I’d argue, ‘less than you think’.


During a large audit for a client, we discovered why the test and learn process is mostly flawed. Agencies are deriving misleading insights, as the testing strategy fails to account for a single hard to measure variable. Let’s look at why.

Firstly, testing strategies can be nuanced. Experts will debate:

  1. How much to spend per test
  2. How many creatives to include
  3. The size of your target audience  
  4. Which metrics to use to evaluate a winner

Regardless of the approach, we’re still missing a key variable.

More sophisticated testing strategies will also provide a framework for how derive insights. This is in some senses is better. By structuring tests into phases, marketers can build from learnings in the previous phase.

Phase 1: Core benefit / headline testing  
Phase 2: Imagery (lifestyle vs product)
Phase 3: Ad format (still image / video / carousel / placement)

The concept of phased testing is very appealing to performance marketers. So much to learn. So much to optimize!  

Unfortunately it all comes off the rails when you don’t account for the missing variable. And measurement of it won’t show up in any of your Facebook dashboards. Curious yet?

The missing variable is “Ad Sentiment.”

Ad sentiment refers to the feedback / comments received on ads from users. One or two positive comments on an ad can create a positive sentiment. Facebook trolls are far less likely to add negative comments to an ad with positive sentiment. Previous customers are more likely to share positive thoughts once others have done so. In reverse, a couple of early negative comments can create a negative sentiment.  We’ve seen that once an ad starts to go negative, trolls will pile on the negative comments and destroy any chance for the ad to win within a creative test.

You may be asking, isn’t testing for sentiment part of why you are testing to begin with? Yes. Perhaps certain ads are more likely to garner negative feedback than others. This is true, but the ad platforms’ optimization algorithms don’t appear to accurately account for this when doing creative testing. Here’s why we believe this.

We’ve been running creative tests for a client that has spent 6 figures per month, over many months. In that time we’ve performed hundreds of tests. These tests have been excellent at identifying winning ads and improving the account’s performance. But these same tests have struggled to consistently identify key learnings about WHY certain creatives win over others. Winning ad performance correlated to ad sentiment more closely than it did to any slight variations in ad creative.

Thinks of it this way. You could spin up 20 versions of essentially the same ad and target them to the same audience. Given enough time and budget, you would likely find some ads receive a positive sentiment while others do not. How much will depend on your industry and product? If you are in a sector that is polarized, it will happen more quickly. Ads with positive sentiment will start to win against the ads without. You’ll confidently pick a winner, even though there was no visible difference in the ad creative.

If you are running creative testing on paid social, you may think you are testing the value of different headlines or images. However, in reality your test is far less significant than you assumed. If you want to truly understand what creative elements are driving results, you need to account for sentiment. To do this requires careful moderation and community management, along with a more carefully managed testing process. In addition, an ad with poor creative but a long history of positive sentiment will out-perform a new ad with excellent creative but no comment history. If you are planning a creative refresh within your ad account, be careful to factor this in.

Let's chat

Core Services

Paid Search

Discover the opportunities your competitors are missing

READ MORE

Paid Shopping

Align your product data with how consumers search

READ MORE

Paid Social

Use social testing to discover which content drives growth

READ MORE

Display

Minimize spend wasted on non-human traffic

READ MORE

Measurement

Untangle your data and go beyond just insights

READ MORE

Growth Strategy

Craft a cross-channel strategy to fit your business case

READ MORE

CRO

Remedy the bottlenecks in your customer buying journey

READ MORE

AB Testing

Utilize rapid testing to learn and improve

READ MORE

Expand to see more

Lessons From Attending CXL 10

Last week Quiet Owl’s co-founder David Sinton took part in CXL 10. A growth marketing conference in Austin, TX. 

I was so impressed that on the day of his return to the office he brought the team together to discuss his takeaways from the event. One of things that sets Quiet Owl apart is our mission to help brands develop a growth marketing engine that is based on listening-first. It sounds like David listened a lot at CXL and I thought I would share a few things he shared with our team. Things that we can continue to do well, things that set us apart, and things that we can work on doing better. 

Transparency

Transparency is obviously a hot topic these days in business. This came up in multiple discussions. First with the cofounders of a health and wellness company based out of Minneapolis. David asked them if they had ever had a situation where they felt they overshared or cases where it was unwise to be too transparent. The resounding answer from them, and everyone at the round table, was “No!”. They share it all the good and the bad. Transparency came up again when he struck up a conversation with the CMO of a publishing company based out of Austin. He said their CEO is huge on transparency and publishes their profit and loss statements on a shared drive and gives in-depth quarterly updates for everyone in the company. 

David told all of his employees at his previous agency, “we had a culture where this was forbidden. We didn’t want employees to know how much money we were making or how profitable we were.” He told us how excited he was about changing this and having all of us included in how things are going. He plans to craft a company update by December 1st.

Creating a Quiet Owl Core Values Bible and 3 Year Plan

David spoke about seeing several impressive leaders tout the value of creating company core values and how important that everyone in the company be clear on these. David shared that he wants us all to put more thought into the values / culture / vision of our company. He wants us all to contribute to this with what Quiet Owl means to you, and what you feel our culture and values should embody before producing a company hand book. This will help us find more people to resonate with our frequency and values, because we will all be in alignment. I feel so excited about this, the best leaders have a purpose beyond self-interest / self-profit that is put into action on behalf of the team.

Turning Heads At A Small Table Discussion

While at CXL, David got to share a few ideas and concepts we have been working on here at Quiet Owl. Being open to educate and share information about the cool things we are doing, really opened the doors to new contacts and a greater respect from our kindred spirits in the industry. It’s fun to work for a company where other marketing agencies are realizing that we do very unique and groundbreaking work here. 

The Importance of Quiet time.

Both agencies and leaders of high growth business at the conference spoke about the importance of battling burn-out in their teams. David wants to be transparent and knows a few of us have been feeling the intensity of our business growing and keeping up.  A couple of recommendations he found interesting from the conference were. 

No meeting Mondays.  This allows people to ease into the week and focus on deep / productive / flow work while they are at their freshest.  

The last Friday of every month is a culture day.  Again, no client meetings. Time is spent on team bonding, brainstorming, planning, team therapy, with an early afternoon finish.   This is something he wants to start exploring in December or in the New Year?  He was curious about our team’s thoughts and was excited about the conversations it brought up. 

The Importance of Conflict

I thought this one was super interesting. What boss talks about wanting conflict? If your company isn’t actively experiencing, discussing, resolving some internal conflict then you have a problem. If everyone is just trying to get along, then it is quite likely real problems are being swept under the rug. I think it is amazing that David said he wants each of us to disagree, challenge, voice a concern, complain, surface a problem, or argue in a respectful way at least once a month. A company that, Resolves instead of Reacting, is going to grow and flow as a team with limitless potential. 

One final thought. Conferences are supposed to inspire. And our leader came back inspired and ready to implement some new things and continue to move on the things we are doing well. And, if you have been paying attention, you know that this is much more than an organizational topic. It’s personal. He wants each and every member of the Quiet Owl team to have a voice and be a part of something bigger than themselves. I couldn’t be more excited about this. 

Let's chat

Core Services

Paid Search

Discover the opportunities your competitors are missing

READ MORE

Paid Shopping

Align your product data with how consumers search

READ MORE

Paid Social

Use social testing to discover which content drives growth

READ MORE

Display

Minimize spend wasted on non-human traffic

READ MORE

Measurement

Untangle your data and go beyond just insights

READ MORE

Growth Strategy

Craft a cross-channel strategy to fit your business case

READ MORE

CRO

Remedy the bottlenecks in your customer buying journey

READ MORE

AB Testing

Utilize rapid testing to learn and improve

READ MORE

Expand to see more

How modern day mobsters hold your business ransom online

Imagine a scenario where your business is offered “protection” in exchange for a large sum of money.  And if you decline, bad things start happening.  Really bad things….

For me it is not hard to imagine a scenario like this.  Growing up in a rough suburb of Liverpool, it was common that you were asked to pay a sketchy teenager to “watch” your bike as you locked it up outside the cinema.  What was left of your bike was not pretty if you declined the fee. And  we’ve all seen and loved good gangster movies. Instinctively we know this behaviour is wrong and unjust, even if it is amusing to watch on TV.

Yet business owners and marketers seem more than willing to pay larger and larger portions of their advertising budgets to the largest mobsters of our time.  These payments, if looked at closely, are nothing other than a shake down.  We even thank the mobster for delivering qualified, high performing traffic to our website.

You’ve probably guessed it by now, but the mobsters I’m referring to are search engines.  The irony is not lost on me, that as a growth marketer much of my livelihood has been built on search engines.  But what started as a powerful marketing tool with the potential to be used for great things, has turned into something far more sinister.

Search engines today are holding brands ransom, on their own brand search terms, for larger and larger sums of money.  If you are a business owner or marketer that has invested in building a loyal customer base and name for your business, then you know how important it is to protect that name.  Unfortunately for you, the search engines and your competitors know the value of this traffic too.  When people go to search for your business by name, if you are not paying the search engines for ads to protect that name, then you are replaced by competitors who do, and who in turn siphon off your most valuable traffic.

When a competitor places an ad on another business’ branded search terms this is referred to as “brand bidding”. Brand bidding has been happening for many years now.  It may not be the most high-moral practice, but in itself it is not terrible either.  Most can agree that some competition can bring out the best in us and that there is value in a free-market economy.   Does allowing brand bidding really make the search engines modern day mobsters?

Not really.  Brand bidding itself is not the problem.  The real issue lies deep within the blackbox of the algorithms that determine how to charge advertisers for clicks.

In the earlier days of advertising and brand bidding, you could protect your brand for a relatively small portion of your ad budget.  Let’s use a hypothetical example to illustrate this.

“Joe’s Shoe Store” in 2012 might have spent 90% of Joe’s ad budget on keywords like “Women’s Sneakers” (also known as non-brand keywords) and 10% of the budget on keywords like “Joe’s Shoe Store”.  It might cost Joe $3 per click to advertise for Women’s Sneakers, but likely only about $0.20 per click to capture the vast majority of the traffic searching for Joe’s Shoe Store.  

The reason Joe could protect his brand traffic for as little as $0.20 per click is because the algorithm weighted the value of ad relevance more than the bid.  If you were “Mike’s Shoe Store”, and you wanted to advertise using “Joe’s Shoe Store” as a keyword, you were likely going to be paying several times more than Joe to get any kind of visibility. This was because Mike’s ad had lower relevance to Joe’s branded search term.

In the last two years, I believe the algorithms have shifted.  Search engines appear to no longer be weighing ad relevance as heavily in the bid auction. Our agency has seen brands go from spending $0.20 per click to protect their brand search terms, to  spending upwards of  $4.00 per click.  This on its own is pretty bad, as the search engines are not driving any additional value, they are just charging more for traffic that should really belong to the business in the first place.  But it gets worse….

Smart shopping campaigns and dynamic search ads are driving more and more branded traffic under the disguise of prospecting campaigns.  Your shopping ads are brand-bidding on competitors, even if you are not aware of it.  And their shopping ads are brand bidding on you.  All this goes to help the search engines drive up costs on traffic they played a very small part in delivering.

And if that is not enough, changes in tracking capabilities, as a result of Apple’s recent push for privacy, make attributing results to non-brand search terms harder and harder.  Businesses are seeing a drop in performance in non-brand search, and so they are choosing to invest more in keywords that directly drive conversions.  The result is that brand’s are spending more and more on capturing their own brand search traffic.

I chose to write this article because in the last 6 weeks I’ve audited 13 search advertising accounts for businesses seeking improvements to their digital strategy.  In only 1 ad account was the advertiser spending less than 50% of their marketing budget on branded search terms. In the other 12 accounts the advertisers were not even aware that the majority of their budget was going to their own brand search terms. In two cases less than 30% of their ad budgets were being spent on non-brand keywords.

Here is my advice to companies doing business online and anyone with a search advertising account:

5 Steps to taking back control

  1. Set low manual CPCs on your brand campaigns. Smart bidding on brand campaigns is going to drive up the cost because these keywords convert really well.  You may lose some traffic, but invest what you save in prospecting and building your brand.
  2. Separate out brand search terms in your shopping campaigns and have a different bidding strategy for them.
  3. Stop directly brand bidding on your competitors.  When they see your ads, they will start bidding on your brand search terms.  And as costs escalate, the only winners  are the search engines.
  4. Balance your ad budget to make sure you are spending enough on growth and non-brand search terms. Unless you are a huge brand that is under attack, we advise clients not to spend more than 10% of their search engine marketing budget on brand protection.
  5. If your brand is under attack, consider signing up for The Search Monitor’s free Brand Protection course and learn how you can fight back!

Let's chat

Core Services

Paid Search

Discover the opportunities your competitors are missing

READ MORE

Paid Shopping

Align your product data with how consumers search

READ MORE

Paid Social

Use social testing to discover which content drives growth

READ MORE

Display

Minimize spend wasted on non-human traffic

READ MORE

Measurement

Untangle your data and go beyond just insights

READ MORE

Growth Strategy

Craft a cross-channel strategy to fit your business case

READ MORE

CRO

Remedy the bottlenecks in your customer buying journey

READ MORE

AB Testing

Utilize rapid testing to learn and improve

READ MORE

Expand to see more

Quiet Owl Graced with a 5-Star Review from B2B Software Company Journyx on Clutch

Digital marketing has become more valuable in recent years. As most of the world continues to deal with the pandemic, marketing online has proven to be the best way to communicate with customers for many businesses. Additionally, digital marketing trends are constantly changing the landscape. Most importantly, however, having a robust marketing strategy can exponentially improve your digital reach. 

Quiet Owl is a digital marketing agency that aims to disrupt the traditional focus on flamboyance. We believe that digital marketing should be a long-term growth plan, not just a nearsighted method to garner a massive following without effective retention. At Quiet Owl, we deliver digital strategies that allow your business to be memorable and relevant for the long-term more than competing in a saturated scene of being the loudest.

It all begins with listening to what our clients want to truly achieve. For us, it’s critical to understand our partners to tailor the perfect growth strategy for them. Thus, we’re excited to share with you that we’ve received our first 5-star review on Clutch.

To give you a better insight into their work, Clutch is an established B2B reviews platform that helps firms across the globe connect with the solution providers that they need in order to improve effectiveness and increase productivity.

Clutch is the leader in connecting global service providers with corporate buyers from around the world. The ratings and reviews platform publishes the most extensive and referenced client reviews in the B2B services market.

The first review in our Clutch catalog is from Journyx, a B2B software company. The review discussed the main challenge that the client was faced with, and how we’ve since helped them overcome these obstacles.

“We were looking to grow our digital marketing strategy, optimize PPC campaigns, improve SEO and CRO, and ultimately generate more leads for our business.

After just a few months of working with Quiet Owl, we have seen an improvement in click-through rate and 35% lower cost per click from our Google Ads campaigns. We are also getting more direct leads from Google Ads. Our total sales leads have improved YOY.”

— Michelle Barreto, Marketing Manager at Journyx

Let's chat

Core Services

Paid Search

Discover the opportunities your competitors are missing

READ MORE

Paid Shopping

Align your product data with how consumers search

READ MORE

Paid Social

Use social testing to discover which content drives growth

READ MORE

Display

Minimize spend wasted on non-human traffic

READ MORE

Measurement

Untangle your data and go beyond just insights

READ MORE

Growth Strategy

Craft a cross-channel strategy to fit your business case

READ MORE

CRO

Remedy the bottlenecks in your customer buying journey

READ MORE

AB Testing

Utilize rapid testing to learn and improve

READ MORE

Expand to see more

A/B Testing

Utilize Rapid Testing to Learn and Improve

In the beginning we run A/B tests to validate improvements to the usability and efficiency of your website. This is typically the easy part of testing. As you run out of easy fixes, A/B testing becomes a bit more difficult. This is when the ‘why’ becomes increasingly important.

“Why are you running A/B tests?”

  1. To improve the conversion rates of our website
  2. To learn about what matters to our customer so we can better serve them

If you align more with 1, then it is likely you’ll lose your appetite for testing after 4 or 5 tests.

If you align more with 2, then it is likely we’ll get along.

Let's chat

Core Services

Paid Search

Discover the opportunities your competitors are missing

READ MORE

Paid Shopping

Align your product data with how consumers search

READ MORE

Paid Social

Use social testing to discover which content drives growth

READ MORE

Display

Minimize spend wasted on non-human traffic

READ MORE

Measurement

Untangle your data and go beyond just insights

READ MORE

Growth Strategy

Craft a cross-channel strategy to fit your business case

READ MORE

CRO

Remedy the bottlenecks in your customer buying journey

READ MORE

AB Testing

Utilize rapid testing to learn and improve

READ MORE

Expand to see more

Conversion Rate Optimization

Remedy the Bottlenecks in Your Customer Buying Journey

Conversion Rate Optimization is like seeking enlightenment for your website. It is a process of self improvement, which to some may appear painful, and is never complete. The more you engage in CRO, the more you realize there is no end goal of perfection, only varying degrees of better. To the lucky few who find the process of rooting out weakness gratifying, the opportunities for growth are tremendous.

With each degree of better, your website will better serve your customer, while also making all your other marketing initiatives more effective and competitive.

We collaborate with design and development agencies to continually improve upon their masterful creations. We use a variety of tools to listen for friction points in the customer buying journey, and then propose solutions, which we validate with testing.

Let's chat

Core Services

Paid Search

Discover the opportunities your competitors are missing

READ MORE

Paid Shopping

Align your product data with how consumers search

READ MORE

Paid Social

Use social testing to discover which content drives growth

READ MORE

Display

Minimize spend wasted on non-human traffic

READ MORE

Measurement

Untangle your data and go beyond just insights

READ MORE

Growth Strategy

Craft a cross-channel strategy to fit your business case

READ MORE

CRO

Remedy the bottlenecks in your customer buying journey

READ MORE

AB Testing

Utilize rapid testing to learn and improve

READ MORE

Expand to see more

Growth Strategy

Craft a Cross-Channel Growth Strategy to Fit Your Business

Every business is unique. Every growth strategy starts with an analysis of the business and opportunities. We help clients with:

  1. Business goal planning
  2. Competitor research
  3. Platform strategy and budgeting
  4. Campaign benchmarking
  5. Seasonality forecasting

Do you know that your business could be doing better at digital? Are you unsure which opportunities to tackle first?

Let's chat

Core Services

Paid Search

Discover the opportunities your competitors are missing

READ MORE

Paid Shopping

Align your product data with how consumers search

READ MORE

Paid Social

Use social testing to discover which content drives growth

READ MORE

Display

Minimize spend wasted on non-human traffic

READ MORE

Measurement

Untangle your data and go beyond just insights

READ MORE

Growth Strategy

Craft a cross-channel strategy to fit your business case

READ MORE

CRO

Remedy the bottlenecks in your customer buying journey

READ MORE

AB Testing

Utilize rapid testing to learn and improve

READ MORE

Expand to see more

Measurement

Untangle Your Data. Discover Actionable Insights.

We thrive working with clients that are hungry to learn about their customers and truly understand their marketing growth engine. What better way to learn than to study customer and marketing performance data. However, getting accurate data, and navigating conflicting data sources can put a dampener on businesses seeking to build a data-driven marketing engine.

“As long as it makes money, we don’t care how it works.”

— Not a Quiet Owl Client

Gaining insight into your marketing engine through more accurate data is only the first step. We believe that data has to be more than just intriguing. Data should be used to do good, which means it has to lead to action and improvement.

Let's chat

Core Services

Paid Search

Discover the opportunities your competitors are missing

READ MORE

Paid Shopping

Align your product data with how consumers search

READ MORE

Paid Social

Use social testing to discover which content drives growth

READ MORE

Display

Minimize spend wasted on non-human traffic

READ MORE

Measurement

Untangle your data and go beyond just insights

READ MORE

Growth Strategy

Craft a cross-channel strategy to fit your business case

READ MORE

CRO

Remedy the bottlenecks in your customer buying journey

READ MORE

AB Testing

Utilize rapid testing to learn and improve

READ MORE

Expand to see more

Digital Display Advertising

Minimize Spend Wasted on Non-Human Traffic

Display advertising is fraught with fraud and gives digital marketing a bad rap.

  1. Non-human (bot) traffic inflates both impressions and clicks.
  2. Non-relevant in-app traffic coerces game players to unwillingly click ad banners to keep playing.
  3. DSPs rely on some fuzzy performance metrics to justify high ad costs.

We see Display advertising as a diamond in the rough. It is an essential component to a robust cross-channel strategy and should not be ignored.


We help our clients:

  1. Minimize spend on non-human or irrelevant traffic
  2. Utilize testing to validate incremental growth
  3. Determine where Display fits into a growth engine
Let's chat

Core Services

Paid Search

Discover the opportunities your competitors are missing

READ MORE

Paid Shopping

Align your product data with how consumers search

READ MORE

Paid Social

Use social testing to discover which content drives growth

READ MORE

Display

Minimize spend wasted on non-human traffic

READ MORE

Measurement

Untangle your data and go beyond just insights

READ MORE

Growth Strategy

Craft a cross-channel strategy to fit your business case

READ MORE

CRO

Remedy the bottlenecks in your customer buying journey

READ MORE

AB Testing

Utilize rapid testing to learn and improve

READ MORE

Expand to see more

Paid Social

Use Paid Social Testing to Discover Which Content Drives Growth

Questions we he help clients answer:

  1. What types of content should we be investing in for marketing?
  2. How much content do we need?
  3. Which audiences should we be creating content for?
  4. How much of our budget should we be spending creating content versus promoting it?
  5. How much of our budget should be spent developing new audiences versus retargeting existing users?
  6. How should we allocate budget between social platforms?
  7. What should our ROAS target be?
  8. How much growth can we anticipate in this next quarter?

Navigating a paid social strategy comes with complexity. We strive to make it simple by answering these questions with a deliberate testing strategy. The benefits of paid social testing extend beyond optimization of your marketing budget; instead social testing becomes a methodology for brands to learn about their prospects and customers.

Let's chat

Core Services

Paid Search

Discover the opportunities your competitors are missing

READ MORE

Paid Shopping

Align your product data with how consumers search

READ MORE

Paid Social

Use social testing to discover which content drives growth

READ MORE

Display

Minimize spend wasted on non-human traffic

READ MORE

Measurement

Untangle your data and go beyond just insights

READ MORE

Growth Strategy

Craft a cross-channel strategy to fit your business case

READ MORE

CRO

Remedy the bottlenecks in your customer buying journey

READ MORE

AB Testing

Utilize rapid testing to learn and improve

READ MORE

Expand to see more