In the world of email marketing, where everyone is competing for customers’ attention, your emails need to stand out from the crowd. Crafting emails that your customers engage with isn’t just about being creative – it’s about using data in a smart, systematic way. That’s where A/B testing with Klaviyo comes into play. It’s a powerful tool that helps marketers optimize their emails to perform better.
In this guide, we’ll walk you through everything you need to know about A/B testing emails in Klaviyo. We’ll cover the basics, why it’s essential, how it works, and how you can use A/B testing to optimize your email campaigns and flows. Plus, we’ll create an example A/B test to try out different subject lines and see which one gets more customers to open our email.
By the end of this article, you’ll be able to create A/B tests that improve key metrics like open rates, click-through rates, and ultimately, sales. Let’s dive in!
Table of Contents
What is A/B Testing?
Picture this: you’ve crafted the perfect email campaign, pouring your heart into every word, every image, and every call to action. You hit send with excitement and anticipation, eager to see the results flood in. But what if there was a way to make that email even better? That’s where A/B testing comes in.
A/B testing, also known as split testing, is the process of experimenting with different versions of your emails to see which one delivers better results. It’s all about comparison: sending out slightly different variations of an email to the same group of subscribers and then analyzing the results.
Let’s say you want to improve your email subject line. You could test two subject lines – maybe one’s fun and catchy, while the other is more straightforward. Then, you see which one gets more people to open your email. It’s like a little experiment to find out what your customers like best.
But it’s not just about subject lines. You can test anything in your emails, like the content in the email, the offers you make, or even when you send them.
With A/B testing, you’re always learning, improving, and fine-tuning your emails to get the best results possible.
Why A/B testing matters in email marketing
A/B testing isn’t just another fancy marketing buzzword – it’s a game-changer for businesses trying to nail their email marketing strategies. Let’s dive into why A/B testing is so important.
First, A/B testing gives you valuable insights into what’s working and what’s not. No more guessing games or relying on your gut feelings. With A/B testing, you’ve got solid data to back up your decisions. Whether you’re tweaking your subject line or changing the color of your call-to-action button, A/B testing helps you see what’s working with your customers and what’s not.
But A/B testing is more than just crunching numbers – it’s also a creative playground. Trying out different designs, wording, and colors might feel daunting, but it’s worth it. By experimenting and refining your approach, you’ll uncover clever ideas that hit home with your customers and get them converting.
And here’s the best part: A/B testing isn’t just a one-time benefit. Once you’ve figured out what works for your customers, you can use those insights to boost your email marketing long-term. Whether you’re changing your sign-up forms, sprucing up your landing pages, or promoting standout products, the data from your A/B tests is a gold mine you can keep using to drive results.
A/B testing isn’t just about making tiny tweaks here and there – it’s about giving your business the power to make smarter decisions that pay off. With the right tools and mindset, you’ll be amazed at what you can achieve with your email marketing.
Getting started with A/B testing
Don’t worry, diving into A/B testing isn’t as daunting as it sounds. Let’s break down the A/B testing process step by step so you can try out different versions of your emails without any stress. Plus, we’ll show you how Klaviyo’s A/B testing features make running tests even easier.
Set the goal of your A/B test
First, figure out what you want to achieve with your A/B test. Do you want more people to open your emails, click on links, or buy your products? Having a clear goal will guide your testing in the right direction and make it easier to track your progress. Once you’ve set your goal, we can move on to the fun part—testing different ideas to see what works best!
Choose the variable to test
Next, let’s choose what variable of your email you want to test. This could be anything from the subject line, layout, images, or buttons. Keep in mind, testing only one variable at a time is crucial so it’s easier to see what’s working. For example, if you choose to test the subject line, don’t test different call-to-action buttons at the same time. If you’re stumped on what to test, no worries, we’ve got some ideas coming up soon.
Split your email list for testing
Now that you’ve chosen what variable of your email to test, it’s time to split your email list into two groups. One group will receive version A (the original), while the other will get version B (the test variant). Don’t worry, Klaviyo makes this process simple – we’ll go through the steps of setting up A/B tests in Klaviyo campaigns and flows later in the article. Splitting your list ensures a fair comparison between the two versions and removes outside influences that could sway the results.
Track key metrics
During your A/B test, keep an eye on key metrics like open rates, click-through rates, and conversion rates. Klaviyo’s analytics tools make it easy to track these metrics in real time, allowing you to see how each version of your email is performing. With this information, you can make informed decisions to improve your email performance and engagement.
Choose a winner
After finishing your test, choose the winning version based on your goal metric and implement this version for your future emails. It’s essential to wait until you’ve gathered enough data for reliable results. Luckily, Klaviyo can handle this for you by automatically choosing a winner based on statistical significance. This helps you save time and make decisions backed by solid data.
Now that you understand the process of A/B testing, let’s explore what elements of your emails you should focus on testing.
What variables to A/B test to optimize your emails
Welcome to the fun part! We’re about to explore some variables you can A/B test to optimize your emails. Remember, you can test just about anything, but it’s important to test only one variable at a time to know exactly what drives the results.
Now, let’s dive into some common variables you can test. This list isn’t comprehensive, but it should spark some ideas:
- From name
- Subject line
- Preview text
- Email layout
- Images and visuals
- Text content
- Call-to-action (CTA)
- CTA button design
- CTA placement
- Send time
From name
When you test the “from name,” you’re essentially deciding who the email appears to come from. Because it’s one of the first things customers notice in their inbox, changing the “from name” can make a big difference in whether they open your email. The right “from name” helps build trust and credibility with customers.
Examples:
- Company name – “ABC Company”
- Specific person’s name – “Sarah from ABC Company” or “Sarah Smith”
- Specific department or role – “Sarah, Founder of ABC Company”
Subject line
The subject line is the first thing customers see when they receive your email, and it plays a major factor in whether they decide to open it. To increase your open rates, test different subject line variations to see what works best for your customers.
Examples:
- Length: Short vs. Long – “Limited Time Offer!” vs. “Discover Our Exclusive Limited Time Offer – Act Now!”
- Personalization: Personal vs. Generic – “Hey [Name], Check Out This Exclusive Deal!” vs. “Check Out This Exclusive Deal!”
- Emojis: With vs. Without – “🎉 Don’t Miss Out on Our Big Sale! 🎉” vs. “Don’t Miss Out on Our Big Sale”
Preview text
The preview text is like a teaser that appears alongside the subject line in the inbox, giving customers a quick preview of the email’s content. Testing different preview texts helps you discover what intrigues your customers the most, potentially increasing your open rates.
Examples:
- Length: Brief vs. Detailed – “Exclusive offer inside!” vs. “Unlock savings with our exclusive offer. Limited time only!”
- Tone: Casual vs. Urgent – “Hey there, take a look!” vs. “Act fast to claim your discount!”
- Call-to-action: Directly prompting action vs. Description – “Click to save now!” vs. “Learn more about this special offer!”
Email layout
Testing different email layouts can make a big difference in how customers interact with your emails. By experimenting with layout variations, you can improve readability and visual appeal, making it easier for your customers to engage with your content.
Examples:
- Single-column layout with minimal text and a prominent CTA button.
- Multi-column layout with more visual elements and embedded links.
- Image-heavy layout with minimal text for a visually engaging email.
Images and visuals
Adding images and visual elements can make your emails more appealing and engaging – they grab attention. By trying out different images, or adding videos to your email, you can see what clicks with your customers.
Examples:
- Product images showcasing different items or features
- Lifestyle images showing how the product is used
- Videos relevant to the product, like showcasing features or customer reviews
- Infographics relevant to the product or content
Text content
Your email’s text content is crucial—it’s how your brand speaks to your audience. By testing different wording and messaging, you can better tailor your emails to resonate with your customers, making them more engaging and effective. Whether you’re adjusting the tone, refining the length, or trying out different offers, each tweak can bring you closer to crafting emails that truly convert.
Examples:
- Clear and direct messaging highlighting benefits and creating urgency
- Storytelling approach showcasing customer testimonials or success stories
- Educational content like tips and guides
Call-to-action (CTA)
Your call-to-action (CTA) is like a signpost guiding customers toward the action you want them to take, such as making a purchase or signing up. Testing different CTAs, like tweaking the wording, color, or placement, can help you discover what works best with your audience, ultimately boosting conversion rates.
Examples:
- “Shop Now” vs. “Learn More”
- “Claim Your Discount” vs. “Claim Your 10% Discount”
CTA button design
When it comes to your call-to-action (CTA) button, its design can make a big difference in getting people to click. By trying out different button styles, colors, and sizes, you can figure out which ones result in more clicks. This way, you’ll make it easier for them to take action and engage with your email offers.
Examples:
- Button shape – rounded button vs. rectangular button
- Button color – bold color vs. your brand’s primary color
- Text only link with bold font and underlined text
CTA placement
Where you put your call-to-action (CTA) button in your email can affect whether people notice it and click on it. By testing out different spots, like at the top, middle, or bottom of your email, you can see which placement gets the most clicks and helps you achieve your goals.
Examples:
- Above the fold at the top of the email
- Immediately after the product or offer
- At the end of the email
- Inline CTAs within the email text
Send time
Finding the perfect time to send your emails can significantly boost their effectiveness. By testing different send times—such as mornings, afternoons, or evenings—you can determine when your audience is most likely to engage with your emails, leading to better open rates and increased conversions.
Examples:
- Sending at different times during the day
- Sending on different days of the week
- Changing the frequency of emails
How to A/B test emails in Klaviyo
Now that we’ve got a handle on what tweaks can help make your emails better, let’s try it out with A/B testing in Klaviyo. It’s easier than you think, and we’ll break it down step by step. From testing different versions of your emails to using Klaviyo’s helpful testing features, we’ve got you covered. So, let’s optimize your campaign and flow emails with some A/B tests!
How to A/B test campaign emails
First, let’s look at how to set up A/B tests for your campaign emails. In this example, we’ll set up a subject line A/B test to see which subject line leads to more opens.
- In Klaviyo, click “Campaigns” in the left-hand menu. Select an existing campaign or click the “Create campaign” button. If you’re creating a new campaign, enter the campaign details.
- On the Campaign content page, enter the subject line, preview text, sender name, and sender email address. Next, choose the “Drag and drop” option to design your email and select a template.
- Next, create your email by adding your copy, images, and links. Then click “Save.”
- You should now be back on the Campaign content page. At the bottom of the page, click “Create A/B test.”
- This creates a second variation of your email and takes you to the Campaign A/B test page.
- In the “Variation Name” field for each variation, type in a name that describes what you’re A/B testing. In this example, we’re testing whether including the discount percentage in the subject line results in more opens.
- Under Variation B, enter the subject line you want to test. In this example, we’re testing the subject line “Spring into Savings: Save up to 25% Now!” Remember, it’s important to test only one variable at a time, so don’t change anything else.
- Next, choose the metric you want to use to determine a winner. In this case, we want to know if our subject line leads to more opens, so we choose “Open rate.”
- Finally, click “Continue to review” to review and schedule your campaign.
All done! We’ve set up an A/B test in our Klaviyo campaign to see if adding the discount percentage to our subject line improves the open rate.
Before we dive into A/B testing in Klaviyo flows, let’s walk through a few tips for A/B testing your campaigns.
Cloning variations
If you’re feeling adventurous, you can create multiple variations to test. It’s not something we suggest when starting with A/B testing, but we thought we’d mention it in case you’re feeling bold!
Remember, we’re only testing one variable at a time, but if you want to test multiple versions of your variable, you can do that. For example, we could clone a variation and test a third subject line that includes both the discount percentage and emojis.
To clone variations, click the “Clone” button on the Campaign A/B test page. If you decide to go down this route, we recommend sticking with a maximum of 3 variations.
Testing send times
To test different send times, click the “Switch to testing send times” link at the top right of the page. When you’re testing send times, keep everything else about your emails exactly the same – the content and subject lines of both variations should be identical. The only difference will be the time the email is sent.
Winning metric
When choosing the winning metric, select the metric you want to see improve. In the example above, we chose open rate because we were curious if adding the discount percentage in the subject line would get more customers to open the email.
You have three options to choose from:
- Open rate – This is great for testing variables like the subject line, preview text, sender name, or sender email address.
- Click rate – Choose this if you’re testing content inside the email, like the layout, images, or your call-to-action button placement.
- Placed order rate – This is only available for accounts with a Placed Order metric. It’s recommended to test variables in your email content that may lead to more people making a purchase. For example, when testing whether featuring best sellers or new items leads to more conversions.
How to A/B test flow emails
Now, let’s look at how to set up A/B tests for your flow emails. In this example, we’ll again set up a subject line A/B test to see which subject line leads to more opens for our abandoned cart flow emails.
- In Klaviyo, click “Flows” in the left-hand menu. Select an existing flow or click the “Create flow” button.
- On the flow builder page, click on the individual email you want to A/B test. If you haven’t already, set up the content for the email.
- Click “Create A/B test” in the Email details pane on the right side of the page.
- This creates a second variation of your email and takes you to the A/B Test tab on the Message content page.
- In the “Name” field for each variation, type in a name that describes what you’re A/B testing. In this example, we’re testing whether including a personalized subject line results in more opens.
- Under Variation B, enter the subject line you want to test. In this example, we’re testing the subject line “Hey {{ first_name }}, you left something behind…”. Remember, it’s important to test only one variable at a time, so don’t change anything else.
- Next, we’ll tell Klaviyo how to choose the winner. By default, tests automatically choose a winner based on click rate. In our case, we’re curious if the personalized subject line improves the open rate, so we’ll need to change the winner selection metric. Click the “Configure” link under winner selection.
NOTE: Generally, it’s best to stick with the default setting of using the click rate to pick the winner, even if you’re testing the open rate, like we did here. That’s because several factors can lead to inflated open rates, such as Apple Mail Privacy Protection (MPP), email preview panes, and even bots. We changed the winner selection option in this example just to show you how it’s done, but sticking with the click rate is best.
- In the winner selection settings modal, click “Open rate.” Then, click the “Save settings” button.
- Finally, click “Publish test.”
That’s it! We’ve kicked off an A/B test in our Klaviyo flow to find out if adding a personal touch to the subject line helps bump up our email open rate.
9 Best practice tips for A/B testing Klaviyo emails
Now that we know how to set up A/B tests in Klaviyo campaigns and flows, let’s discuss some best practice tips to help you get the most out of your tests. These tips will help you fine-tune your A/B testing strategy to make your emails more effective.
1. Develop a hypothesis
Before you run an A/B test, take a moment to think about what you want to happen. This is your hypothesis – it’s your guess about what might work best, and it will help you choose which variable to test. For example, “offering a percentage discount instead of a flat dollar amount discount will drive more sales.”
2. Test one variable at a time
We’ve talked about this already, but it’s worth repeating. When running A/B tests, stick to testing one variable at a time. Whether it’s the subject line, email content, CTA button, or design, limiting your tests to one variable lets you know exactly what is creating results.
3. Test with a large sample size
Make sure your test group is large enough to give reliable results. Your data might not tell you much if you test with a small group. For instance, deciding on a winning subject line after testing with just 20 people won’t give you a clear picture. So, aim for a good-sized group to make sure your findings mean something.
4. Give your tests enough time to run
Don’t rush your tests. We just talked about how you need enough people in your test group to be sure your results are accurate. Luckily, Klaviyo’s automatic winner selection takes care of this tip for you. It’s okay to check how your test is going as it runs but hold off on making any email changes until you have a clear winner.
5. Test low-effort variables first
Start by A/B testing the easy stuff to score some quick wins. Test variables like the subject line, sender name, using emojis, and the CTA copy. These are simple tweaks that could make a big difference in how many people open and click your emails. Save the higher-effort tests, like changing the email layout and copy, for later once you’ve optimized the basics.
6. Focus on the most important emails
Similarly, start by focusing your efforts on the emails that bring in the most money or reach the most people. This way, you’ll see a quick return on your testing efforts. For instance, you might start by testing your abandoned cart emails to try and capture lost sales.
7. Limit tests to 3 variations
Test no more than 3 variations at a time. Keeping it simple ensures that each variation gets sent to enough people for better results. If you have more than three ideas to test, run a few separate tests. For example, test Variation 1 against Variation 2. If Variation 1 wins, pit it against Variation 3 in another test. Keep testing the winning version against your other ideas until you have a clear winner.
8. Preview and test emails across devices
Make sure your emails look and perform correctly on all devices. With Klaviyo, it’s easy to check how your emails appear on desktop and mobile devices. Don’t skip this step – it’s important to test your emails across devices and email clients before sending them out. Make it a habit to preview and test your emails before launching any campaigns or flows.
9. Think about your audience
Remember, not everyone in your audience is the same. When setting up your tests, always think about what your target audience likes. Split your email list into different segments and run tests for each segment separately to truly optimize your emails. For instance, if you have two segments, frequent shoppers and occasional buyers, run different tests for each group to see what they respond to.
By following these best practices in your Klaviyo A/B tests, you’ll get accurate results to optimize your email campaigns and flows. With a clear understanding of what works with your audience, you’ll be ready to drive significant improvements across all the key email marketing metrics.
Final thoughts
A/B testing your emails is a powerful way to help your emails stand out in your customers’ inboxes. It’s all about testing different variables and using data to optimize how your emails perform. Now that you know how to create A/B tests in Klaviyo, you’re ready to run tests on your emails to improve open rates, click-through rates, and most importantly, sales.
Remember, it’s all about continuously testing and refining your emails to connect with your customers and get them to take action, like making a purchase.
Happy testing!
Mo Sheikh is an entrepreneur, developer, and marketer. As the Founder of Adaptyve Digital, he helps teach ecommerce business owners ways to manage, grow, and scale their business.
