Listing your products on different marketing channels is almost rudimentary in today’s ecommerce landscape. The real competitive edge comes from optimizing your listings for each channel, and leveraging your data to identify ways that you can improve your selling strategies. A/B testing is one of the most powerful methods you can use to find opportunities for improved performance, and Feedonomics gives you the ability to organize and transform your data so that your tests are as effective as possible.
What is A/B Testing?
A/B testing, also known as split testing or A/B split testing, is frequently used in web design and marketing. By testing a variable and comparing the performance of the changed version to the unchanged version, it’s possible to conclude which version is more effective.
For example, a marketing team might try different strategies to increase the open rates for its emails. Improving subject lines could help with this, so the marketing team would create multiple subject lines for the same email and send each variation to an equal portion of the mailing list. After a while, the team should have enough data to determine which subject line performs best.
Feedonomics enables A/B testing on your product listings. You can make deliberate and methodical changes to the product data in your ecommerce feed and identify ways to improve the performance of your ads or sales. You can test changes in your titles, descriptions, and images, gain clarity about your brand recognition, and learn if certain product types respond to changes more positively than other product types. The ultimate goal is to yield better results on your product listings.
Why is A/B testing valuable?
A/B testing is valuable because it helps you make data-driven decisions. Rather than relying on pure intuition or anecdotes, you can use A/B testing to gather data that supports or refutes a hypothesis. In addition to getting some definitive answers in your testing, the additional trends you notice along the way can provide valuable insights for further testing. With the information you gather from an A/B test, you can implement meaningful changes in a scalable way.
Some examples of A/B testing include finding which ad copy works better improve return on ad spend for PPC campaigns, or finding which image will generate more sales.
How does Feedonomics help with A/B testing?
Feedonomics is a valuable A/B testing tool for a number of reasons, including optimizing your product feeds for different channels. Feedonomics can categorize your products in a granular way and combine data from multiple sources into a single database. Having all of this data organized in one place gives you the ability to methodically transform your product listings with a high level of control.
The sky’s the limit when it comes to the different types of changes you can make to your product data for A/B testing. We’ve identified some of the most useful and common A/B tests below.
Gauge your brand recognition.
If you’re unsure about how effective it is to include your brand name in product titles, or how much brand recognition you have, you can run a test to learn more. Deciding whether to put your brand name at the beginning or end of your titles can be determined with A/B testing.
Big companies like Levi or Nike benefit from including brand names at the beginning of their product titles because their customers often include those brands in the search query.
Unlike some designer brands, you might find that shoppers click on your products more often when you show your product nouns (“denim jacket,” “cable-knit sweater”) at the beginning of your titles and move your brand name to the end. Google Shopping does not weigh words at the front of your titles more heavily than words at the end, but you still want the most important words at the front because customers read them first. Your goal is to get people to click on what they want.
“Remember, the point of A/B testing is to figure out what works best for you. Because Feedonomics categorizes your products at a granular level, you may even discover that some of your product types respond to changes differently than other product types. Hats with your brand name at the beginning of their titles might do better than pants, which would be valuable to know.”
Sales Engineer at Feedonomics
Nail down the language and messaging in your product descriptions.
Your product descriptions can add to your search relevancy, and they can also increase the likelihood of a conversion. Testing different versions of your description copy can give you insights about what to include in the text. Customers may respond to messaging about promotions, Black Friday, or your product assortment in general. As long as you can provide the source material for multiple versions of your product descriptions, Feedonomics can map the new descriptions to some products and use the old descriptions for other products. We would then apply labels to each group so you could monitor the results of the test.
Determine what kind of information to include in your titles.
If you’re listing products on Google Shopping, it’s recommended that you include any relevant information—like brand, material, color, and size—somewhere in your title. It’s not necessary for every business to include the manufacturer’s part number (MPN) in their product titles, but for hardware stores and original equipment manufacturers it could be useful because customers who need to replace or repair appliances might search by specific part numbers. It’s worth testing.
Test the performance of different images and product angles.
On comparison shopping engines like Google Shopping or Microsoft Shopping, your main product image shows the product by itself on a white background. However, you still might want to see which angle of your product is more appealing to consumers. A side view of a shoe might not perform as well as a three-quarters angle.
When it comes to ads on social commerce channels, like Facebook, Instagram, Pinterest, and others, there’s more leeway for the types of main product images you use. Test out some lifestyle images and see if people respond to more dynamic pictures.
What is the process for A/B testing?
There are six main steps for an effective A/B test.
1. Form a hypothesis.
Decide what you want to test and keep it simple. To run an A/B test, you only need to change one variable. Test one thing at a time so you can easily analyze cause and effect.
2. Gather data to have a point of comparison.
Before beginning an A/B test, you should gather data to know the current state of your performance metrics. For example, if you want to see how your conversion rate is impacted when you make changes to your product descriptions, you need to track the conversion rate before you make any changes in order to have a point of comparison.
3. Split your products into groups.
When you split your products into groups, it’s very important to avoid bias that could skew your results. Your groups should be random and representative. It’s possible to do this manually, but it’s difficult to get a truly random sample. It’s also time consuming to create your groups manually, especially if you have thousands of SKUs. Feedonomics is able to easily split products into many random and representative groups, though we don’t recommend running tests on more than four groups at a time.
Feedonomics is capable of limiting the test to the scope of your choosing. You may choose to test with a single product type, many product types, or even the entire product feed.
Once you split your products into groups, label the groups so you can track their performance. You can use custom labels in the feed to achieve this. For example, if you’re testing out the placement of your brand name within the title, you would apply a custom label like “brand first” to one group, and a custom label like “brand last” to the other group. Then you would be able to track the performance of these two groups.
4. Run the test.
Remember to stay focused on the objective and keep everything the same in your campaign setup for your two groups. Make sure you can attribute any performance differences to the change you made in the product listing and no other variables.
Ideally, allow the test to run for four to six weeks. Enough time must pass for you to have an impression load that carries any statistical significance.
5. Analyze the results.
Once you’ve run your test long enough, you can analyze the results to draw some conclusions. Compare the A/B groups to determine the impact of your change.
You can also use the granular product categorization in your feed to identify other trends. For example, you may discover that vests performed well with your changes, jackets stayed the same, and belts performed worse. There’s a lot of value in seeing these nuances, because you can implement changes strategically, instead of applying changes to the entire catalog and negatively impacting some product types.
6. Implement your changes.
Now that you’ve completed your test and determined if the A group or B group performed better, you can apply the changes from the successful group to the other group. Keep an eye out for any product types that should be excluded from the changes.
What are some challenges with A/B testing?
Here are some of Aubry’s tips for running effective A/B tests:
- To run an A/B test correctly, you need enough time to accumulate good data, and you need a sample size that is big enough to allow you to draw meaningful conclusions.
- You can’t do true A/B testing on most shopping channels because they don’t allow you to list the same product twice. A true A/B test would split traffic between two versions of the same product. The next best thing is to split your products into random and representative groups for comparison.
- Promotional periods and holidays are usually not a good time for A/B testing. It’s best to run tests during stable seasons, when consumer behavior is not as volatile and there’s less outside noise that could skew your results.
Feedonomics is a powerful, full-service feed management solution, and there’s a lot you can do with it to take your product listings to the next level. Beyond optimizing your data according to best practices and using granular category taxonomy to improve your search relevancy, Feedonomics enables you to split test different strategies and identify scalable improvements. Best of all, our specialists take care of the heavy lifting on the feed side. Book a demo today to see how Feedonomics can help your business.
Brian Roizen is the Cofounder and Chief Architect of Feedonomics, a full-service feed optimization platform that optimizes product data for hundreds of channels. He has been featured on numerous podcasts and eCommerce webinars, and regularly contributes to Search Engine Land and other industry-leading blogs. Brian graduated summa cum laude from UCLA with both a Bachelor’s and Master’s degree in Mechanical Engineering.
