Split Test Analysis
Split Test Analysis Overview
Split Test Analysis allows for the comparison of a control and test set of pages based on either Page Tag groups, specific pages or Content Type patterns. The comparison results provide a correlation analysis on the impact of the test including a confidence level.
See how easy it is to set up, validate, run and evaluate SEO tests with seoClarity Split Test Analysis:
Background & Requirements
Once the inputs for: Metric, Type, Test, and Implementation Date are selected via the Test Details section, select the Analyze Correlation button to view the correlation analysis and the subsequent Analyze Impact button to view the impact analysis. Metric selection is based on Google Search Console and/or analytics integration.
- For a split test analysis to run, there needs to be data available for the selected metric in the pre and post period of the date the test was implemented.
- Wait 24 hours before running the analysis after a new tag is created.
Split Test Analysis Use Cases
- Split testing helps eliminate the guesswork from site changes and allows for data-backed decisions to be made
Split Test Analysis
Located via the Tools Dropdown:
Test Details
Metrics: Select from Search Analytics metrics for Clicks, Impressions, CTR, or from Site Analytics metrics for Traffic or any other integrated analytics metric.
Type: Select the type of comparison from Page Tag groups, specific pages or Content Type patterns.
Test: Select the Page Tag, specific pages or Content Type patterns for the Test group. This displays a list based on the type selected.
Control: Select the Page Tag, specific pages or Content Type patterns for the Control group. This displays a list based on the type selected.
Implementation Test Date: Select the date the test was implemented.
Pre-period to evaluate: This will use the previous 60 days prior to the date the test was implemented.
Post-period to evaluate: This will use up to the following 60 days after the date the test was implemented.
Analyze Correlation
Control: Displays the name of the selected page tag, page(s), or content type for the Control group.
Correlation %: Displays a correlative percentage range from 0-100% for the Control group.
Correlation Strength: Displays a text based representation of the Correlation percentage.
Use in Analysis: Displays a checkbox selected by default that enables the Control group to be used in the analysis.
Analyze Impact
Result: Displays the result of the test as Positive or Negative based on the test inputs. Positive or negative results indicate if performance improved (for the variable) compared to the control group. It is important to take into consideration whether or not there is enough data to be statistically significant.
Statistically Significant: Displays whether the test is statistically significant as Yes or No based on the test inputs. Yes would indicate the probability is less than or equal to 0.05 (95%). This can be calculated as: (1 - p-value)*100
Relative Effect: Displays the numerical value of relative effect based on the test inputs. Relative difference provides the scale of difference.
Absolute Effect: Displays the numerical value of absolute effect based on the test inputs. Absolute difference provides the literal number difference.
Confidence Level: Displays the confidence level as a percentage based on the test inputs. How likely would the same results be the outcome if the test was repeated. Confidence level of 95% would be an indication that it is very likely to get the same results if done again.
Original: This displays a trend of the predicted and actual values, with upper and lower bounds available when hovering over a selected date, based on the test inputs. Predicated values are based on Bayesian Time Series models, using the pre-period data for the test and both pre and post period for the control. This estimates what the post-period data for the test should be.
Pointwise: This displays a trend of the difference between predicted and actual values, with upper and lower bounds available when hovering over a selected date, based on the test inputs.
Cumulative: This displays a trend of the cumulative impact, with upper and lower bounds available when hovering over a selected date, based on the test inputs.
Overview: The final section below the trend charts provides an overview of the split test analysis based on the test inputs.
Related Articles
Split Test Analysis Overview (Video)
See how easy it is to set up, validate, run and evaluate SEO tests with seoClarity Split Test Analysis:ClarityAutomate settings
Overview ClarityAutomate is an add-on feature that assists with implementing critical fixes, deploying new enhancements, and SEO testing to your site. The ClarityAutomate settings is where you designate publishers and approvers, enable QA/DEV ...Split Tester (enabled feature)
Split Tester Overview Split Tester enables the dynamic optimization of content in any web page. It can be used to replace or update content in the HTML of the page for a given location. Background & Requirements Once this feature is enabled for an ...Internal Links Analysis
Internal Links Analysis Overview Internal Links Analysis is a summary of the internal (own domain) linking data based on a Site Audit crawl. Internal Links data takes up to 24 hours to process once a crawl is completed and is based on indexable (200 ...Why Isn't my Internal Link Analysis data Showing after a Site Audit Crawl?
If you've completed a Site Audit crawl and your Internal Link Analysis data isn't appearing, don't worry! Here's what you need to know: Internal Links data can take up to 24 hours to process after a crawl is completed; the analysis is based on pages ...