AB Test API Version 2.0 - Documentation

AB Test API Version 2.0 - Docs

Bayesian Chance to Beat into Google Sheets

Introduction

The Bayesian API at abtestapi.io provides an endpoint that returns the Bayesian "Chance to Beat" scores for A/B/n tests. The primary use case is integrating Bayesian results into Google Sheets (using the "IMPORTDATA" function). By default the output with be returned in .csv (download) format, since this is required for the Google Sheet ImportData function.

JSON formatted output for other uses is also possible by using the 'output=json' parameter.

Access / usage limits

Access is granted by using a company-specific token that is provided to you by Online Dialogue. Please note after March 3rd 2020, requests without a valid token will no longer work.



Input

Calling the url 'https://abtestapi.io/bayes', followed by required query parameter token with your company-specific token, and parameters a & b (required) for users and conversion in the control group (separated by '|' pipe symbol). Optionally additional variations can be added using c, d, ... j parameters.

Output

The url request will return a .csv formatted result, which can be used in the import function in Google Sheets.
The output will show a chance between 0 and 1 for each of the variants, with 4 digits accuracy. The number of samples and output format can be configured (see API Reference).

Examples

Title Description
URL (json) https://abtestapi.io/bayes?token=[YourTokenHere]&a=3010|153&b=3020|153&output=json
URL (csv) https://abtestapi.io/bayes?token=[YourTokenHere]&a=3010|153&b=3020|153
Google Sheets Formula =IMPORTDATA("https://abtestapi.io/bayes?token=[YourTokenHere]&a="&B2&"|"&C2&"&b="&B3&"|"&C3)
Output options Google Sheet showing various output options.

API Reference

Parameter Type Options
token required Unique company-specific token, provided to you by info@onlinedialogue.com.
Please note after March 3rd 2020, requests without a valid token will no longer work.
a required Users & Conversions in A: Control, separated by a pipe '|' symbol.
For example for 3010 users with 153 conversions: a=3010|153
b required Users & Conversions in B: Variant, separated by a pipe '|' symbol.
For example: b=3020|164
c, d, e .. j optional Additional variants for AB Test with more than 2 variants. Same format as a, b.
output optional full (default) - all output options enabled (Variant, Bayesian Score, 95% Credible interval (Low95, High95).
singleScore - Only the Bayesian Score of variant B is returned (A/B-test only).
scores - Return scores for all variants
scoresInterval - Return scores & credible intervals for all variants
json - Return all output in json format with (as opposed to .csv which is otherwise returned)

For more information on output options, see this example sheet.
samples optional Number of samples to use for the Monte Carlo Bayesian inference. Default = 50000. The higher the number, the longer processing time it will take. Maximum 5 million samples is allowed.