The Bayesian API at abtestapi.io provides an endpoint that returns the Bayesian "Chance to Beat" scores for A/B/n tests.
The primary use case is integrating Bayesian results into Google Sheets (using the "IMPORTDATA" function). By default the output with be returned in .csv (download) format, since this is required for the Google Sheet ImportData function.
JSON formatted output for other uses is also possible by using the 'output=json' parameter.
Access is granted by using a company-specific token that is provided to you by Online Dialogue. Please note requests without a valid token will no longer work.
Calling the url 'https://abtestapi.io/bayes', followed by required query parameter token with your company-specific token, and parameters a & b (required) for users and conversion in the control group (separated by '|' pipe symbol). Optionally additional variations can be added using c, d, ... j parameters.
The url request will return a .csv formatted result, which can be used in the import function in Google Sheets.
The output will show a chance between 0 and 1 for each of the variants, with 4 digits accuracy. The number of samples and output format can be configured (see API Reference).
| Title | Description |
|---|---|
| URL (json) | https://abtestapi.io/bayes?token=[YourTokenHere]&a=3010|153&b=3020|153&output=json |
| URL (csv) | https://abtestapi.io/bayes?token=[YourTokenHere]&a=3010|153&b=3020|153 |
| Google Sheets Formula | =IMPORTDATA("https://abtestapi.io/bayes?token=[YourTokenHere]&a="&B2&"|"&C2&"&b="&B3&"|"&C3) |
| Output options | Google Sheet showing various output options. |
| Parameter | Type | Options |
|---|---|---|
| token | required | Unique company-specific token, provided to you by [email protected].
Please note requests without a valid token will no longer work. |
| a | required | Users & Conversions in A: Control, separated by a pipe '|' symbol.
For example for 3010 users with 153 conversions: a=3010|153 |
| b | required | Users & Conversions in B: Variant, separated by a pipe '|' symbol. For example: b=3020|164 |
| c, d, e .. j | optional | Additional variants for AB Test with more than 2 variants. Same format as a, b. |
| output | optional | For information on all output options, please refer to this example sheet. |
| decSep | optional | Decimal separator, 'comma' or 'dot' (default='dot'). When 'comma' is selected, the csv is suitable for import in a 'European' locale Google Sheets that uses comma as decimal separator. The csv delimiter then changes to semicolon; beware to use the undocumented second argument of ";" in the IMPORTDATA function. |
| samples | optional | Number of samples to use for the Monte Carlo Bayesian inference. Default = 50000. The higher the number, the longer processing time it will take. Maximum 5 million samples is allowed. |
| header | optional | true or false. Show a header row with column names. |
| intervalSignificance | optional | Significance of the confidence interval of the difference. Default = 0.95 |
| srmThreshold | optional | Threshold (p-value) for the Sample Ratio Mismatch check. Default = 0.00005 |
| Business Case feature (only applicable when Business Case results are required) | ||
| aov | required | Average Order Value. Required only when outputing Business Case results. |
| duration | required | Duration of the Test in days |
| percInTest | optional | Percentage traffic in test. Default 100% |
| effectDuration | optional | How long uplift effect of variation will sustain, in months. Default 6 months. |
| Parameter | Type | Options |
|---|---|---|
| token | required | Unique company-specific token, provided to you by [email protected].
Please note requests without a valid token will no longer work. |
| traffic | required | Weekly total number of users that are exposed to the control or variation. |
| conversions | required | Weekly total number of users with a conversion, that are exposed to the control or variation. |
| power | optional | Statistical Power (decimal '.' dot-separated). Default: 0.8 |
| sign | optional | Statistical Significance (decimal '.' dot-separated). Default: 0.95 |
| sides | optional | Two ('2') or one ('1') sided test. Default: 1 |
| weeks | optional | Number of weeks to return results for. Default: 1 |