splitchAlpha

Built for coding agents

Control what ships. Learn what works.

Feature flags and A/B experiments your coding agent can create, promote, and measure end to end.

claude mcp add --transport http splitch https://mcp.splitch.dev
# Sign in in your browser when prompted (or run /mcp). No key to copy.
Set up a feature flagRead the agent guide
Your agent, in the dev Environment
splitch use --app my-app --env devsplitch flags create --key new-checkout --variants on,offsplitch flag-config update new-checkout \
    --enabled true --rollout 100splitch flags verify new-checkout \
    --targeting-key test-user-1 --json{"value":true,"variantName":"on","reason":"SPLIT"}

Start with a switch. Ask a question later.

A Flag turns a feature on per Environment. To learn whether it helped, start an Experiment on the same Flag. The evaluate call you already ship records the Exposures it counts.

const d = await splitch.evaluateDetails("new-model", {
  targetingKey: userId,
});
if (d.reason === "ERROR") renderFallback(d.errorCode);
else respondWith(d.value);
As a Flag
Change the rollout without redeploying. Each Environment has its own configuration, so dev and prod move independently.
As an Experiment
The same call records an Exposure for each user, so results count exactly who saw each Variant.
Model comparisonExample experiment

Control

Current model

The model you use today

Treatment

Candidate model

A cheaper model to try

Can a cheaper model compete? Collect user feedback and compare the results before deciding.

Refusals your agent can act on.

When a change is gated, your agent gets a stable code and a pending request it can review.

Prod asks first
New Apps gate prod changes behind a confirmation. Dev applies immediately.
The fix is in the error
CLI errors come back as JSON with a stable code, remediation guidance, and a docs link, so your agent can act without guessing.
Your permissions, no more
Your agent signs in as you. The CLI and the MCP server share one typed client, so both see the same answers.

Browse the error catalog

Promoting the Flag to prod for the first time
splitch flags promote new-checkout --env prod \
    --from-environment-id env_dc3de58178eb7361e9f9319e \
    --body-json '{"select":{"enabled":true,"rollout":true}}' \
    --idempotency-key promote-1 --json{
  "code": "APPROVAL_REVIEW_REQUIRED",
  "message": "Approval Request is pending Review",
  "remediation": "Review Approval Request apr_01KWHB6FG0J93EMPZAJ0MK0MJ7 (splitch approval-requests get apr_01KWHB6FG0J93EMPZAJ0MK0MJ7), or rerun the same command with --confirm if you hold approver rights.",
  "docsUrl": "https://splitch.dev/docs/error/APPROVAL_REVIEW_REQUIRED",
  "details": {
    "approvalRequestId": "apr_01KWHB6FG0J93EMPZAJ0MK0MJ7",
    "status": "pending",
    "policyContexts": [
      {
        "environmentId": "env_21e75f3cd8224ed3a99b4868",
        "changeTypes": ["enabled_state", "targeting_rollout_value"],
        "level": "confirm"
      }
    ],
    "recommendedAction": "REVIEW_APPROVAL_REQUEST"
  },
  "outcome": "user_action_required"
}splitch flags promote new-checkout --env prod \
    --from-environment-id env_dc3de58178eb7361e9f9319e \
    --body-json '{"select":{"enabled":true,"rollout":true}}' \
    --idempotency-key promote-1 --confirmsplitch flags verify new-checkout --env prod \
    --targeting-key test-user-1 --json{"value":true,"variantName":"on","reason":"SPLIT"}

Statistical rigor

See the difference, and the uncertainty.

An Experiment can show an improvement, a regression, or an inconclusive result. splitch tells you which, with the interval behind it.

  • Look whenever you like

    Sequential, always-valid tests are the default, so checking a running Experiment does not inflate false positives.

  • Count who actually saw it

    The denominator is deduplicated first Exposures, computed from the raw event log rather than a rollup.

  • Catch a broken split

    A sample ratio mismatch check flags uneven traffic before you trust the comparison.

  • Tighter intervals

    CUPED and winsorization run by default when the data supports them, with variance computed per randomization unit.

We use it ourselves. In Neuron, one of our own apps, splitch compares a newer, cheaper model against the current one using user feedback. An inconclusive result does not mean the models perform equally, so we keep collecting.

Put your next change behind a Flag.

The quickstart goes from install to a verified Flag and checks every step.

Set up a feature flagRead the CLI guide

Why splitch? Split testing and feature switches, fused into one word.