TECHNOLOGY

The platform behind time‑based AI.

We own the full path from GPU to API: we host and operate every model on GPUs we run ourselves, behind one gateway. We measure every layer before we sell it, so capacity can be sold as time instead of tokens.

Talk to us Read the explainer

01 — The platform, layer by layer

One gateway in front of a fleet we run.

The gateway handles keys, sessions, admission and queueing, and routing, including automatic model choice per conversation. It speaks OpenAI Chat Completions, OpenAI Responses and Anthropic Messages, so existing tools work by changing a base URL and a key. Prompts are not stored, only metadata.

Diagram: the platform from your tools to the GPU fleet

We never proxy to another inference provider and never resell capacity. Frontier proprietary models stay with their labs.

02 — Measured before it is sold

Capacity is a measurement, not a guess.

Our benchmark harness replays real coding-agent sessions against the endpoint at rising concurrency: multi-turn, with tool calls, with contexts of 50,000 to 130,000 tokens and users running parallel sessions.

03 — Where the load limit sits

The knee sets how many users a GPU serves.

The harness finds the point where speed starts to fall, and sizes how many users and sessions each GPU serves.

  • Real agent traffic, not synthetic prompts
  • Seven GPU classes measured
  • Speed floor held for 9 in 10 requests

Time to first token is measured alongside decode speed, across two open-source serving engines. The same harness can also drive the MLCommons MLPerf Agentic Inference client.

04 — Serving at scale

Four ways to split a model across GPUs.

We serve open-weight models on open-source serving engines, and use the techniques we measure: parallelism across GPUs, speculative decoding, prefix (KV) cache reuse and long contexts.

  • Data parallel

    The whole model is copied on each GPU and requests are spread across the copies.

  • Tensor parallel

    Each layer is split across GPUs, and they work on every layer together.

  • Pipeline parallel

    Groups of layers are staged across GPUs, and each request flows from one stage to the next.

  • Expert parallel

    Different experts of a mixture-of-experts model live on different GPUs, and each token goes to the experts it needs.

Read the full explainer

Our explainer, “How one prompt travels through a giant AI model”, covers data, tensor, pipeline and expert parallelism and attention designs, with every claim linked to its source.

05 — Research in the open

Research, published in the open.

We collaborate with Token By Token, an independent inference lab. It runs open model-serving engines under recorded load and publishes the results one claim at a time: what a user waits for (time to first token, time per output token), what the hardware does, and where the evidence stops.

Recorded so far

  1. Episode 00Warm-up
  2. Episode 01Measuring what matters
  3. Episode 02An equal-work runtime baseline
  4. Episode 03Prefix reuse
  5. Episode 04Speculative decoding

Planned (twelve)

  • Mixture-of-experts
  • Scheduling and precision
  • Parallelism within and across nodes
  • Prefill/decode disaggregation and cache-aware routing
  • Slurm and Kubernetes orchestration
  • Saturation, SLOs and cost
  • Release gates and recovery, and more
Visit Token By Token

06 — Operated, not just deployed

Run with the MIRASTACK operations stack.

Hardware-to-business observability, explainable root cause analysis and human-approved remediation keep the fleet healthy. It runs air-gapped when it has to.

See Day 2 operations

Talk to us

Tell us what you need to run.

Every agreement is shaped to the workload: seats, sessions, capacity, term and location.

Talk to us hello@mirastacklabs.ai