[ Infrastructure ]

Systems at scale.

Serving, batching, isolation, and training systems research for production models.

Papers: 6
Latest: 2025-09-25
Area: Infrastructure

Focus areas

Serving

GQA, continuous batching, canaries, and cache hit rates.

Enterprise

Isolation patterns for dedicated and VPC deployments.

Publications

6 papers in this area

2025-09-25Infrastructure

Enterprise Isolation Patterns for Model Serving

Tenant isolation, VPC patterns, and audit logging considerations for IdonAI enterprise deployments.

Read →

2024-08-29Infrastructure

Continuous Batching Notes — Fairness, Prefill, and Cache Hits

Practical serving research: scheduling fairness between short chat and long prefill jobs on shared Origin pools.

Read →

2024-05-02Experiments

Long-Context Mid-Training — From 4K to 128K and Toward 1M

Curriculum and position-extension methods that enabled Origin 128K and paved the way for NextGen 1M.

Read →

2023-12-05Infrastructure

Origin Serving Canaries — Latency, Batching, and Failure Budgets

Operational research on validating Origin under load before Quark promotion: SLOs, canaries, and rollback.

Read →

2023-05-18Infrastructure

Grouped-Query Attention for Interactive Serving

Why Origin adopted GQA: KV-cache memory, batchability, and quality tradeoffs measured on internal canaries.

Read →

2023-02-14Datasets

Data Quality Filtering v1 — Provenance, Dedup, and Toxicity Screens

First production-oriented data pipeline for IdonAI pretraining: filter architecture, near-dedup strategy, toxicity screens, PII heuristics, and measured impact on training quality versus unfiltered baselines.

Read →

[ Research ]

Explore the full archive.

Browse every area, or email the research team about collaborations, datasets, and eval access.

All research Email research