> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tensormesh.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Welcome to Tensormesh Platform

> Self-hosted KV caching for your vLLM fleet on Kubernetes.

**Tensormesh Platform** runs a KV-cache engine on your own Kubernetes cluster to cut
time-to-first-token (TTFT) and free GPU memory. It installs as a Kubernetes operator: a single
controller manages a fleet of cache engines next to your vLLM pods, so repeated and long-context
prompts are served from cache instead of recomputed on the GPU — and that cache survives across
requests, pods, and optionally nodes.

> **[Getting Started](/v1.0.1/installation/getting-started)**
>
> Preflight your cluster and pick an install method.

> **[End-to-End Example](/v1.0.1/installation/example)**
>
> Install → run inference → benchmark the cache win.

## Benefits

> **Faster prefills**
>
> Cached prefixes skip recomputation, so warm requests return their first token sooner.

> **More GPU headroom**
>
> KV cache lives in CPU memory and on disk, freeing HBM for active batches.

> **Reuse that persists**
>
> Cache outlives a single request — across pods, restarts, and optionally across nodes.

> **Kubernetes-native**
>
> One operator, standard CRs, Helm/GitOps install, and metrics through Prometheus.

## How caching tiers fit together

Each tier is larger and cheaper than the one above it; the engine promotes hot data upward and
spills cold data downward:

| Tier | Where it lives | Configured by | Best for |
| - | - | - | - |
| **GPU HBM (L0)** | vLLM's own paged KV cache | vLLM | The active batch |
| **CPU offloading (L1)** | Host DRAM (system RAM) | [`engine.spec.l1.sizeGB`](/v1.0.1/configuration/cpu-offloading) | Fast warm reuse beyond what fits on the GPU |
| **External storage (L2)** | Local disk, NFS, or a remote store | [`engine.spec.l2Backend`](/v1.0.1/configuration/filesystem-offloading) | Large working sets and reuse across restarts/nodes |

Start with **[CPU offloading](/v1.0.1/configuration/cpu-offloading)**; it's on by default. Add
**[external storage](/v1.0.1/configuration/filesystem-offloading)** when your working set outgrows RAM or
must survive restarts.

## What gets deployed

> **Operator**
>
> A controller-manager Deployment that watches the Tensormesh CRs cluster-wide and reconciles
> them into running cache infrastructure.

> **Cache engine**
>
> A DaemonSet (one cache-engine pod per GPU node) that stores and serves KV cache to the
> vLLM pods on that node.

> **Coordinator**
>
> A fleet-wide service the engines register with, enabling cross-node cache awareness.

> **Observability**
>
> An optional OpenTelemetry stack that ships cache metrics and traces to Prometheus and Tempo.

## Explore the docs

> **[Install with Helm](/v1.0.1/installation/helm)**
>
> The primary install path, plus upgrades and rollbacks.

> **[CPU offloading](/v1.0.1/configuration/cpu-offloading)**
>
> The default host-DRAM cache tier.

> **[Metrics](/v1.0.1/observability/metrics)**
>
> The signals that show whether the cache is reducing recompute.

> **[Operator UI](/v1.0.1/ui/introduction)**
>
> A console for monitoring your cache fleet, thresholds, and alerts.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.