# Streams · Neural Inverse Govern > Durable, ordered streams on Govern's own partitioned log, with consumer groups, compaction, transactions, DLP, classification and schemas, and endpoints that give producers and consumers their own governed way in. Source: https://neuralinverse.com/govern/streams/ --- ![](/photos/facility.jpg) [Neural Inverse Govern / Streams](https://neuralinverse.com/govern/) # Event streams that check what is written and who may read it. Durable, ordered streams on Govern's own partitioned log, with consumer groups, compaction, transactions, DLP, classification and schemas, and endpoints that give producers and consumers their own governed way in. Runs on Govern's own log, no external system Durability Local disk, replicated or object storage Delivery At least once, transactions all or nothing Governance DLP, classification and schemas on write Endpoints IAM, JWT, custom, CEL and mTLS authorizers Overview Streams is a managed stream service: named, durable logs of records split into partitions, written by producers and read by consumer groups that track their position. Its brokers keep each partition on their own volume and coordinate through PostgreSQL, so it needs no external system and works air-gapped. Records are checked as they are written, against DLP, the stream's schema and its classification, and endpoints give outside producers and consumers a hostname with their own authorizers and protection. [What it does ↓](https://neuralinverse.com/#capabilities)[How it works ↓](https://neuralinverse.com/#workflow)[Enforced and recorded ↓](https://neuralinverse.com/#enforcement)[Where it runs ↓](https://neuralinverse.com/#where)[Get started ↓](https://neuralinverse.com/#get-started) What it does ## What Streams provides. The stream service and its endpoints, as they run today. - **Streams and partitions:** Create streams with partitions, retention, a record size limit, classification and tags. Records with the same key keep their order; partitions can be added, with a warning that keys are remapped. - **Health and durability:** Health is computed on the server from the partitions' leaders and lists its reasons. Durability is local disk with fsync, replicated across zones, or acknowledged only once in object storage. - **Write and read:** Write batches of records and read by offset through the leader broker, with per-minute traffic metrics. Every write action honours an idempotency key, so a retried request is not written twice. - **Consumer groups:** Committed offsets per partition with a generation: a reset moves the group on, and a commit from an older generation is refused. Lag per partition is computed on the server; resets go to earliest, latest or a time. - **Managed membership:** Consumers join a group and are assigned a balanced, sticky share of the partitions. Members that stop heartbeating are removed and the group rebalances; delivery is at least once. - **Schema registry:** A stream can require a JSON Schema subject; records that do not match its latest version are refused. Backward, forward and full compatibility are checked conservatively: an incompatible change is never accepted. - **DLP on produce:** Allow, redact or block per stream, with the account's secret, personal and health data patterns and custom patterns. If the patterns cannot be loaded, nothing is written. - **Classification ceilings:** Every record carries the classification it was written under. Reading confidential or restricted records needs its own permission, and records above the reader's level are withheld. - **Data explorer:** Browse records from the console by offset or time, redacted by DLP before they are shown. Binary values are never sent to the browser. - **Compaction:** Delete, compact or both: keep the latest record of every key, with tombstones to remove a key and a minimum lag before compaction. - **Transactions:** Write up to 500 records to up to ten streams as one transaction under a transactional ID. Readers see all of it or none of it, and a newer writer fences an older one. - **Endpoints:** A hostname of its own per endpoint, serving only the data actions, streams and classification it allows. Custom domains need a domain verified for the account and are served by SNI with certificates reloaded without a restart. - **Endpoint authorizers:** Govern IAM, JWT from your identity provider, your own HTTP decision service, CEL expressions or client certificates. An open endpoint is refused unless a rate-limited usage plan covers it. - **Usage plans and protection:** Rates, bursts, request sizes and daily or monthly quotas per caller, held across replicas, plus address allow and deny lists and temporary blocks for request spikes and credential stuffing. - **AI governance of payloads:** Off, monitor or block per endpoint, with detectors for prompt injection, sensitive data and secrets. In block mode, sensitive data can be replaced with tokens instead; secrets are always refused. - **Encryption and receipts:** Record values sealed in segments under the account's managed key or your own key, refused while the key is revoked. Each batch gets a signed receipt in a Merkle proof log that can be verified offline. How it works ## How a record written through an endpoint is checked. Each check runs before the record is stored; any refusal stops it. 1. Authorize The endpoint's authorizer decides the caller; the decision is audited. 2. Limit The caller's usage plan and the endpoint's protection apply. 3. Govern AI governance inspects the payload, then the stream's DLP and schema check it. 4. Seal The value is sealed under the account's key when the endpoint encrypts. 5. Store The leader writes it with the stream's durability. 6. Receipt The batch receives a signed receipt in the proof log. Who uses it ## Built for every role that answers for AI. - **Platform engineers:** Run the brokers, create streams and set durability, retention and quotas. - **Application developers:** Produce and consume through the API, the Go client or an endpoint, with consumer groups and transactions. - **Security teams:** Set classification, DLP and AI governance, and choose authorizers and encryption for endpoints. - **Auditors:** Verify batch receipts offline and read every control-plane change and authorizer decision. Enforced and recorded ## Checked as records are written and read. Governance is enforced on the server and fails closed. Enforced - **DLP and schemas:** A record that breaks a block policy or its schema is refused; when DLP cannot scan, nothing of the request is written. - **Readers:** Confidential and restricted records need their own read permission; records above the reader's level are withheld. - **Endpoints:** Every request passes an authorizer, a usage plan and protection; non-IAM callers stay within the endpoint's actions, streams and classification. - **Keys:** While the account's stream key is revoked, writing and reading encrypted records is refused on every path. Recorded - **Control plane:** Stream, consumer group, schema, tag and endpoint changes, in the transaction of the change. - **Authorizer decisions:** Each denial with its reason and source address; allows aggregated per caller and minute. - **Data plane:** Records are counted per minute and metered rather than audited one by one, as stream services do. Where it runs ## Wherever Govern runs, with no external system. [Deployment options](https://neuralinverse.com/deployment/) - **Self-hosted:** Brokers run as part of your installation and coordinate through its PostgreSQL. They are off until you deploy them. - **Air-gapped:** Object durability and tiered storage can use a local directory instead of a bucket. - **Hosted by us:** Streams in the region that serves your account, with limits in service quotas. Get started ## Set it up with the documentation. - [Kubernetes deployment](https://neuralinverse.com/docs/deploy/kubernetes/) - [Air-gapped installation](https://neuralinverse.com/docs/deploy/air-gapped/) - [Govern concepts](https://neuralinverse.com/docs/govern/) What's next ## Being built now. These are in development and not part of the product today. - **Endpoint hostnames in DNS:** Publishing each endpoint's hostname through Govern's own DNS service instead of a wildcard record. Related [AI Gateway → Approved models, DLP, guardrails, budgets and rate limits for every model request.](https://neuralinverse.com/govern/ai-gateway/)[DNS → Authoritative DNS with public and private zones, routing policies, health checks and DNSSEC.](https://neuralinverse.com/govern/dns/)[Audit and evidence → A hash-chained audit log you can verify offline, and evidence toward your frameworks.](https://neuralinverse.com/govern/audit/) ![](/photos/team.jpg) ## Talk to our team. Neural Inverse is available by request. We work with your engineering, security and procurement teams from the first conversation. [Request access](https://neuralinverse.com/contact/?topic=access)[Contact us](https://neuralinverse.com/contact/)