CASE STUDY · PRODUCTS & PLATFORMS · AGENTIC AI

Building an agentic AI engine: Exybit AI

Every product I ship wants AI capabilities. Rather than wiring each one to a model vendor separately, I built the layer they all talk to: Exybit AI, my own agentic engine, consumed as an API or driven through its own terminal client. It's in active development, and it's already running production systems.

Why build an AI engine at all

The naive way to add AI to a product is to call a vendor API directly from application code. Do that across several products and you've scattered the same hard problems (conversation state, caching, credentials, configuration) into every codebase, each solved slightly differently and none solved well.

I wanted to own the layer between my applications and the model. So I built Exybit AI (ai.exybit.org): a single engine where the agentic behavior, the caching, the sessions, and the operational plumbing live in one place I control. Every product I ship gets its AI capabilities from that one service. When the engine improves, everything downstream improves with it, and no application ever needs to know or care what sits underneath.

Architecture: a NestJS service around a proprietary agentic core

The engine is a NestJS service wrapping my proprietary agentic core, built on a frontier language model. The core is where the agentic behavior lives; the NestJS layer around it is deliberate, boring engineering, the same discipline I bring to any backend: typed modules, clear boundaries between transport, orchestration, and state, and infrastructure concerns solved once instead of per-client.

That separation is the whole point of the design. Clients speak stable protocols to the engine; the engine speaks to the core; the core speaks to the model. Each layer can evolve without breaking the ones above it.

Semantic response caching: pgvector + fastembed

Caching AI responses by exact string is nearly useless. No two people phrase a question identically, so an exact-match cache almost never hits. Exybit AI caches semantically instead: incoming requests are embedded with fastembed, and the vectors are stored and searched with pgvector inside Postgres. A new request that lands close enough to a previously answered one can be served from the cache rather than a fresh model call.

Keeping the vector store in Postgres was a deliberate choice: it means the cache lives in infrastructure I already operate and back up, not in yet another specialized database. It's one more example of the engine absorbing a hard problem, meaning-aware caching, so that no client application ever has to think about it.

Sessions, tokens, and config that survive production

Agentic conversations are stateful, and state is where AI demos go to die. The engine treats it as a first-class engineering problem:

• Redis-backed session persistence. Conversations survive restarts and redeploys instead of living in process memory. A client can reconnect and pick up where it left off.
• OAuth token refresh. The engine manages its upstream credentials itself, refreshing tokens before they expire so a long-running service doesn't fall over on authentication.
• Hot-reloadable configuration. Behavior can be retuned on a live engine without a restart, which matters when production systems are talking to it around the clock.

None of this is glamorous. All of it is the difference between an AI prototype and an AI system.

Three surfaces: API, MCP server, terminal client

The engine is dual-surface by design. The primary surface is a REST/WebSocket API: REST for request/response work, WebSocket for streaming conversation. Alongside it runs an MCP (Model Context Protocol) server for device management, exposing the engine's capabilities to any MCP-speaking client over the emerging standard for agent tooling.

Then there's the client I use every day: @exybit/cli, a TUI chat client that talks to the engine over WebSocket, published on npm. It's the fastest way to work with the engine directly from a terminal, and shipping it publicly keeps me honest: the API has to be clean enough for a standalone client to consume, because one does.

In production today

Exybit AI is in active development, and it's already earning its keep. It powers two production systems right now:

• AIDA, my self-extending personal assistant. Its entire intelligence layer is a consumer of this engine.
• The chat assistant on exybit.org itself. The console you can open on this very site is a live client of the engine, answering visitors in production.

That's the arrangement I wanted from the start: one engine, multiple real workloads, each one exercising the sessions, the cache, and the streaming path daily.

What this proves

Anyone can call a model API. Building the engine layer (agentic orchestration, semantic caching, durable sessions, credential lifecycle, multiple client protocols) is custom software engineering, and it's the same discipline I apply to every system I build at Exybit, a custom software solutions practice in Cebu, Philippines. When I tell a client I can put AI at the core of their operations, this is the receipt: I built the core itself, and I run my own products on it.