laloop designs and runs self-hosted RAG and LLM systems. Below is the assistant that answers Se Liga na SKY participants on WhatsApp: embeddings, vector search, a reranker and a local model, with every answer traceable to its source.
Conversation translated from Brazilian Portuguese; scores are illustrative. Parameters match the production code.
Five containers in one docker-compose. The API only enqueues; the worker serialises access to the GPU. Knowledge lives in a separate repository and is reindexed without rebuilding the image.
Vector search buys cheap recall; the cross-encoder makes the call. Skills and documents run on separate tracks with their own cut-offs and thresholds.
Context budget 12,000 characters in the system prompt; a skill is cut in half if it exceeds that.
History last 4 messages per session_id, in Postgres, expiring after 30 minutes idle.
Query current message concatenated with the user's previous message.
Output sources: [{type, name, chunk, score}] on every answer.
The reranker returns sigmoid(logit). When it sees neither relevance nor contradiction the logit sits near zero and the score near 0.50. With a 0.50 cut-off, "what is the capital of Brazil?" got through with irrelevant context and the model made things up. At 0.55 it falls into the deterministic path.
If no skill or document clears the threshold, grounded=False and the answer is a fixed string. The LLM is never called: the guarantee doesn't depend on the model obeying the prompt, and off-topic questions don't burn GPU time.
"It says my password is wrong but I can't remember it" isn't similar enough to any skill on its own. Concatenated with "I can't log in to the portal", it lands on login-bloqueado. No conversational search or LLM query rewriting needed.
The GPU serves one generation at a time. The API returns 202 with a job_id immediately and the channel polls; the worker drains the queue serially, with graceful shutdown on SIGTERM. More throughput means more workers on more GPUs.
The model associated SKY with the TV operator and suggested other carriers in login questions. The prompt now states, in capitals, that this is an incentive and points programme.
Skills change with operations, not with code. They live in seliga-rag-content, mounted as a volume; make ingest-skills reindexes and the next query already uses the new vector. Descriptions stay in Portuguese, because that's what gets embedded.
The frontmatter drives retrieval; the body becomes instructions for the model. The description carries the words participants actually type. Shown in Portuguese, as in production.
skills/antecipar-boleto/SKILL.md
# Antecipar boleto
## Quando usar
- "quero pagar antes", "adiantar o boleto"
## Procedimento
1. Pedido até 3 dias úteis antes
do vencimento, antes das 12h.
...Asynchronous, authenticated with X-API-Key. The channel never blocks waiting for the GPU.
POST /chat
{ : "c1f4…", : "esqueci a senha" }
202 { "job_id": "550e…", "status": "queued" }
GET /chat/status/550e…
200 {
"status": "done",
"answer": "Sem problema! Na tela…",
"sources": [
{ "type": "skill",
"name": "login-bloqueado",
"score": 0.91 }
]
}
DELETE /chat/{session_id} 204
GET /health 200| Layer | Choice | Why |
|---|---|---|
| Embeddings | BAAI/bge-m3 | Multilingual and strong in Portuguese, 1024 dimensions, runs locally via sentence-transformers. |
| Reranker | BAAI/bge-reranker-v2-m3 | Cross-encoder from the same family; scores the question–passage pair, not just distance. |
| Vectors | Postgres 16 + pgvector, HNSW | One database for vectors, sessions and metadata. No extra service to operate. |
| LLM | Qwen2.5 14B · Ollama | Follows instructions far better than 4B models; OpenAI-compatible API, so swapping models is one variable. |
| Queue | Redis 7 | Serialises GPU access; jobs expire after one hour. |
| API | FastAPI + Uvicorn | Small contract, OpenAPI docs at /docs. |
| Ingestion | docling, markitdown | PDF, DOCX, PPTX and XLSX become markdown, cached by mtime, before chunking. |
| Integration | MCP server (FastMCP) | The same index, queryable from Claude, Cursor and other MCP clients. |
| Deploy | Docker Compose | Same compose on a Mac (LM Studio) and on the server; the GPU comes in through an override file. |
An incentive and points programme for employees. Questions arrive on WhatsApp and go straight to the API: portal access, missing points, redemption options and deadlines, prepaid cards, bill payments.
Skills were written from real support conversations, using the human agent's tone as the reference. The operations team reads reports built on the sources trail: which skills fire most, which never fire, where refusals run high.
The first thing the reports showed: access is the topic of people locked out. Blocked logins and forgotten passwords come precisely from those who can't get in, and that became a new first-access skill.
Fixed-scope projects or engagement by scope. From prototype to a monitored system in production, with code and documentation handed over to your team.
Tell us the problem, the data and where it needs to run. We'll come back with an architecture and scope proposal.
Email contato@laloop.ai