# Agent orchestrator

Python service providing AI agent chat, MCP tools integration, and RAG on CRM documents, based on FastAPI and LangChain. Called by the Worker's Consumer processes via HTTP/SSE.

---

### System Requirements

`llama-cpp-python` is installed as a **pre-built wheel** (not compiled from source). `requirements.txt` specifies the **Vulkan** variant via `--extra-index-url https://abetlen.github.io/llama-cpp-python/whl/vulkan`. Other backends are available by changing the index URL:

<table id="bkmrk-variant-table"><tbody><tr><th>Backend</th><th>Index URL suffix</th></tr><tr><td>cpu</td><td>`.../whl/cpu`</td></tr><tr><td>vulkan **(default)**</td><td>`.../whl/vulkan`</td></tr><tr><td>cuda</td><td>`.../whl/cuda`</td></tr><tr><td>rocm</td><td>`.../whl/rocm`</td></tr><tr><td>metal</td><td>`.../whl/metal`</td></tr><tr><td>sycl</td><td>`.../whl/sycl`</td></tr></tbody></table>

Docker image installs `libvulkan1`. GPU access (`/dev/dri`) is commented out in `compose.yaml` by default — uncomment for hardware acceleration. Falls back to CPU without GPU.

The x86-64-v2 baseline or equivalent is required by NumPy's pre-built wheels (see [NumPy SIMD build options](https://numpy.org/doc/stable/reference/simd/build-options.html)). CPUs without these instructions can still run the orchestrator by **recompiling NumPy from source** with reduced SIMD flags (`NPY_DISABLE_CPU_FEATURES`), or by using a distro that ships a compatible build.

---

### Architecture Summary

Three layers:

1. **PHP CRM** (Worker Consumer), calls orchestrator via HTTP
2. **Python orchestrator** (FastAPI, Docker)
3. **LLM / MCP servers / Chroma**

Relevant Consumer methods: `llmChat()` (chat relay), `elaborateRag()` (document upload + vector build).

---

### RAG Document Building

#### Indexing (Elaboration)

Triggered on agent save with the Documents feature enabled. The Worker Consumer:

1. Reads selected CRM Documents
2. `POST /rag/keep` — prunes stale docs from orchestrator
3. `POST /rag/upload` — uploads new/changed files (multipart), stored as `docs/shared/<md5>.<ext>`, symlinked into `docs/<agent_id>/`
4. `POST /rag/build` — indexes all docs for the agent into Chroma at `vectors/<agent_id>/`.

#### Querying (Runtime)

With `rag: true` in `/agent/run`, Python injects a `query_documents` tool. The LLM decides when to call it. The orchestrator decomposes the question into ≤3 sub-questions (`needs_retrieval` flag), queries Chroma per sub-question, deduplicates by `doc_id`, re-ranks with FlashRank, and returns context. The LLM answer is grounded strictly in retrieved context.

---

### Python Orchestrator Endpoints

The Python service exposes the following REST endpoints. All are mounted on the FastAPI app at port 8120.

<table id="bkmrk-endpoint-table"><tbody><tr><th>Endpoint</th><th>Description</th></tr><tr><td>`POST /agent/run`</td><td>Agent loop: LLM + MCP tools + guardrails + optional RAG. SSE or JSON.</td></tr><tr><td>`POST /tools/inspect`</td><td>Introspect MCP server tools.</td></tr><tr><td>`POST /rag/build`</td><td>Index documents for an `agent_id` into Chroma.</td></tr><tr><td>`POST /rag/run`</td><td>Query vector store with question decomposition.</td></tr><tr><td>`POST /rag/keep`</td><td>Prune agent's doc symlinks to match `{filename: md5}`.</td></tr><tr><td>`POST /rag/upload?agent_id=`</td><td>Upload file to shared pool + symlink into agent's dir.</td></tr></tbody></table>

---

### Installation &amp; Configuration

#### Docker Setup

The Python orchestrator runs in Docker. Quick start:

```bash
cd plugins/agent
docker compose up -d --build
```

Verify: `curl http://localhost:8120/docs` should show the Swagger UI.

**Port** `127.0.0.1:8120:8120` — bound to localhost only, **MUST NOT** be publicly exposed.

#### Environment Variables

<table id="bkmrk-env-table"><tbody><tr><th>Variable</th><th>Default</th></tr><tr><td>`HOST` (inside the container)</td><td>`0.0.0.0`</td></tr><tr><td>`PORT`</td><td>`8120`</td></tr><tr><td>`EMBED_MODEL`</td><td>`nomic-ai/nomic-embed-text-v2-moe-GGUF:Q8_0`</td></tr><tr><td>`RERANK_MODEL`</td><td>`ms-marco-MiniLM-L-12-v2`</td></tr><tr><td>`HF_CACHE_DIR`</td><td>`/app/hf_cache`</td></tr><tr><td>`DOCUMENTS_DIR`</td><td>`/app/docs`</td></tr><tr><td>`VECTORS_DIR`</td><td>`/app/vectors`</td></tr></tbody></table>

---

### Troubleshooting

- **Startup slow:** Embedding model downloads from HuggingFace on first container start — cached in `HF_CACHE_DIR` afterwards.
- **Empty docs:** `/rag/build` raises `RuntimeError` if `docs/<agent_id>/` is empty.
- **GPU not used:** Uncomment `/dev/dri` in `compose.yaml`. Falls back to CPU otherwise.
- **CPU compat:** Verify with `/lib64/ld-linux-x86-64.so.2 --help`

---

### File Reference

```mysql
plugins/agent/
├── compose.yaml
├── app.Dockerfile
├── requirements.txt
├── src/vte_agent/
│   ├── __main__.py
│   ├── config.py
│   ├── schemas.py
│   ├── agent.py              # /agent/run, /tools/inspect, calculator + rag tools
│   ├── rag.py                # /rag/* endpoints
│   ├── docs.py               # doc loaders
│   ├── models.py             # GGUFEmbeddings
│   ├── user_manual.py        # builtin vtenext user manual search tool
│   └── utils.py
├── docs/          
│   ├── shared/        # <md5>.<ext> — deduplicated by content hash
│   └── <agent_id>/    # symlinks → ../shared/<md5>.<ext>
└── vectors/
    └── <agent_id>/    # chroma.sqlite3, parent_docs.json, description.txt

cache_local/
└── huggingface/       # local models cache (embedding, rerank)
```