ROADMAP
The technical roadmap of OpenMake LLM, driven by community feedback.
Development Phases
Local Gateway Hardening
Moving the runtime onto a self-hosted vLLM + LiteLLM stack and making it predictable
- vLLM + LiteLLM MigrationOllama fully removed; the local model is served by vLLM behind a LiteLLM proxy on one OpenAI-compatible endpoint
- Context-Fit Safety NetPrompt tokens estimated on entry; past the 262K window input truncates, max_tokens shrinks, and overflow returns 413 with an audit record and webhook alert
- Cluster Health & Circuit BreakerCLOSED/OPEN/HALF_OPEN node states with latency-sorted failover across gateway nodes
- Per-User Token QuotasHourly and weekly token budgets moved from a global counter to per-user calendar buckets
- Public RoadmapPublic roadmap page to ensure community trust
One Path, Many Models
Collapsing the strategy layer and replacing presets with orthogonal controls
- Single Execution PathLocal and external models share one dispatch with an always-on MCP tool loop; the per-strategy layer and the LLM classifier were retired
- Orthogonal User ControlsBehavior is set by Model, Style, Mode toggles, and Custom Agent instead of opaque brand presets
- Role-Based Model OrchestrationEight functional roles resolve through one registry: per-user mapping → admin global → env → local default, fail-open
- Next.js FrontendThe Vanilla JS SPA was replaced by a Next.js 16 + React 19 workspace with a 4-language UI
- Thinking DisplayReasoning renders as a live timeline with a generated one-line headline, persisted and restored on re-open
Agent Autonomy & Tool Governance
Autonomous runs, artifacts, and safely isolated tools
- Agent Tasks in a Docker SandboxMulti-turn autonomous runs with shell, Python, browser, and file tools, human-in-the-loop approval, and honest non-achievement reporting
- Templates & Recurring SchedulesSave a task definition for reuse or put it on a schedule; admins can review every scheduled run
- MCP Sandbox HardeningExternal MCP servers, task sandboxes, and artifact execution all run with --cap-drop ALL, non-root, memory+swap caps, and realpath-guarded mounts
- Artifacts & PublishingSandboxed iframe rendering, optional Docker code execution, and a separate-origin strict-CSP shared viewer
- Procedural Skills & MemoryReplayable skills captured from successful runs, plus automatic semantic memory formation across conversations
Operations Readiness
Documentation, admin controls, and compliance for self-hosted teams
- Multi-Tenant SupportIsolated environments per organization or team
- SSO / SAML AuthenticationEnterprise authentication integration (Okta, Azure AD, etc.)
- SLA Monitoring DashboardDashboard for response times, local runtime availability, and tool error rates
- On-premise GPU Scale-OutOperational guidance for larger local model gateway deployments
- Compliance FrameworkSOC2 and GDPR compliance frameworks
Runtime / Capability Matrix
The self-hosted components OpenMake LLM runs on today.
| Component | Version | Type | Status | Note |
|---|---|---|---|---|
| vLLM + LiteLLM Gateway | Shipped | runtime | COMPLETE | vLLM serve behind a LiteLLM proxy — the primary self-hosted OpenAI-compatible endpoint set by LLM_BASE_URL |
| Local Model Catalog | Shipped | model | COMPLETE | Qwen 3.6 35B-A3B chat (262K, vision + tool calling), BGE-M3 embeddings, FLUX.2-klein-9B image generation, and an OpenAI-compatible alias |
| Role-Based Orchestration | Shipped | model | COMPLETE | Eight roles — chat, agent, judge, research, spawn, review, router, summary — resolved per user or org-wide, fail-open to the local default |
| OpenRouter | Shipped | optional | COMPLETE | BYO key (sk-or-) at openrouter.ai/api/v1 — one endpoint for 300+ models including GPT-5, Claude Opus 4.5, Gemini 2.5 Pro, and DeepSeek R1 |
| Ollama (Local) | Shipped | optional | COMPLETE | Your own Ollama server on :11434/v1 — no key needed, and pulled models are listed automatically; LAN hosts require an SSRF_ALLOWED_HOSTS entry |
| Ollama Cloud | Shipped | optional | COMPLETE | BYO key at ollama.com/v1 for hosted large models — DeepSeek V3.1 671B, GPT-OSS 120B, and Qwen3 Coder 480B |
| NVIDIA NIM | Shipped | optional | COMPLETE | BYO key (nvapi-) at integrate.api.nvidia.com/v1 — Llama 4 Maverick, Qwen3 Next 80B A3B, and Mistral Nemotron on NVIDIA GPU cloud |
| External Key Management | Shipped | optional | COMPLETE | One key per provider per user, AES-256-GCM encrypted and validated on save; operators can add a server-wide key with a mandatory daily token cap |
| PostgreSQL 16 | Shipped | data | COMPLETE | Raw parameterized SQL, no ORM — conversations, users, audit logs, usage, skills, MCP metadata, agent tasks, and artifacts |
| MCP Tools | Shipped | tool | COMPLETE | 22 built-in tools available to every user, plus Docker-isolated external MCP servers with audit-aware execution |
| Agent Tasks | Shipped | tool | COMPLETE | Autonomous multi-turn runs in a persistent Docker sandbox, with approval gates, templates, and recurring schedules |
| Artifacts | Shipped | tool | COMPLETE | Sandboxed iframe rendering, optional Docker code execution, and a strict-CSP separate-origin shared viewer |
| Deep Research | Shipped | tool | COMPLETE | Fan-out search → source fetch → claim verification → cited synthesis, at quick / standard / deep depth |
| Web Search | Shipped | optional | COMPLETE | Google Custom Search only when GOOGLE_API_KEY and GOOGLE_CSE_ID are configured; scraping tools need no key |
| Google · GitHub · Kakao OAuth | Shipped | optional | COMPLETE | Optional sign-in integrations; local email auth and guest sessions remain available |
| NotebookLM Grounding | Shipped | optional | COMPLETE | Pin one of your Google NotebookLM notebooks as conversation context, using your own session cookie |
| Discord Gateway Bot | Shipped | optional | COMPLETE | Optional standalone workspace relaying Discord messages to /api/v1/chat/completions with role and mention access control |
| OpenAI-Compatible API | Shipped | tool | COMPLETE | /api/v1 chat completions, streaming, tool_calls, artifacts extension, API keys, usage, and model listing |
| Local GPU Scale-Out | v2.0 | runtime | FUTURE | Operational guidance for larger self-hosted model gateway deployments |
vLLM + LiteLLM Gateway
runtimevLLM serve behind a LiteLLM proxy — the primary self-hosted OpenAI-compatible endpoint set by LLM_BASE_URL
Local Model Catalog
modelQwen 3.6 35B-A3B chat (262K, vision + tool calling), BGE-M3 embeddings, FLUX.2-klein-9B image generation, and an OpenAI-compatible alias
Role-Based Orchestration
modelEight roles — chat, agent, judge, research, spawn, review, router, summary — resolved per user or org-wide, fail-open to the local default
OpenRouter
optionalBYO key (sk-or-) at openrouter.ai/api/v1 — one endpoint for 300+ models including GPT-5, Claude Opus 4.5, Gemini 2.5 Pro, and DeepSeek R1
Ollama (Local)
optionalYour own Ollama server on :11434/v1 — no key needed, and pulled models are listed automatically; LAN hosts require an SSRF_ALLOWED_HOSTS entry
Ollama Cloud
optionalBYO key at ollama.com/v1 for hosted large models — DeepSeek V3.1 671B, GPT-OSS 120B, and Qwen3 Coder 480B
NVIDIA NIM
optionalBYO key (nvapi-) at integrate.api.nvidia.com/v1 — Llama 4 Maverick, Qwen3 Next 80B A3B, and Mistral Nemotron on NVIDIA GPU cloud
External Key Management
optionalOne key per provider per user, AES-256-GCM encrypted and validated on save; operators can add a server-wide key with a mandatory daily token cap
PostgreSQL 16
dataRaw parameterized SQL, no ORM — conversations, users, audit logs, usage, skills, MCP metadata, agent tasks, and artifacts
MCP Tools
tool22 built-in tools available to every user, plus Docker-isolated external MCP servers with audit-aware execution
Agent Tasks
toolAutonomous multi-turn runs in a persistent Docker sandbox, with approval gates, templates, and recurring schedules
Artifacts
toolSandboxed iframe rendering, optional Docker code execution, and a strict-CSP separate-origin shared viewer
Deep Research
toolFan-out search → source fetch → claim verification → cited synthesis, at quick / standard / deep depth
Web Search
optionalGoogle Custom Search only when GOOGLE_API_KEY and GOOGLE_CSE_ID are configured; scraping tools need no key
Google · GitHub · Kakao OAuth
optionalOptional sign-in integrations; local email auth and guest sessions remain available
NotebookLM Grounding
optionalPin one of your Google NotebookLM notebooks as conversation context, using your own session cookie
Discord Gateway Bot
optionalOptional standalone workspace relaying Discord messages to /api/v1/chat/completions with role and mention access control
OpenAI-Compatible API
tool/api/v1 chat completions, streaming, tool_calls, artifacts extension, API keys, usage, and model listing
Local GPU Scale-Out
runtimeOperational guidance for larger self-hosted model gateway deployments
Shape the Future
We build together. Your feedback and contributions are always welcome!