CaYaDev AI Hub

Yayın Tarihi: 2026-08-21

<div align="center"> <img src="assets/icon.png" width="112" alt="CaYaDev AI Hub"> CaYaDev AI Hub Capable coding agents on local Ollama models — including the small ones. <img src="docs/media/chat.png" width="880" alt="A session in progress: reasoning folded away, two tool calls, and a markdown reply with syntax-coloured code"> </div> --- The problem Point a general-purpose agent tool at a local model and one of two things usually happens: the session dies before any work starts, or it crawls. Both have specific, fixable causes, and neither is really "the model is too small". | Symptom | Actual cause | | --- | --- | | Session ends immediately | The model named a tool that does not exist, and the runner read that as an answer | | "No tool calls" on a tool-capable model | The model emitted bare JSON in the message body, and a strict parser saw prose | | Says hello, then takes a screenshot | Twenty tools on offer and nothing telling the model when not to use one | | Hangs, then dies | A total request timeout fired during a legitimately slow prefill | | Painfully slow | A model too large for available VRAM spilled to system RAM | Every row is measured, and every fix is verified against a real model. See docs/BENCHMARKS.md and docs/SPEED.md. What makes it fast Measured on the development machine — an 8 GB laptop GPU: | Model | Decode | TTFT | Placement | | --- | ---: | ---: | --- | | qwen3.8 27B Q4 K M | 5.4 tok/s | 1.5 s | 80% CPU / 20% GPU | | qwen2.5-coder:7b | 42.3 tok/s | 0.40 s | 100% GPU | | qwen2.5-coder:0.5b | 316.3 tok/s | 0.19 s | 100% GPU | A 7B model that fits in VRAM decodes 8x faster than a 27B that does not. No sampling flag closes that gap — context size was measured across a 16x range on the 27B and every result landed within noise of 5 tok/s. So the core feature is task-based routing : planning goes to the strongest model that will actually fit, edits go to the fast resident coder, and the constant yes/no decisions go to a 0.5B that answers in 200 ms....

Özellikler

Tolerant tool-call parsing — recovers calls from <tool_call> tags, fenced
Invented tool names are corrected rather than treated as an answer
Loop detection that forgets what it saw whenever the run changes something,
Schema validation, and a step ceiling sized to bound an unattended run
Idle-based timeouts, so a slow prefill is never mistaken for a hang
Stop actually stops: the flag is polled before every chunk, and it also
The conversation is trimmed before it is sent, from the middle, keeping the
Earlier turns are replayed, with each tool call folded to one summary line, so
The agent asks rather than guessing: ask_user offers choices or a written
Task-based routing across reasoning / coding / edit / classify / vision
GPU memory read from the driver, not assumed
Byte-stable prompt prefixes — a reused prefix prefills 14.8x faster