AI Tinkerers #16: Vector DB Evolution & Persistent Agent Memory

Adib Mohsin from Imperative Machines presented The art of inferencing everywhere, showing how to embed language models natively into apps and websites offline. He showcased Chyral, a privacy-first browser with a built-in small LM as a web-browsing companion, and shared tricks to wire these capabilities into apps. The approach emphasizes on-device security and low latency. Survey feedback suggested it resonated with builders seeking private, fast AI, pointing to practical paths for production-ready local pipelines.

Mir Sakib from Imperative Machines presented Building a Persistent Memory & Stateful Second Brain AI Agent, showcasing Slatekore, starter kit giving Gemini CLI persistent memory via Obsidian as storage layer. It demonstrates management without infrastructure, with the filesystem as memory and knowledge base, plus context engineering through GEMINI.md prompts and workflow files. Tools fire via language commands backed by structured templates to manage files and build knowledge graphs. Survey whispers noted its appeal for builders, hinting at a practical takeaway.

Rach Pradhan presented EmergentDB, a vector database that auto-evolves its index configurations with MAP-Elites to optimize recall, latency, and memory. The system maintains a quality-diversity grid and can switch between HNSW, Flat, and IVF indices with evolved hyperparameters, using Gemini embeddings for benchmarking on a Rust core with SIMD optimizations. Audience responses hinted at its practical appeal, showing how this approach can cut manual tuning time and scale AI infrastructure. For builders, it’s a glimpse of auto-tuning powered by evolution.

Gabriella Hachem, Cofounder at Dessn.ai, presented Design directly in prod, showing how Dessn starts from a codebase to create a visual design environment around it. The demo lets non-developers build in the codebase with zero setup cost, using design-language extraction to prototype in production React code, and it runs components in ephemeral microVMs with no data retention. Builders saw a practical path to production-ready design workflows, backed by user testimonials. It matters because it lowers setup time and speeds design-to-code.

Asyrique T from Vase.ai, the engineering lead, presented How to build good skills for LLMs. The talk presents rules of thumb for crafting skills that keep LLMs useful without flooding the context, drawing on Anthropic guidance and hands-on experience. Audience feedback suggested the practical focus resonated, and the approach could become a production toolkit for teams. Overall, it underscores a trend toward modular skill-based tooling that helps LLMs work reliably in real apps.

Marcus Leiwe from Leiwe & Partners presented The Geometry of Identity: High-Performance Matching with LightGlue. The talk presents an end-to-end pipeline that starts with SIFT keypoint extraction and uses LightGlue’s transformer-based neural matching for sparse, edge-optimized identity comparison. It featured two real-world demos and an interactive Google Colab notebook. Survey feedback suggested strong interest (audience loved it). If released as a product, it could enable on-device, high-speed matching for robotics and spatial interfaces.

Anthony Martin, software architect and founder of Cadenz.ai, delivered a talk titled "WTF is a Decision Plane - and why are agents more reliable with one?" at an AI engineering conference. The presentation outlined the Decision Plane as a dedicated architectural layer that decouples an agent's decision-making process—encompassing timing and rationale—from its action execution mechanisms. Analogous to MVC patterns in user interfaces, this separation imposes explicit boundaries between high-level logic and low-level tooling. Externalizing decisions into a formalized, observable structure enhances agent reliability: it enables rigorous testing, auditing, and predictable outcomes, particularly in extended reasoning sequences involving tools. This approach mitigates implicit dependencies among prompts, memory states, and external integrations, thereby improving stability for complex, long-term tasks. From a systems design perspective, the Decision Plane offers a pragmatic blueprint for engineering robust, modular agents suitable for production environments, transcending mere prototypes.

Nick Githinji of IntelliResume Health presented Building an LLM-Powered Job Classification Pipeline for Healthcare Recruiting, a production system that scrapes 15+ healthcare employers and classifies jobs by specialty, shift, experience, and remote status using OpenAI, while turning messy ATS descriptions into structured text with employer-specific prompts. He walked through the actual code and prompts, including edge cases like 36 hours/week. The 4,000+ weekly jobs showed production heft, and feedback highlighted practical value and reusable prompts, with strong product potential.

BinBin He from Smol Machines presented Building Computers for Agents, a lightweight microVM sandbox for AI agents. It runs embeddable microVMs with libkrun, KVM/Hypervisor.framework, and OCI images, delivering ultra-fast startup and host isolation, as noted by peers. The project is open-source, led by a Seattle-based founder, and targets secure, prod-ready runtimes for autonomous tasks. This aligns with safer agentic workflows and could serve as a building block for local AI. Takeaway: sandboxing is a production-ready concern for agent ecosystems.

Hayden Whayne, a data scientist at Neptu, Inc., presented Radiology Lab, a web app prototype that creates segmentation training data on medical images across planes and modalities. It features a live code walkthrough and a deployable SAM3 API, with an iterative loop that feeds back into fine-tuning models. The UI and ML co-evolve through human-in-the-loop refinements, illustrating a practical data-labeling-to-model-improvement path in medical imaging. For builders, it shows a production-ready workflow with real-world potential.