mindcraft-ce
The open-source platform for crafting intelligent, collaborative agents in Minecraft using Large Language Models.
A New Era of AI in Minecraft
Andy-4.1 Model
A 3B multimodal model with vision, chain-of-thought reasoning, and a 256k context window — fine-tuned specifically for Minecraft and free to run locally.
Powerful Plugin System
Extend agent capabilities without modifying core code. Create custom actions, integrate new tools, and share your creations with the community.
Collaborative Agents
Test multi-agent coordination with the included MineCollab benchmark, featuring complex cooking, crafting, and construction tasks.
Advanced Voice Interaction
Engage with your agents using both Text-to-Speech (TTS) and Speech-to-Text (STT) for a fully immersive, hands-free experience.
Smarter Vision Modes
Fine-tune how your agent perceives its world with multiple vision modes (`off`, `prompted`, `always`) for any scenario.
Upgraded Pathfinding
Agents move more like human players. They can navigate doors, swim efficiently, and get themselves unstuck from tricky situations.
What's Being Built
The dev branch is where the next generation
of mindcraft-ce is taking shape. These features are not
yet in stable, but you can try them today.
Multi-Agent Architecture
Agents no longer operate alone. A layered system splits responsibilities across specialized agents. The Brain delegates to Task or RP directly; the Task Agent can further call the Code Agent or RP Agent.
The Brain
Central decision-maker. Receives all input and delegates to Task Agent or RP Agent directly.
Task Agent
Goal-directed execution with a specialized system prompt. Can call Code Agent and RP Agent.
RP Agent
Handles character interaction and dialogue. Callable by both Brain and Task Agent.
Code Agent
Reworked for multi-agent integration. Executes sandboxed code on the Task Agent's behalf.
Tools System
Commands have been replaced by a fully dynamic Tools system. Tools are easier to implement, more composable, and lay the groundwork for everything coming next.
Dynamic Tools
Where commands were rigid and hard to extend, Tools are modular functions the AI can call naturally. Adding a new tool no longer requires touching core systems.
Skills — Coming Soon
Skills are additional tools that extend what agents can do. They plug directly into the Tools system, meaning new capabilities can be shipped and loaded without core changes.
Augments
A new kind of persistent intelligence. Augments are reusable code snippets the AI generates for itself — saved and recalled whenever the same need arises again.
AI-Generated, AI-Used
Unlike Skills (which handle big, named tasks), Augments are general-purpose utilities — a loop that works correctly with Minecraft's tick system, a helper that extracts block data in a specific way, or any small but reusable piece of logic the agent finds itself needing repeatedly. The agent writes them once, stores them, and reuses them forever.
Want to try the dev branch?
View Dev Branch on GitHubMeet the Andy-4.1 Model Family
Open-source AI models built specifically for Minecraft. Andy-4.1 is the latest — with vision, chain-of-thought reasoning, and a 256k context window. Requirements shown are for running the model and Minecraft together.
🧠 Andy-4.1
A 3B-parameter multimodal specialist built on a modified Qwen3 VL architecture. The first Andy model with vision understanding and DeepSeek-R1 style chain-of-thought reasoning. Trained on 130,000 examples in just 42 hours.
- Parameters: 3 Billion
- Architecture: Modified Qwen3 VL
- Context Length: Up to 256,000 tokens
- Dataset: 130,000 examples
- Training Time: 42 hours
- Hardware: 1x NVIDIA RTX 3090
Available Formats
| BF16 | ~6 GB VRAM (recommended) |
| GGUF | Various quantizations |
| OpenVINO | CPU / iGPU optimized |
Previous Models
🧠 Andy-4
An 8B-parameter specialist model trained for advanced reasoning and robust in-game decision-making. Trained on a single RTX 3090 over three weeks.
- Parameters: 8 Billion
- Base Model: Llama-3.1-8B
- Tokens Trained: 42 Million
- Hardware: 1x NVIDIA RTX 3090
VRAM Requirements
| F16 | 20 GB+ (Broken, do not use) |
| Q8_0 | 12 GB+ |
| Q5_K_M | 8 GB+ |
| Q4_K_M | 8 GB |
| Q2_K | 6 GB |
🤏 Andy-4-micro
A lightweight 1.5B-parameter variant, optimized for responsive local inference on constrained hardware. Trained on a single RTX 3070 over four days.
- Parameters: 1.5 Billion
- Base Model: Qwen-2.5-1.5B
- Tokens Trained: 42 Million
- Hardware: 1x NVIDIA RTX 3070
VRAM Requirements
| F16 | 6 GB |
| Q8_0 | 6 GB |
| CPU | CAPABLE |
| Q5_K_M | 4 GB |
| Q3_K_M | 2 GB |
Mindcraft vs. Mindcraft-CE
Mindcraft-CE is an experimental fork of the original Mindcraft project. It tracks upstream closely while layering in cutting-edge, community-driven features that aren't yet stable enough for the main project.
| Feature | mindcraft | mindcraft-ce |
|---|---|---|
| Minecraft Version | Up to 1.21.6 | Up to 1.21.4 (1.21.11 on dev *) |
| Default LLM | Generic LLM (e.g. llama3.1) | Andy-4.1 — fine-tuned for Minecraft |
| Voice Interaction | TTS | TTS + Speech-to-Text (STT) |
| Agent System | Multi-agent framework, split Code Agent, RP hints in main loop | Dedicated Brain, Task, RP & Code agents (dev *) |
| Skills | Experimental | Integrated via dynamic Tools system (dev *) |
| Action System | Commands | Dynamic Tools system (dev *) |
| Extensibility | Limited | Plugin System + Skills + Augments (dev *) |
| Self-generated Code | None | Augments — AI-written reusable utilities (dev *) |
* These
features are on the
dev branch
and are not yet stable.
Get Started in Minutes
Download
Get the latest release from GitHub. You'll also need Minecraft: Java Edition, Node.js, and Git.
Download NowInstall Dependencies
Open your terminal in the project folder and run the installation command to fetch all required packages.
npm installRename keys.example.json to keys.json
By default, pollinations is used for free, meaning you do not need to fill any API keys unless usinga specific provider.
Configure & Run
Start a single-player world, open it to LAN, then run the main script to bring your agent to life!
node main.js