Your conversational JSONL goes in. Your fine-tuned GGUF model comes out.
AarForge is an opinionated local LLM fine-tuning and deployment pipeline that transforms conversational JSONL datasets into fine-tuned GGUF models, ready to run locally with Ollama.
Built around Unsloth, LoRA, 4-bit quantization, and Hugging Face tools, AarForge makes local LLM fine-tuning more accessible on NVIDIA consumer GPUs.
No massive infrastructure. No complicated cloud setup. Just your GPU, your dataset, and a questionable amount of patience.
- Automated Setup: Creates a Python virtual environment and installs the required dependencies.
- 4-bit Fine-tuning: Uses Unsloth and BitsAndBytes to reduce GPU memory requirements.
- Conversational JSONL: Supports OpenAI-style datasets containing
system,user, andassistantmessages. - LoRA Fine-tuning: Trains lightweight adapters instead of updating the entire base model.
- GGUF Export: Merges and exports the fine-tuned model into GGUF format.
- Ollama Integration: Generates an Ollama Modelfile and deploys a local model when Ollama is installed and reachable; training and export can complete without Ollama.
- NVIDIA GPU with CUDA support.
- Recommended: 6GB or more VRAM (requirements vary depending on the model and configuration).
- Python 3.10–3.13 (the installer creates a Python 3.13 environment).
- Linux environment with Bash.
curlfor installation; Git, CMake,make, and a C++ compiler for GGUF export.- Internet connection for downloading dependencies and model weights.
- Store Requirements may vary from 10GB to 50GB, I would say a minimum of like 20GB for your first run.
Actual resource requirements depend on the selected base model, training configuration, and export process.
1. Clone the repository
git clone https://github.com/AaritDev/AarForge.git
cd AarForge2. Make the installer executable
chmod +x install.sh3. Run the installer
./install.sh4. Choose your workspace
The installer will ask where you want to create your AarForge workspace.
In the folder selector, you can enter a sequence of navigation keys at once: for example, uu310 goes to the parent twice, enters folder 3, then folder 1, and selects that directory.
It will then:
- Install Astral UV if necessary.
- Install Ollama if necessary.
- Create a Python virtual environment.
- Install the required Python dependencies.
- Optionally pre-download the default base model if it is not already cached.
- Copy
build.shinto your workspace. - Generate a sample conversational dataset.
Pre-downloading is not required: the installer checks the Hugging Face cache and skips the download if the default model is present. Otherwise, the optional prompt defaults to No; the first training run downloads missing files, which Hugging Face caches for later runs. Installer dependency and Unsloth diagnostics are saved as JSONL in AarForge/Logs/installer.jsonl instead of being printed verbatim.
Once installation finishes, navigate to your selected workspace.
AarForge uses OpenAI-style conversational JSONL.
Each line in your .jsonl file must contain one valid JSON object with a top-level messages array.
Every message contains:
role: The message speaker (system,user, orassistant).content: The message text.
{"messages": [{"role": "system", "content": "You are AarBot, a rude and sarcastic AI."}, {"role": "user", "content": "Hello"}, {"role": "assistant", "content": "Oh get lost, I don't care about your hello."}]}
{"messages": [{"role": "system", "content": "You are AarBot, a rude and sarcastic AI."}, {"role": "user", "content": "What is 2 + 2?"}, {"role": "assistant", "content": "Seriously? It's 4. Even a dog could answer that."}]}- Each line represents one independent conversation.
- Each conversation must have its own
messagesarray. - Each conversation can contain its own system prompt.
- Messages must follow the correct conversational order.
- Message content must be a string.
- Every line must contain valid JSON.
- Each conversation must include at least one assistant response.
The system message defines the behavior, personality, and instructions for that particular conversation. User and assistant messages provide the examples used during fine-tuning.
Save your dataset with a .jsonl extension.
For example:
my_dataset.jsonl
A small tip: Quality matters more than quantity. A thousand well-written examples are generally more useful than ten thousand conversations that look like they were generated during a caffeine overdose.
1. Navigate to your AarForge workspace
cd /path/to/your/workspace2. Start the build wizard
chmod +x build.sh
./build.sh3. Configure your model
The interactive wizard will request the following information:
| Setting | Description |
|---|---|
| Custom AI Name | Name of your Ollama model |
| Dataset Path | Path to your conversational JSONL dataset |
| Base Model ID | Hugging Face or Unsloth base model identifier (default: huihui-ai/Llama-3.2-3B-Instruct-abliterated) |
| Training Epochs | Number of passes through the training dataset (1–999999; default: 3) |
| Training Steps | Maximum optimizer steps (0–999999; default: 60; enter 0 to use epochs) |
| System Prompt | System instructions for Ollama inference |
| Temperature | Controls response randomness during inference |
At the start of each build, choose a previously used model or enter a name for a different model. Settings are shared across the workspace in AarForge/settings.json; each build updates those shared prompt defaults, rather than saving separate settings per model. install.sh initializes the workspace settings, and existing .aarforge-settings.json files are migrated automatically. Previously created per-model settings files are no longer used and are left untouched.
The dataset browser also accepts chained actions, such as uud2f1 to go up twice, enter folder 2, and select JSONL file 1. Its existing u, dN, fN, and 0 choices remain available.
When both training limits are configured, a positive step count takes precedence over epochs. The default run is therefore 60 steps; enter 0 steps to train for the configured number of epochs.
Each run gets a 16-bit numeric code and is grouped under its custom model name: artifacts are stored in AarForge/Runs/<model-name>/<code>, and its settings snapshot, JSON report, and JSONL event log in AarForge/Logs/<model-name>/<code>. Only the newest run's artifacts are kept for each model: older run folders for that model are removed when its next run starts, while runs for other models are unaffected. Only the 30 newest run-log folders for that model are retained. The event log records timestamp, run code, stage, severity, and message. Existing run folders in the old flat layout are left untouched. Detailed output from Unsloth, model loading, export, and Ollama is captured in that JSONL log. Loading/export progress is shown live and updates in place; full training metrics are preserved in the event log, and a periodic notice appears if a stage is quiet for a while.
Both scripts use the same terminal status labels and confirmation prompts. Press Enter to accept a displayed default; set NO_COLOR=1 to disable terminal colors.
4. Wait for training and export
AarForge automatically handles the following pipeline:
- Dataset loading and preprocessing.
- LoRA fine-tuning.
- Adapter weight saving.
- GGUF export.
- Ollama Modelfile generation.
- Ollama model creation, when Ollama is installed and reachable. Otherwise, training and export still complete, and deployment can be done later.
Depending on your hardware, model size, and training configuration, this process can take some time.
Go grab a drink. Your GPU has work to do.
5. Run your model
After successful completion:
ollama run my_custom_botReplace my_custom_bot with your chosen model name.
That's it. Your fine-tuned model is now ready for local inference.
After installation and a successful build, your workspace will contain files similar to:
workspace/
├── .venv/
├── build.sh
├── sample_dataset.jsonl
├── <model-name>_dataset.jsonl
└── AarForge/
├── settings.json
├── capture_output.py
├── Logs/
│ ├── installer.jsonl
│ └── <model-name>/
│ └── <run-code>/
│ ├── events.jsonl
│ ├── report.json
│ └── settings.json
└── Runs/
└── <model-name>/
└── <run-code>/
├── adapter/
├── gguf/
├── preflight.py
├── train_bot.py
├── export_gguf.py
├── read_manifest.py
├── write_modelfile.py
└── Modelfile
Generated model names, run codes, and output files depend on your configuration. Only the newest run is retained under each model's Runs folder.
AarForge builds on several excellent open-source projects:
- Unsloth — Efficient LLM fine-tuning.
- Hugging Face Transformers — Model and tokenizer management.
- PEFT — Parameter-efficient fine-tuning.
- BitsAndBytes — 4-bit quantization and quantized model loading.
- Astral UV — Fast Python package and environment management.
- Ollama — Local model deployment and inference.
- Requires an NVIDIA GPU with compatible CUDA support.
- Training performance depends heavily on available VRAM.
- Not every Hugging Face model architecture is necessarily supported.
- GGUF conversion compatibility depends on the selected base model.
- Fine-tuning quality depends on dataset quality and training configuration.
AarForge wouldn't exist without the work of the Unsloth, Hugging Face, and Ollama communities.
A big thank you to everyone building and maintaining the tools that make local AI development possible.
Base models retain their respective licenses and usage restrictions. Make sure to review the applicable model license before using or distributing fine-tuned models.
If you find AarForge useful, consider giving the repository a star!
It costs you nothing, but it makes me little programmer dopamine receptors go YES.
Copyright (C) 2026 Aarit Pandey
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.


