| --- |
| license: llama2 |
| library_name: peft |
| tags: |
| - solana |
| - rust |
| - anchor |
| - smart-contracts |
| - finance |
| - crypto |
| - unsloth |
| - codellama |
| base_model: codellama/CodeLlama-7B-Instruct-hf |
| datasets: |
| - synthetic-solana-anchor-10k |
| language: |
| - en |
| --- |
| |
| # Solana-CodeLlama-7B-v1 (Anchor Specialized) |
|
|
| ## Overview |
| **Solana-CodeLlama-7B-v1** is a domain-specialized language model fine-tuned for writing production-ready **Solana Smart Contracts** using the **Anchor Framework**. |
|
|
| While general coding models (like GPT-4 or standard CodeLlama) often hallucinate outdated syntax or struggle with Rust's strict ownership rules, this model was trained on a **high-purity synthetic dataset** of 10,000 algorithmic examples, focusing specifically on: |
| * **Anchor Macros:** Correct usage of `#[derive(Accounts)]`, `#[program]`, `#[account]`. |
| * **Security Constraints:** Proper PDA seed validation and constraint checks (e.g., `#[account(mut, seeds = [...], bump)]`). |
| * **Rust & SPL Tokens:** Accurate CPI calls to the SPL Token program. |
|
|
| ## Performance & Benchmarks |
| The model was evaluated against the base `CodeLlama-7B-Instruct` model on a specific "Solana Hold-Out Set". |
|
|
| | Metric | Base Model (Zero-Shot) | **Solana-CodeLlama-7B-v1** | |
| | :--- | :---: | :---: | |
| | **Accuracy (Validation)** | ~35% (Hallucinates Python/Solidtiy) | **97.26%** | |
| | **Accounts Struct** | ❌ FAIL | ✅ PASS | |
| | **Context Validation** | ❌ FAIL | ✅ PASS | |
| | **PDA Initialization** | ❌ FAIL | ✅ PASS | |
| | **SPL Token Transfer** | ❌ FAIL | ✅ PASS | |
|
|
| *> "The model didn't just learn; it absorbed the syntax structure instantly, dropping loss to 0.02 in < 2 epochs."* |
|
|
| ## Dataset |
| * **Source:** 100% Synthetic (Algorithmic Generation). |
| * **Size:** 10,000 Verified Examples. |
| * **Methodology:** We utilized a "Textbook Quality" approach, generating examples with perfect compile-ready logic rather than scraping noisy GitHub repositories. |
|
|
| ## Usage |
|
|
| ### 1. Using Unsloth (Fastest) |
| ```python |
| from unsloth import FastLanguageModel |
| |
| model, tokenizer = FastLanguageModel.from_pretrained( |
| model_name = "your-username/Solana-CodeLlama-7B-v1", |
| max_seq_length = 2048, |
| dtype = None, |
| load_in_4bit = True, |
| ) |
| |
| prompt = """Write a Solana Anchor program to initialize a user vault.""" |
| # ... Apply chat template ... |
| ``` |
|
|
| ### 2. Using GGUF (Ollama / LM Studio) |
| This model is available in GGUF format for local deployment on consumer hardware (MacBook M1/M2/M3, NVIDIA RTX 3060/4090/5090). |
| * `Solana-CodeLlama-7B-v1.Q4_K_M.gguf` (Recommended for 8GB+ RAM) |
| * `Solana-CodeLlama-7B-v1.Q8_0.gguf` (High Precision) |
|
|
| ## Training Details |
| * **Hardware:** NVIDIA RTX 5090 (32GB VRAM). |
| * **Framework:** Unsloth (Open Source). |
| * **Precision:** Mixed Precision (BF16). |
| * **LoRA Rank:** 16. |
| * **Batch Size:** 8 (Effective). |
|
|
| ## License |
| Based on CodeLlama (Llama 2 Community License). |
|
|
| --- |
| *Fine-tuned with ❤️ using [Unsloth](https://github.com/unslothai/unsloth).* |
|
|
