next-codex / README.md

Lamapi

Update README.md

7a1a096 verified 7 days ago

preview code

raw

history blame contribute delete

6.62 kB

metadata

language:
  - tr
  - en
  - de
  - es
  - fr
  - ru
  - zh
  - ja
  - ko
license: mit
tags:
  - turkish
  - türkiye
  - ai
  - lamapi
  - next-codex
  - coder
  - codex
  - text-generation
  - open-source
  - 30b
  - moe
  - mixture-of-experts
  - code-generation
  - coding
  - llm
  - transformer
  - artificial-intelligence
pipeline_tag: text-generation
datasets:
  - mlabonne/FineTome-100k
  - google/code_x_glue_ct_code_to_text
  - bigcode/the-stack-v2
  - neulab/agent-data-collection
  - openai/gsm8k
  - princeton-nlp/SWE-bench_Verified
  - microsoft/orca-math-word-problems-200k
  - qwedsacf/competition_math
  - hotpotqa/hotpot_qa
  - wics/strategy-qa
  - glaiveai/glaive-function-calling-v2
  - Anthropic/hh-rlhf
  - ccdv/cnn_dailymail
  - allenai/ai2_arc
  - allenai/sciq
  - google-research-datasets/mbpp
  - openai/openai_humaneval
  - allenai/openbookqa
  - baber/piqa
  - allenai/winogrande
  - Rowan/hellaswag
  - allenai/social_i_qa
  - databricks/databricks-dolly-15k
  - truthfulqa/truthful_qa
  - HuggingFaceH4/ultrachat_200k
  - OpenAssistant/oasst1
  - iamtarun/python_code_instructions_18k_alpaca
  - nickrosh/Evol-Instruct-Code-80k-v1
  - arcee-ai/agent-data
  - GreenerPastures/All-Your-Base-Full
  - FreedomIntelligence/Socratic
  - qihoo360/Light-R1-SFTData
  - dongguanting/ARPO-SFT-54K
library_name: transformers

💻 Next-Codex (L846MoE)

Code your future with our models.

📖 Overview

Next-Codex is a high-performance, specialized Mixture-of-Experts (MoE) Large Language Model designed specifically for code generation, debugging, and software engineering tasks.

Unlike traditional dense models, Next-Codex utilizes a sparse architecture with 30 Billion total parameters, but only activates 3 Billion parameters per token. This unique design allows it to deliver the deep reasoning capabilities of a massive model while maintaining the ultra-low latency and inference cost of a lightweight 3B model. It is fine-tuned on a massive corpus of code across 20+ programming languages, making it the most efficient coding assistant in its class.

⚡ Highlights

🇹🇷 Türkiye’s First Specialized MoE Coding Model: Designed for speed and precision.
🚀 Hyper-Efficient Inference: Runs with 3B active parameters, enabling deployment on consumer GPUs (e.g., RTX 3090/4090).
💻 SOTA Coding Performance: Surpasses Claude Sonnet 4 and rivals o3-High in Python & JavaScript benchmarks.
🌍 Polyglot Programming: Master-level proficiency in Python, JS/TS, Rust, Go, C++, SQL, and Swift.
🧠 Context-Aware Debugging: Excellent at understanding large codebases and suggesting architectural improvements.
🏢 Production Ready: Optimized for autocomplete, unit test generation, and docstring creation.

📊 Benchmark Performance (Coding & Logic)

Next-Codex achieves state-of-the-art results among open-weights coding models, balancing extreme efficiency with high accuracy.

Benchmarks are being conducted...

🚀 Installation & Usage

Note: Due to the MoE architecture, this model is memory efficient. You can run it comfortably on 24GB VRAM GPUs (4-bit quantization highly recommended for lower VRAM).

!pip install unsloth transformers

from unsloth import FastLanguageModel

# Load the MoE Model
model, tokenizer = FastLanguageModel.from_pretrained(
    "Lamapi/next-codex",
    load_in_4bit = True, # Optimized for 24GB VRAM
)

messages = [
    {"role": "system", "content": "You are Next-Codex, an expert software engineer and AI coding assistant."},
    {"role" : "user", "content" : "Write a highly optimized Rust function to calculate the Fibonacci sequence using memoization."}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize = False,
    add_generation_prompt = True
)

from transformers import TextStreamer
_ = model.generate(
    **tokenizer(text, return_tensors = "pt").to("cuda"),
    max_new_tokens = 2048,
    temperature = 0.2, # Lower temperature for code precision
    top_p = 0.95,
    streamer = TextStreamer(tokenizer, skip_prompt = True),
)

🧩 Key Features

Feature	Description
🔀 Smart Routing (MoE)	Dynamically routes tokens to the best "expert" layers, activating only 3B params for speed.
🛠️ Full-Stack Mastery	Trained on frontend (React, Vue), backend (Django, Spring), and systems (C, Rust) code.
🇹🇷 Code Support	Exceptional ability to understand Turkish variable names and comments in legacy codebases.
🐞 Deep Debugging	Analyzes stack traces and logic errors to provide instant fixes.
📝 Docstring & Testing	Automatically generates Javadoc, PyDoc, and Unit Tests (Pytest/Jest).
🔒 Secure Coding	Aligned to avoid common vulnerabilities (SQLi, XSS) in generated code.

📐 Model Specifications

Specification	Details
Architecture	Mixture of Experts (MoE) Transformer
Total Parameters	30 Billion
Active Parameters	3 Billion (per token)
Context Window	32k Tokens
Experts	8 Experts (Top-2 Routing)
Training Data	1T+ Tokens of Code (The Stack v2, GitHub, Synthetic)
Quantization	GGUF, AWQ, GPTQ supported

🎯 Ideal Use Cases

IDE Autocomplete Plugins — Low latency makes it perfect for "Copilot" style completions.
Legacy Code Refactoring — Converting outdated code to modern standards (e.g., Java 8 to Java 21).
SQL Generation — Text-to-SQL for complex data analytics.
Turkish/English Development — Teams working in bilingual environments.
Algorithm Optimization — Reducing time complexity of existing functions.

📄 License

Licensed under the MIT License — free for commercial and non-commercial use.

📞 Contact & Support

📧 Email: [email protected]
🤗 HuggingFace: Lamapi

Next-Codex — Smart as a giant, fast as a lightweight. The future of coding is MoE.