Uber-assistant QLoRA Adapter

This is a LoRA adapter finetuned on Uber Annual Report 2024

Base Model

meta-llama/Llama-3.1-8B-Instruct

Dataset

Finetuned using the Uber Annual Report 2024 Dataset

Quantization & Training Hyperparameters

• Quantization: 4-bit (NF4)
• Compute Dtype: torch.bfloat16
• Double Quantization: True
• LoRA rank: 16
• LoRA alpha: 32
• Learning rate: 2e-5
• Max steps: 100
• Batch size (effective): 16
• Max length: 512

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

model_id = "vishalgimhan/uber-assistant"
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True
)

base_model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.1-8B-Instruct",
    quantization_config=bnb_config,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)

License & Attribution

This adapter inherits the license of the base model and dataset. Check those licenses before use or redistribution.