woojin0412/common

This model is a fine-tuned version of the meta-llama/Meta-Llama-3.1-8B-Instruct model, specifically adapted for enhanced Korean text generation and question answering. It has been trained on the woojin0412/common dataset.

Features

This model offers the following key features.

• Korean Text Generation: Optimized for generating coherent and contextually relevant text in the Korean language.
• Instruction Following: Fine-tuned to understand and respond to user instructions effectively.
• Question Answering: Capable of providing answers to questions posed in Korean.
• Efficient Inference: Designed for efficient inference by utilizing 4-bit quantization and float16 precision.

How to Get Started with the Model

1. Import huggingface library and login huggingface

This code imports the huggingface_hub library, which is used to interact with the Hugging Face Hub, a platform for sharing and storing machine learning models, datasets, and tokenizers. The huggingface_hub.login() function then prompts the user to log in to the Hugging Face Hub, typically requiring an API key. This login is often necessary for accessing private models or uploading models to the Hub. This initial block handles the authentication process with the Hugging Face Hub.

import huggingface_hub
huggingface_hub.login()

2. Run korean text generation

This code segment focuses on loading and utilizing a pre-trained causal language model. It begins by importing necessary libraries from transformers and torch. It then specifies the model to be loaded from the Hugging Face Hub (loadModel = "woojin0412/common"). The code loads the model using AutoModelForCausalLM.from_pretrained, optimizing memory usage and setting data types for efficiency. It also loads the corresponding tokenizer. An example Korean input is defined, formatted into a chat-style message, and then converted into a prompt suitable for the model. The model generates a response using the model.generate function with specified generation parameters. Finally, the generated tokens are decoded back into a human-readable text response, cleaned up, and printed to the console. This block essentially demonstrates the process of loading a language model, preparing input, generating text, and displaying the output.

from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
import torch

# Specify the model name
loadModel = "woojin0412/common"

# Load the fine-tuned model
model = AutoModelForCausalLM.from_pretrained(loadModel, low_cpu_mem_usage=True, return_dict=True, torch_dtype=torch.float16, device_map= "auto")
model.eval() # Set the model to evaluation mode

# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(loadModel, trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right"

# Example input
input_text = "한국어로 '안녕하세요'를 세 번 반복해서 말해줘."

messages = [{"role": "user", "content": input_text}]

prompt_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

inputs = tokenizer(prompt_text, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=True,
    temperature=0.1,
    top_p=0.95,
    eos_token_id=tokenizer.eos_token_id
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)

if "assistant" in response:
    response = response.split("assistant")[-1].strip()
    
# Decode and print the output
print(response.strip())