EphAsad/FireGenEmbedder

FireGenEmbedder

FireGenEmbedder is a fine-tuned version of the MiniLM model, specifically adapted for sequence classification tasks. The model has been fine-tuned on the Stanford Natural Language Inference (SNLI) dataset to predict the relationship between two sentences, classifying them into three categories: Entailment, Neutral, and Contradiction. It is designed for applications in legal and other domains requiring inference tasks.

Model Details

Base Model: sentence-transformers/all-MiniLM-L6-v2

Fine-tuned Dataset: Stanford Natural Language Inference (SNLI)

Labels:

0: Contradiction

1: Neutral

2: Entailment

Training Epochs: 3

Batch Size: 16 (both train and eval)

Precision: Mixed precision for training on GPU

Model Usage

You can use this model to make inferences on sentence pairs by classifying their relationship.

Install Dependencies

To use this model, install the following libraries:

pip install transformers datasets sentence-transformers torch

Example Code

Here’s an example of how to load and use the FireGenEmbedder model for inference:

from transformers import AutoTokenizer, AutoModelForSequenceClassification import torch

Load the tokenizer and model

model_name = "path_to_firegenembedder_model" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForSequenceClassification.from_pretrained(model_name)

Move model to device (GPU or CPU)

device = "cuda" if torch.cuda.is_available() else "cpu" model.to(device)

Prepare input

premise = "The sky is blue." hypothesis = "The sky is not blue."

inputs = tokenizer(premise, hypothesis, return_tensors="pt", padding=True, truncation=True, max_length=128).to(device)

Inference

with torch.no_grad(): outputs = model(**inputs) predictions = torch.argmax(outputs.logits, dim=-1)

Print the prediction

labels = ["Contradiction", "Neutral", "Entailment"] print(f"Prediction: {labels[predictions.item()]}")

Model Fine-Tuning Process

Data: The model was fine-tuned using the Stanford Natural Language Inference (SNLI) dataset. The SNLI dataset contains labeled pairs of sentences with three classes: Entailment, Neutral, and Contradiction.

Training:

The model was fine-tuned for 3 epochs with a batch size of 16 on a GPU.

The training used mixed precision for faster computation if a GPU was available.

The model is based on the MiniLM architecture, known for being lightweight and efficient, making it suitable for real-time inference tasks.

Post-Training:

The model was saved and zipped for easy distribution.

The tokenizer and model were saved to the directory: miniLM-legal-finetuned-SNLI.

Model Evaluation

The model was evaluated using the validation set from the SNLI dataset, and results can be accessed as follows:

Load the model and evaluate

results = trainer.evaluate() print(results)

Zipped Model

You can download the model as a zip file containing both the model weights and the tokenizer:

Download Model

Citation

If you use this model in your research or application, please cite the following:

@misc{firegenembedder, author = {Your Name}, title = {FireGenEmbedder: Fine-tuned MiniLM for Legal Inference Tasks}, year = {2026}, url = {Link to your Hugging Face model page}, }