requirement_check/README.md

metadata

license: apache-2.0
language:
  - en
pipeline_tag: text-generation
library_name: transformers

Requirement Checker Adapters

Model Summary

This Requirement Checker family of adapters are designed to check if specified requirements were satisfied by the last model generation. Only one requirement is checked at a time (multiple requirements can be checked with parallel model calls).

• Developer: IBM Research
• License: Apache 2.0

Usage

Intended use

Usage steps Given a generation task and a set of requirements:

1. Use the base model to generate a response as normal (via the assistant role), with the prompt describing the task followed by "Requirements:"" and the list of active requirements.
2. Repeat the requirement to be checked.
3. The Requirement Checker model will respond with "true" or "false", where "true" means the requirement is satisfied.

Quickstart Example

Evaluation

The model was evaluated on 200 rows of held-out synthetic data. Error rates are as follows:

aLoRA models

• Granite 3.3 2B: 6.0%
• Granite 3.3 8B: 5.75%
• GPT-OSS 20B: 5.75%

LoRA models

• Granite 3.3 2B: 4.5%
• Granite 3.3 8B: 4.0%
• GPT-OSS 20B: 4.0%

Training Data

Synthetic data generated by Mixtral 8x22b and GPT-OSS 120B.

Model Card Authors

Kristjan Greenewald Bo Wu