BEST-RQ-2 (xares-llm encoder)

This folder contains the BEST-RQ-2 audio encoder integration for xares-llm.

Benchmark repository: xiaomi-research/xares-llm

Setup

1. Make sure the environment to run xares-llm is set up properly (virtual environment initialized and the xares-llm package downloaded/installed).

2. Add the BEST-RQ-2 folder to the xares-llm (current) directory so it is available at ./BEST-RQ-2.

3. Before running a xares-llm evaluation, you must install the required packages for BEST-RQ-2:

uv pip install -r BEST-RQ-2/audio-embeddings/pyproject.toml

Run an evaluation

Single task (e.g. to test that everything works):

uv run -m xares_llm.run BEST-RQ-2.BEST-RQ-2_encoder.BestRQ2Encoder <task> <task> # e.g. replace <task> with esc-50, score should be around 0.48

All tasks:

uv run -m xares_llm.run BEST-RQ-2.BEST-RQ-2_encoder.BestRQ2Encoder all

Help

If you encounter any problems, contact: [email protected]