Update README.md
Browse files
README.md
CHANGED
|
@@ -17,7 +17,7 @@ GitVac is like a vacuum cleaner for code fixes. It's a series of 3B, 8B, 14B, an
|
|
| 17 |
|
| 18 |
# Training Data
|
| 19 |
*The data to train the models came from r1 outputs via distalation.*
|
| 20 |
-
However to gauge accuracuy of the model o3/r1 were used to do eval with the same prompts.
|
| 21 |
|
| 22 |
# How is verification done?
|
| 23 |
A lot of models are already trained on function calling syntax.
|
|
@@ -354,6 +354,8 @@ Tested against the remainder 1300 r1 could not pass. These datasets were never s
|
|
| 354 |
|
| 355 |
| Model | Success Rate | Notes |
|
| 356 |
|-------|--------------|-------|
|
|
|
|
|
|
|
| 357 |
| GitVac-3B-Turbo | 42% | Fast but not accurate. Missing tags in outputs. |
|
| 358 |
| Llama-GitVac-7B-Turbo | 90% | Sweet spot for speed/quality/cost. No think tags. |
|
| 359 |
| GitVac-R-14B | 92% | Excellent reasoning capabilities for price/speed |
|
|
|
|
| 17 |
|
| 18 |
# Training Data
|
| 19 |
*The data to train the models came from r1 outputs via distalation.*
|
| 20 |
+
However to gauge accuracuy of the model o3/r1 were used to do eval with the same prompts.
|
| 21 |
|
| 22 |
# How is verification done?
|
| 23 |
A lot of models are already trained on function calling syntax.
|
|
|
|
| 354 |
|
| 355 |
| Model | Success Rate | Notes |
|
| 356 |
|-------|--------------|-------|
|
| 357 |
+
| O3 | 30% | used for eval |
|
| 358 |
+
| R1 | 15% | used for eval/training |
|
| 359 |
| GitVac-3B-Turbo | 42% | Fast but not accurate. Missing tags in outputs. |
|
| 360 |
| Llama-GitVac-7B-Turbo | 90% | Sweet spot for speed/quality/cost. No think tags. |
|
| 361 |
| GitVac-R-14B | 92% | Excellent reasoning capabilities for price/speed |
|