vkerkez commited on
Commit
fc35caa
·
verified ·
1 Parent(s): 5d1f03e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -17,7 +17,7 @@ GitVac is like a vacuum cleaner for code fixes. It's a series of 3B, 8B, 14B, an
17
 
18
  # Training Data
19
  *The data to train the models came from r1 outputs via distalation.*
20
- However to gauge accuracuy of the model o3/r1 were used to do eval with the same prompts.
21
 
22
  # How is verification done?
23
  A lot of models are already trained on function calling syntax.
@@ -354,6 +354,8 @@ Tested against the remainder 1300 r1 could not pass. These datasets were never s
354
 
355
  | Model | Success Rate | Notes |
356
  |-------|--------------|-------|
 
 
357
  | GitVac-3B-Turbo | 42% | Fast but not accurate. Missing tags in outputs. |
358
  | Llama-GitVac-7B-Turbo | 90% | Sweet spot for speed/quality/cost. No think tags. |
359
  | GitVac-R-14B | 92% | Excellent reasoning capabilities for price/speed |
 
17
 
18
  # Training Data
19
  *The data to train the models came from r1 outputs via distalation.*
20
+ However to gauge accuracuy of the model o3/r1 were used to do eval with the same prompts.
21
 
22
  # How is verification done?
23
  A lot of models are already trained on function calling syntax.
 
354
 
355
  | Model | Success Rate | Notes |
356
  |-------|--------------|-------|
357
+ | O3 | 30% | used for eval |
358
+ | R1 | 15% | used for eval/training |
359
  | GitVac-3B-Turbo | 42% | Fast but not accurate. Missing tags in outputs. |
360
  | Llama-GitVac-7B-Turbo | 90% | Sweet spot for speed/quality/cost. No think tags. |
361
  | GitVac-R-14B | 92% | Excellent reasoning capabilities for price/speed |