Correct HumanEval scores
#79
by Muennighoff - opened
Previous scores did not strip end of sequence tokens. The updated scores ignore the end of sequence tokens (</s>).
This is equivalent to evaluating on code generations decoded with tokenizer.decode(code_tokens, skip_special_tokens=True)
Muennighoff changed pull request title from Update README.md to Correct HumanEval scores
Muennighoff changed pull request status to merged