MMLU-by-task-Leaderboard / result_data_processor.py

Commit History

Changed error logging from print statements to logger. It is not currently working to save to a file locally
d96fdf9

Corey Morris commited on

Catching exceptions in processing files. As new data is introduced, I want to know which files may have different formats and cause problems, but the application shouldn't halt if it can't process a single file
68bce52

Corey Morris commited on

Returning just a single file per model directory. Manually removing gpt-j-6b for now because there is something that is causing problems with processing the data
794b32b

Corey Morris commited on

removing models that are known to have training data contaminated with evaluations
a5840fb

Corey Morris commited on

Fixed type error
e79bcf3

Corey Morris commited on

WIP commit. Currently have nlargest error
d506f10

Corey Morris commited on

Added organization to dataframe
52d3b03

Corey Morris commited on

removed code to print the number of outliers. could add it back later as logging potentially
cd21f99

Corey Morris commited on

MC1 column had 8 rows with a value of 1. It didn't make sense given the next highest value was 0.47 . Assuming they were data errors, they were removed
e03b231

Corey Morris commited on

truthfulqa data added to dataframe
abac22e

Corey Morris commited on

Refactor to make later code changes easier
6d41115

Corey Morris commited on

Added test for removal of undesired columns. fixed code error in column removal
9549fcc

Corey Morris commited on

Moved rank data into a separate method and dataframe
31bed1a

Corey Morris commited on

Extracting parameter data from the names of the models
dcadab7

Corey Morris commited on

Refactor. Extracted methods.
ee5ac8e

Corey Morris commited on

Refactoring. Moved ResultDataProcessor class to a separate file to make it easier to use with experimentation in a jupyter notebook
843a5ef

Corey Morris commited on