---
license: mit
datasets:
- HermesPenn/athena_data
---
# News Classifier -- The Evaluation Pipeline
## The colab link: https://colab.research.google.com/drive/1OmIHVN0joIgjGgYCdqLu2EO2By4yT5Xd#scrollTo=MsmKRoHuHyIp
## Ziao You, Samuel Vara, Surya Sandeep Akella
----------------------

## The codes here are the same as the colab link. It shows how to call our model to evaluate the test set. Please use the colab link for easier usage.
##

----------------------

### pip install package
```
!pip install datasets > delete.txt
```

### !!! Load Test Set -- Change the file path of test set
```
import pandas as pd
df_test = pd.read_csv('/content/test_data.csv',index_col="Unnamed: 0")
df_test.head()
```

### Load Model from Hugging Face Hub (Don't change)
```
from huggingface_hub import snapshot_download
import keras

# Download model from hugging face
local_path = snapshot_download(repo_id="HermesPenn/athena_model")

# Load model from local
model = keras.saving.load_model(local_path)
```
### Load Training set (Don't change)
```
from datasets import load_dataset

dataset = load_dataset("HermesPenn/athena_data")
dataset = dataset['train']
data = dataset.to_pandas()
data.head()
```

### Fit_transform label_encoder and tokenizer (Don't change)
```
from sklearn.preprocessing import LabelEncoder
from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.preprocessing.text import Tokenizer
# Data preprocessing
le = LabelEncoder()
data['label'] = le.fit_transform(data['source'])
X = data['title']
y = data['label']

# Tokenize and pad text data
tokenizer = Tokenizer(num_words=20000, oov_token="<OOV>")
tokenizer.fit_on_texts(X)
X_seq = tokenizer.texts_to_sequences(X)
X_padded = pad_sequences(X_seq, maxlen=200, padding='post', truncating='post')
```

### Test set Evaluation (Don't change)

```
from sklearn.metrics import classification_report

X_test = df_test['title']
y_test = df_test['label']


X_test_seq = tokenizer.texts_to_sequences(X_test)
X_test_padded = pad_sequences(X_test_seq, maxlen=200, padding='post', truncating='post')

# Predict the labels using the model
y_pred_probs = model.predict(X_test_padded)
y_pred = (y_pred_probs > 0.5).astype(int)

# Evaluate the model
print("Classification Report:")
print(classification_report(y_test, y_pred))


try:
news_outlets = le.inverse_transform(y_pred.flatten()) # le must be pre-fitted
df_test['Predicted News Outlet'] = news_outlets
except NameError:
df_test['Predicted News Outlet'] = y_pred.flatten()
```

```
# Display test set with predictions
print("\nTest Set with Predictions:")

df_test[['title', 'News Outlet', 'Predicted News Outlet']]
```