File size: 2,653 Bytes
fd690bc
62074b2
 
 
fd690bc
62074b2
 
 
 
fd690bc
62074b2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
---
license: mit
datasets:
- HermesPenn/athena_data
---
# News Classifier -- The Evaluation Pipeline
## The colab link: https://colab.research.google.com/drive/1OmIHVN0joIgjGgYCdqLu2EO2By4yT5Xd#scrollTo=MsmKRoHuHyIp
## Ziao You, Samuel Vara, Surya Sandeep Akella
----------------------

## The codes here are the same as the colab link. It shows how to call our model to evaluate the test set. Please use the colab link for easier usage.
##

----------------------

### pip install package
```
!pip install datasets > delete.txt
```

### !!! Load Test Set -- Change the file path of test set
```
import pandas as pd
df_test = pd.read_csv('/content/test_data.csv',index_col="Unnamed: 0")
df_test.head()
```

### Load Model from Hugging Face Hub (Don't change)
```
from huggingface_hub import snapshot_download
import keras

# Download model from hugging face
local_path = snapshot_download(repo_id="HermesPenn/athena_model")

# Load model from local
model = keras.saving.load_model(local_path)
```
### Load Training set (Don't change)
```
from datasets import load_dataset

dataset = load_dataset("HermesPenn/athena_data")
dataset = dataset['train']
data = dataset.to_pandas()
data.head()
```

### Fit_transform label_encoder and tokenizer (Don't change)
```
from sklearn.preprocessing import LabelEncoder
from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.preprocessing.text import Tokenizer
# Data preprocessing
le = LabelEncoder()
data['label'] = le.fit_transform(data['source'])
X = data['title']
y = data['label']

# Tokenize and pad text data
tokenizer = Tokenizer(num_words=20000, oov_token="<OOV>")
tokenizer.fit_on_texts(X)
X_seq = tokenizer.texts_to_sequences(X)
X_padded = pad_sequences(X_seq, maxlen=200, padding='post', truncating='post')
```

### Test set Evaluation (Don't change)

```
from sklearn.metrics import classification_report

X_test = df_test['title']
y_test = df_test['label']


X_test_seq = tokenizer.texts_to_sequences(X_test)
X_test_padded = pad_sequences(X_test_seq, maxlen=200, padding='post', truncating='post')

# Predict the labels using the model
y_pred_probs = model.predict(X_test_padded)
y_pred = (y_pred_probs > 0.5).astype(int)

# Evaluate the model
print("Classification Report:")
print(classification_report(y_test, y_pred))


try:
news_outlets = le.inverse_transform(y_pred.flatten()) # le must be pre-fitted
df_test['Predicted News Outlet'] = news_outlets
except NameError:
df_test['Predicted News Outlet'] = y_pred.flatten()
```

```
# Display test set with predictions
print("\nTest Set with Predictions:")

df_test[['title', 'News Outlet', 'Predicted News Outlet']]
```