SVM Model with TF-IDF

This repository provides a pre-trained Support Vector Machine (SVM) model for text classification using Term Frequency-Inverse Document Frequency (TF-IDF). The repository also includes utilities for data preprocessing and feature extraction:

Start:

Open your terminal.
Clone the repo by using the following command:

git clone https://huggingface.co/CIS5190abcd/svm

Go to the svm directory using following command:

cd svm

Run ls to check the files inside svm folder. Make sure tfidf.py, svm.py and data_cleaning.py are existing in this directory. If not, run the folloing commands:

git checkout origin/main -- tfidf.py
git checkout origin/main -- svm.py
git checkout origin/main -- data_cleaning.py

Rerun ls, double check all the required files are existing. Should look like this:

keep inside the svm directory until ends.

Installation

Before running the code, ensure you have all the required libraries installed:

pip install nltk beautifulsoup4 scikit-learn pandas datasets

Download necessary NTLK resources for preprocessing.

python
import nltk
nltk.download('stopwords')
nltk.download('wordnet')

After downloading all the required packages,

exit()

How to use:

Training a new dataset with existing SVM model, follow the steps below：

Clean the Dataset

from data_cleaning import clean
import pandas as pd
import nltk
nltk.download('stopwords')

You can replace with any datasets you want by changing the file name inside pd.read_csv().

# Load your data
df = pd.read_csv("hf://datasets/CIS5190abcd/headlines_test/test_cleaned_headlines.csv")

# Clean the data
cleaned_df = clean(df)

Extract TF-IDF Features

from tfidf import tfidf

# Transform the cleaned dataset
X_new_tfidf = tfidf.transform(cleaned_df['title'])

Make Predictions

from svm import svm_model

# Make predictions
predictions = svm_model.predict(X_new_tfidf)