# CS634Project

Milestone-3 notebook: https://colab.research.google.com/drive/17-7A0RkGcwqcJw0IcSvkniDmhbn5SuXe

Hugging Face App: 

***********

Results:

***********

XGBoost Model's RMSE: 28986  (Milestone-2)

Baseline LGBM's RMSE: 26233

Optuna optimized LGBM's RMSE: 13799.282803291926

***********

Totalnumber of trials:  120

Best RMSE score on validation data: 12338.665498601415

------------------------------

Best params:

------------------------------

boosting_type :	 goss

reg_alpha :	 3.9731274536451826

reg_lambda :	 0.8825276525195174

colsample_bytree :	 1.0

subsample :	 1.0

learning_rate :	 0.05

max_depth :	 6

num_leaves :	 48

min_child_samples :	 1

***********

Documentation

***********

Dataset: https://www.kaggle.com/competitions/house-prices-advanced-regression-techniques/overview

Data Processing and Feature Selection:

For the feature selection, I started by dropping columns with a low correlation (< 0.4) with SalePrice. I then dropped columns with low variances (< 1). After that, I checked the correlation matrix between columns to drop selected columns that have a correlation greater than 0.5 but with consideration for domain knowledge. After that, I checked for NAs in the numerical columns. Then, based on the result, I used domain knowledge to fill the NAs with appropriate values. In this case, I used 0 to fill the NAs as it was the most relevant value. As for the categorical NAs, they were replaced with ‘None’. Once, all the NAs were taken care of, I used LabelEncoder to encode the categorical values. I, then, checked for a correlation between columns and dropped them based on domain knowledge.

Here are the 10 features I selected:

 'OverallQual',
 'YearBuilt',
 'TotalBsmtSF',
 'GrLivArea',
 'MasVnrArea',
 'BsmtFinType1',
 'Neighborhood',
 'GarageType',
 'SaleCondition',
 'BsmtExposure'

All the attributes are encoded and normalized before splitting into train and test with 80% train and 20% test.

**Milestone 2: 

For milestone 2, I ran an XGBoost Model with objective="reg:squarederror" and max_depth=3. The RMSE score is 28986.

**Milestone 3:

Reference:

https://github.com/adhok/streamlit_ames_housing_price_prediction_app/tree/main