app

.gitignore

@@ -0,0 +1,129 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/

Dockerfile

@@ -0,0 +1,6 @@
+FROM python:3.9-slim
+EXPOSE 8501
+WORKDIR /app
+COPY . .
+RUN pip3 install -r requirements.txt
+ENTRYPOINT ["streamlit", "run", "streamlit_app.py", "--server.port=8501", "--server.address=0.0.0.0"]

README.md

@@ -1,13 +1,17 @@
----
-title: Weather App
-emoji: 👀
-colorFrom: gray
-colorTo: yellow
-sdk: streamlit
-sdk_version: 1.36.0
-app_file: app.py
-pinned: false
-license: apache-2.0
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Challenge General Information
+
+You can read the details of the challenge at [challenge.md](challenge.md)
+
+## Key Items
+
+- __Due Date:__ 7/23/2024
+- __Work Rules:__ You cannot work with others.  You can ask any question you want in our general channel. The teacher and TA are the only ones who can answer questions. __You cannot use code from other student's apps.__
+- __Product:__ A streamlit app that runs within Docker, builds from your repo, and is published on Hugging Face in our DS460 Org.
+- __Github Process:__ Each student will fork the challenge repository and create their app. They will submit a link to the Hugging Face app in Canvas.
+- __Canvas Process:__ Each student will upload a `.pdf` or `.html` file with your results as described in [challenge.md](challenge.md)
+
+
+## Notes & References
+
+- [Fork a repo](https://docs.github.com/en/get-started/quickstart/fork-a-repo)
+- [Creating a pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request)

challenge.md

@@ -0,0 +1,98 @@
+# API ↦ DATA ↦ DASHBOARD
+
+We use APIs to digest streaming data, and this challenge will require you to figure out an API and then build the data from that API into your Dashboard. Our current challenge is the [Open-Meteo API](https://open-meteo.com/). We learned about [Streamlit] (https://streamlit.io/) and [Docker] (https://www.docker.com/) for application development. We will build an app that uses the API to display weather data for decision-making.
+
+__NOTE!__
+
+They provide python code examples with the API.  You will need to tweak yours to use Polars and to not use the Caching.  See these two chunks for examples to prompt your coding (note the second one is not a complete code snippet).
+
+```python
+openmeteo = openmeteo_requests.Client()
+```
+
+
+```python
+start = datetime.fromtimestamp(hourly.Time(), timezone.utc)
+end = datetime.fromtimestamp(hourly.TimeEnd(), timezone.utc)
+freq = timedelta(seconds = hourly.Interval())
+
+df = pl.select(
+  date = pl.datetime_range(start, end, freq, closed = "left"),
+  temperature_2m = hourly_temperature_2m)
+```
+
+## Coding Challenge
+
+### Driving needs
+
+_Each of the items below must be addressed by your app._
+
+1. Allow the user to pull 15 days of [historical forecast data](https://open-meteo.com/en/docs/historical-forecast-api#start_date=2024-07-02) and [historical data](https://open-meteo.com/en/docs/historical-weather-api) for the three BYU locations (Idaho, Hawaii, Provo) at an hourly resolution.
+  - Make sure that the date and time is understandable by the user.
+  - Use display units for the United States of America.
+2. Display summary tables comparing the forecasts and actual weather for all three cities that lets them compare;
+  - The three cities to each other.
+  - The forecast to the actual weather at each city.
+3. Provide multiple visualizations to facilitate these comparisons.
+  - Daily highs over the month for each location.   
+  - A visualization that uses boxplots to show each city's varied hourly temperature readings.
+  - A novel visualization of your creation (no bar charts or pie charts)  
+4. Build KPIs into your dashboard.
+  - Provide KPIs that show the highest value for the selected variable in that period and the respective city.
+  - Provide KPIs that show the lowest value for the selected variable in that period and the respective city.
+5. Include the following user elements in your dashboard.
+  - Allow the user to pick the 15 days they want to compare within the limits of the API.
+  - Allow the user user pick the weather variables of interest from at least ten different options of the API.
+  - Ask the user for their time zone choice (Hawaii or Mountain). Display all data in that time zone.
+  - Add two additional user inputs of your choice (for example; change the graph, additional API inputs, different summaries for your table)  
+
+### Data Science Dashboard
+
+We will use Streamlit as our prototype dashboard tool, but we need to embed that streamlit app into a Docker container.
+
+Within this repository you can simply run `docker compose up` to leverage the `docker-compose.yaml` with your local folder synced with the container folder where the streamlit app is runnning. 
+
+Additionally, you can use `docker build -t streamlit .` to use the `Dockerfile` to build the image and then use `docker run -p 8501:8501 -v "$(pwd):/app:rw" streamlit` to start the container with the appropriate port and volume settings.
+
+### Repo Structure
+
+Your repo should be built so that I can clone the repo and run the Docker command (`docker compose up`) as described in your `readme.md` that allows me to see your app in my web browser without requiring me to install Streamlit on my computer.
+
+1. Fork this repo to your private space
+2. Add me to your private repo in your space (`hathawayj`)
+3. Build your app and Docker container
+4. Update your `readme.md` with details about your app and how to start it.
+5. Include a screen shot of your working app in your repository.
+6. Build your app in Hugging Face using the [ds460 template](https://huggingface.co/spaces/ds460/_template_streamlit_docker).
+  - place the app within our ds460 org.
+  - Change the title of the App to your name and change the color.
+7. Submit the link to your repo and Hugging Face App to me in Canvas within your vocabulary/lingo challenge.
+
+## Vocabulary/Lingo Challenge
+
+_Within a `.md` file in your repository and as a submitted `.pdf` or `.html` on Canvas, address the following items;_
+
+1. A link to your repo that you have shared with me and a screenshot of your app.
+2. Explain the added value of using DataBricks in your Data Science process (using text, diagrams, or tables).
+3. Compare and contrast PySpark to either Pandas or the Tidyverse (using text, diagrams, or tables).
+4. Explain Docker to somebody intelligent but not a tech person (using text, diagrams, or tables).
+
+Your answers should be clear, detailed, and no longer than is needed. Imagine you are responding to a client or as an interview candidate._
+
+- _Clear:_ Clean sentences in nicely laid out format.
+- _Detailed:_ You touch on all the critical points of the concept. Speak at a reasonable level.
+- _Brevity:_ Don't ramble. Get to the point, and don't repeat yourself.
+
+## Submission
+
+Submit your work on Canvas.  It can be a link to your repository or a PDF/HTML file with your vocabulary challenge and a link to your repo included.
+
+## References
+
+- [Streamlit Dashboard](https://streamlit.io/)
+- [Docker](https://www.docker.com/)
+- [Dockerfile cheat sheet](https://kapeli.com/cheat_sheets/Dockerfile.docset/Contents/Resources/Documents/index)
+- [Streamlit deploy in Docker](https://docs.streamlit.io/knowledge-base/tutorials/deploy/docker)
+- [Streamlit and Docker](https://maelfabien.github.io/project/Streamlit/#)
+- [open-meteo/python-requests: Open-Meteo Python Library using `requests`](https://github.com/open-meteo/python-requests?tab=readme-ov-file#polars)
+- [openmeteo-requests · PyPI](https://pypi.org/project/openmeteo-requests/)

docker-compose.yaml

@@ -0,0 +1,12 @@
+services:
+  streamlit:
+    build:
+      dockerfile: Dockerfile
+      context: .
+    container_name: streamlit-app
+    cpus: 0.5
+    mem_limit: 1024m
+    ports:
+      - "8501:8501"
+    volumes:
+      - ".:/app:rw"

requirements.txt

@@ -0,0 +1,8 @@
+pandas
+polars
+streamlit
+plotly
+pytz
+openmeteo_requests
+datetime
+streamlit_letsplot

streamlit_app.py

@@ -0,0 +1,241 @@
+import openmeteo_requests
+import requests_cache
+import polars as pl
+from retry_requests import retry
+import streamlit as st
+from datetime import datetime, timedelta
+import pytz
+from lets_plot import *
+from streamlit_letsplot import st_letsplot
+import numpy as np 
+#API Fetch
+openmeteo = openmeteo_requests.Client()
+
+#Empty Frames
+locations = [
+    {"name": "Rexburg, Idaho", "latitude": 43.8251, "longitude": -111.7924},
+    {"name": "Provo, Utah", "latitude": 40.2338, "longitude": -111.6585},
+    {"name": "Laie, Hawaii", "latitude": 21.6210, "longitude": -157.9753}
+]
+location_names = [location["name"] for location in locations]
+filtered_forecasts = {}
+filtered_histories = {}
+historical_data = []
+forecast_data = []
+timezones = pytz.all_timezones
+def find_max(df):
+       daily_highs = df.group_by(by = 'Date').agg(pl.max("Temperature").alias('max'))
+       return daily_highs, df
+
+
+# Date Variables
+today = datetime.today()
+default = today - timedelta(days=14)
+
+# Streamlit Variables
+start_date = st.sidebar.date_input(
+    'Select start date:',
+    value = default,
+    max_value= default
+)
+
+selected_timezone = st.sidebar.selectbox(
+    'Choose a timezone:',
+    timezones
+)
+end_date = start_date + timedelta(days=14)
+
+temperature_option = st.sidebar.selectbox(
+    'Select Temperature Type',
+    ('Highest', 'Lowest')
+)
+
+city_option = st.sidebar.selectbox(
+    'Select City',
+    ('Rexburg, Idaho', 'Provo, Utah', 'Laie, Hawaii')
+)
+# Display the selected date range
+st.sidebar.write(f"Date Range: {start_date} to {end_date}")
+
+# Get Forecast Data
+for location in locations:
+    url = "https://api.open-meteo.com/v1/forecast"
+    params = {
+        "latitude": location["latitude"],
+        "longitude": location["longitude"],
+        "start_date": start_date.strftime('%Y-%m-%d'),
+        "end_date": end_date.strftime('%Y-%m-%d'),
+        "hourly": "temperature_2m"
+    }
+
+    # Fetch weather data
+    responses = openmeteo.weather_api(url, params=params)
+    response = responses[0]
+    hourly = response.Hourly()
+    hourly_temperature_2m = hourly.Variables(0).ValuesAsNumpy()
+
+    # Convert timestamp to datetime
+    start = datetime.fromtimestamp(hourly.Time())
+    end = datetime.fromtimestamp(hourly.TimeEnd())
+    freq = timedelta(seconds=hourly.Interval())
+
+    # Create Polars DataFrame
+    hourly_data = pl.select(
+  date = pl.datetime_range(start, end, freq, closed = "left"),
+  temperature_2m = hourly_temperature_2m,
+  location = [location["name"]])
+
+    hourly_dataframe = pl.DataFrame(data = hourly_data)
+
+
+    historical_data.append(hourly_dataframe)
+# Concatenate all DataFrames
+combined_historical = pl.DataFrame(pl.concat(historical_data)).explode('location')
+
+
+
+# Get True Historical Data
+for location in locations:
+    url = "https://archive-api.open-meteo.com/v1/archive"
+    params = {
+        "latitude": location["latitude"],
+        "longitude": location["longitude"],
+        "start_date": start_date.strftime('%Y-%m-%d'),
+        "end_date": end_date.strftime('%Y-%m-%d'),
+        "hourly": "temperature_2m"
+    }
+
+    # Fetch weather data
+    responses = openmeteo.weather_api(url, params=params)
+    response = responses[0]
+    hourly = response.Hourly()
+    hourly_temperature_2m = hourly.Variables(0).ValuesAsNumpy()
+
+    # Convert timestamp to datetime
+    start = datetime.fromtimestamp(hourly.Time())
+    end = datetime.fromtimestamp(hourly.TimeEnd())
+    freq = timedelta(seconds=hourly.Interval())
+
+    # Create Polars DataFrame
+    hourly_data = pl.select(
+  date = pl.datetime_range(start, end, freq, closed = "left"),
+  temperature_2m = hourly_temperature_2m,
+  location = [location["name"]])
+
+    hourly_dataframe = pl.DataFrame(data = hourly_data)
+
+
+    forecast_data.append(hourly_dataframe)
+
+combined_forecast = pl.DataFrame(pl.concat(forecast_data)).explode('location')
+
+for name in location_names:
+    filtered_forecasts[name] = (combined_forecast.filter(pl.col("location") == name).drop('location').rename({'date': 'Date'}).rename({'temperature_2m': 'Temperature'}).with_columns(pl.col('Temperature') * 9/5 + 32))
+    filtered_histories[name] = (combined_historical.filter(pl.col("location") == name).drop('location').rename({'date': 'Date'}).rename({'temperature_2m': 'Temperature'}).with_columns(pl.col('Temperature') * 9/5 + 32))
+
+
+tab1, tab2, tab3 = st.tabs(["Data", "Visualisations", "KPIs"])
+
+with tab1:
+  st.title("Forecasted Weather vs Actual Weather by City")
+  st.header('Forecasts')
+  st.markdown("<h2 style='text-align: center; color: white;'>Rexburg</h2>", unsafe_allow_html=True)
+
+# Create two columns for Rexburg content
+  rexburg_col1, rexburg_col2 = st.columns(2)
+
+# Rexburg content
+  with rexburg_col1:
+    st.write("Forecasts")
+    st.dataframe(filtered_forecasts["Rexburg, Idaho"], use_container_width=True, hide_index=True)
+
+  with rexburg_col2:
+    st.write("Historical Data")
+    st.dataframe(filtered_histories["Rexburg, Idaho"], use_container_width=True, hide_index=True)
+
+# Provo Header
+  st.markdown("<h2 style='text-align: center; color: white;'>Provo</h2>", unsafe_allow_html=True)
+
+# Create two columns for Provo content
+  provo_col1, provo_col2 = st.columns(2)
+
+# Provo content
+  with provo_col1:
+    st.write("Forecasts")
+    st.dataframe(filtered_forecasts["Provo, Utah"], use_container_width=True, hide_index=True)
+
+  with provo_col2:
+    st.write("Historical Data")
+    st.dataframe(filtered_histories["Provo, Utah"], use_container_width=True, hide_index=True)
+
+# Laie Header
+  st.markdown("<h2 style='text-align: center; color: white;'>Laie</h2>", unsafe_allow_html=True)
+
+# Create two columns for Laie content
+  laie_col1, laie_col2 = st.columns(2)
+
+# Laie content
+  with laie_col1:
+    st.write("Forecasts")
+    st.dataframe(filtered_forecasts["Laie, Hawaii"], use_container_width=True, hide_index=True)
+
+
+  with laie_col2:
+    st.write("Historical Data")
+    st.dataframe(filtered_histories["Laie, Hawaii"], use_container_width=True, hide_index=True)
+
+with tab2: 
+    st.header('Visualisations by City')
+
+
+
+    st.subheader("Forecasted Data vs. Historical Data")
+    
+    combined_forecast = combined_forecast.rename({'date': 'Date'}).rename({'temperature_2m': 'Temperature'}).with_columns(pl.col('Temperature') * 9/5 + 32).with_columns(pl.col('Date').cast(datetime))
+    combined_historical = combined_historical.rename({'date': 'Date'}).rename({'temperature_2m': 'Temperature'}).with_columns(pl.col('Temperature') * 9/5 + 32).with_columns(pl.col('Date').cast(datetime))
+
+    reg_forecasted = ggplot(combined_forecast, aes(x='Date', y='Temperature', color = 'location')) \
+        + geom_line() \
+        + facet_wrap('location', ncol = 1) \
+        + labs(title = f'Hourly Temperatures',
+            x = 'Date', y = 'Temperature (°F)', color = 'City Name') + \
+            guides(color="none") + \
+            scale_x_datetime(format = '%m/%d')
+    
+    reg_historical = ggplot(combined_historical, aes(x='Date', y='Temperature', color = 'location')) \
+        + geom_line() \
+        + facet_wrap('location', ncol = 1) \
+        + labs(title = f'Hourly Temperatures',
+            x = 'Date', y = 'Temperature (°F)', color = 'City Name') + \
+            guides(color="none") + \
+            scale_x_datetime(format = '%m/%d')
+    
+    st.subheader('Forecasted')
+    st_letsplot(reg_forecasted)
+    st.subheader('Historical')
+    st_letsplot(reg_historical)
+    st.subheader('Boxplot of Average Hourly Temperature')
+    box_plot = ggplot(combined_forecast, aes(x='location', y='Temperature', color = 'location')) + \
+                geom_jitter(alpha = .65) + \
+                geom_boxplot(alpha = .9) + \
+                labs(title=f'Hourly Temperature Readings',
+                 x='City', y='Temperature (°F)') + \
+                 guides(color = "none")
+    st_letsplot(box_plot)
+    maxes = combined_historical.group_by('location').agg((pl.col('Temperature').max()))
+    max_plot = ggplot(maxes, aes(x = 'location', y = 'Temperature', color = 'location')) + \
+                geom_bar(stat='identity') + \
+                labs(x = 'City Name', y = 'Max Temperature in Month (°F)')
+    st.subheader('Maximum Temperature for Period')
+    st_letsplot(max_plot)
+
+with tab3:
+   if temperature_option == 'Highest':
+
+    highest_temp = combined_historical.filter(pl.col('location') == city_option).select(pl.max('Temperature')).item()
+    st.metric(label=f"Highest Temperature in {city_option}", value=f"{round(highest_temp,2)} °F")
+   else:
+    # Get the lowest temperature and its corresponding date
+    lowest_temp = combined_historical.filter(pl.col('location') == city_option).select(pl.min('Temperature')).item()
+    st.metric(label=f"Highest Temperature in {city_option}", value=f"{round(lowest_temp,2)} °F")
+