Update README.md
Browse files
README.md
CHANGED
@@ -24,7 +24,39 @@ In this folder, we provide the following trained model checkpoints:
|
|
24 |
|
25 |
**TL;DR**: We present a method to infer physical properties of liquids from *just* the sound of pouring. We show in theory how *pitch* can be used to derive various physical properties such as container height, flow rate, etc. Then, we train a pitch detection network (`wav2vec2`) using simulated and real data. The resulting model can predict the physical properties of pouring liquids with high accuracy. The latent representations learned also encode information about liquid mass and container shape.
|
26 |
|
27 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
28 |
|
29 |
```python
|
30 |
from huggingface_hub import snapshot_download
|
@@ -35,6 +67,10 @@ snapshot_download(
|
|
35 |
)
|
36 |
```
|
37 |
|
|
|
|
|
|
|
|
|
38 |
## Models
|
39 |
|
40 |
We provide audio models trained to detect pitch in the sound of pouring water.
|
|
|
24 |
|
25 |
**TL;DR**: We present a method to infer physical properties of liquids from *just* the sound of pouring. We show in theory how *pitch* can be used to derive various physical properties such as container height, flow rate, etc. Then, we train a pitch detection network (`wav2vec2`) using simulated and real data. The resulting model can predict the physical properties of pouring liquids with high accuracy. The latent representations learned also encode information about liquid mass and container shape.
|
26 |
|
27 |
+
|
28 |
+
## 💻 Usage
|
29 |
+
|
30 |
+
First, install the repository from `github`.
|
31 |
+
|
32 |
+
```sh
|
33 |
+
git clone git@github.com:bpiyush/SoundOfWater.git
|
34 |
+
cd SoundOfWater
|
35 |
+
```
|
36 |
+
|
37 |
+
Then, install dependencies.
|
38 |
+
|
39 |
+
```sh
|
40 |
+
conda create -n sow python=3.8
|
41 |
+
conda activate sow
|
42 |
+
|
43 |
+
# Install desired torch version
|
44 |
+
# NOTE: change the version if you are using a different CUDA version
|
45 |
+
pip install torch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 --index-url https://download.pytorch.org/whl/cu121
|
46 |
+
|
47 |
+
# Additional packages
|
48 |
+
pip install lightning==2.1.2
|
49 |
+
pip install timm==0.9.10
|
50 |
+
pip install pandas
|
51 |
+
pip install decord==0.6.0
|
52 |
+
pip install librosa==0.10.1
|
53 |
+
pip install einops==0.7.0
|
54 |
+
pip install ipywidgets jupyterlab seaborn
|
55 |
+
|
56 |
+
# if you find a package is missing, please install it with pip
|
57 |
+
```
|
58 |
+
|
59 |
+
Then, use this snippet to download the models:
|
60 |
|
61 |
```python
|
62 |
from huggingface_hub import snapshot_download
|
|
|
67 |
)
|
68 |
```
|
69 |
|
70 |
+
To run our models on examples of pouring sounds, please see the [playground notebook](https://github.com/bpiyush/SoundOfWater/blob/main/playground.ipynb).
|
71 |
+
|
72 |
+
If you would like to use our dataset for a different task, please download it from [here](https://huggingface.co/datasets/bpiyush/sound-of-water).
|
73 |
+
|
74 |
## Models
|
75 |
|
76 |
We provide audio models trained to detect pitch in the sound of pouring water.
|