prithivida
commited on
Commit
•
f208c18
1
Parent(s):
f304b34
added expansion image
Browse files
README.md
CHANGED
@@ -57,9 +57,9 @@ Getting pros of both searches made sense and that gave rise to interest in learn
|
|
57 |
|
58 |
4. What a Sparse model learns ?
|
59 |
|
60 |
-
The model learns to project it's learned dense representations over a MLM head to give a vocabulary distribution.
|
61 |
|
62 |
-
|
63 |
|
64 |
</details>
|
65 |
|
|
|
57 |
|
58 |
4. What a Sparse model learns ?
|
59 |
|
60 |
+
The model learns to project it's learned dense representations over a MLM head to give a vocabulary distribution. Which is just to say the model can do automatic token expansion. (Image courtesy of pinecone)
|
61 |
|
62 |
+
<img src="./expansion.png" width=650 height=500/>
|
63 |
|
64 |
</details>
|
65 |
|