shonenkov
commited on
Commit
•
9df973e
1
Parent(s):
e2c2a00
add readme
Browse files- README.md +28 -0
- emojich.png → pics/emojich_rgba.png +0 -0
- pics/emojich_rgba_100.png +0 -0
- pics/examples.png +0 -0
README.md
ADDED
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# Emojich
|
2 |
+
![](./pics/emojich_rgba_100.png)
|
3 |
+
### generate emojis from text
|
4 |
+
|
5 |
+
Model was trained by [Sber AI](https://github.com/sberbank-ai)
|
6 |
+
* Task: `text2image generation`
|
7 |
+
* Num Parameters: `1.3 B`
|
8 |
+
* Training Data Volume: `120 million text-image pairs` & [`2749 text-emoji pairs`](https://www.kaggle.com/shonenkov/russian-emoji)
|
9 |
+
|
10 |
+
|
11 |
+
### Model Description
|
12 |
+
😋 Emojich is a 1.3 billion params model from the family GPT3-like, it generates emoji-style images with the brain of ◾ Malevich.
|
13 |
+
|
14 |
+
|
15 |
+
### Fine-tuning stage:
|
16 |
+
|
17 |
+
The main goal of fine-tuning is trying to keep the generalization of [ruDALL-E Malevich (XL)](https://huggingface.co/sberbank-ai/rudalle-Malevich)
|
18 |
+
model on text to emoji tasks. ruDALL-E Malevich is a multi-modality big pretrained transformer, that uses images and texts.
|
19 |
+
The idea with freezing feedforward and self-attention layers in pretrained transformer is demonstrated high performance in changing different modalities.
|
20 |
+
Also, the model has a good chance for over-fitting text modality and lost generalization.
|
21 |
+
To deal with this problem is increased coefficient 10^3 in weighted cross-entropy loss for image codebooks part.
|
22 |
+
|
23 |
+
Full version of code is available on Kaggle [here](https://www.kaggle.com/shonenkov/emojich-rudall-e).
|
24 |
+
|
25 |
+
|
26 |
+
### Examples of generated emojis
|
27 |
+
|
28 |
+
![](./pics/examples.png)
|
emojich.png → pics/emojich_rgba.png
RENAMED
File without changes
|
pics/emojich_rgba_100.png
ADDED
![]() |
pics/examples.png
ADDED
![]() |