shonenkov commited on
Commit
9df973e
•
1 Parent(s): e2c2a00

add readme

Browse files
README.md ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Emojich
2
+ ![](./pics/emojich_rgba_100.png)
3
+ ### generate emojis from text
4
+
5
+ Model was trained by [Sber AI](https://github.com/sberbank-ai)
6
+ * Task: `text2image generation`
7
+ * Num Parameters: `1.3 B`
8
+ * Training Data Volume: `120 million text-image pairs` & [`2749 text-emoji pairs`](https://www.kaggle.com/shonenkov/russian-emoji)
9
+
10
+
11
+ ### Model Description
12
+ 😋 Emojich is a 1.3 billion params model from the family GPT3-like, it generates emoji-style images with the brain of ◾ Malevich.
13
+
14
+
15
+ ### Fine-tuning stage:
16
+
17
+ The main goal of fine-tuning is trying to keep the generalization of [ruDALL-E Malevich (XL)](https://huggingface.co/sberbank-ai/rudalle-Malevich)
18
+ model on text to emoji tasks. ruDALL-E Malevich is a multi-modality big pretrained transformer, that uses images and texts.
19
+ The idea with freezing feedforward and self-attention layers in pretrained transformer is demonstrated high performance in changing different modalities.
20
+ Also, the model has a good chance for over-fitting text modality and lost generalization.
21
+ To deal with this problem is increased coefficient 10^3 in weighted cross-entropy loss for image codebooks part.
22
+
23
+ Full version of code is available on Kaggle [here](https://www.kaggle.com/shonenkov/emojich-rudall-e).
24
+
25
+
26
+ ### Examples of generated emojis
27
+
28
+ ![](./pics/examples.png)
emojich.png → pics/emojich_rgba.png RENAMED
File without changes
pics/emojich_rgba_100.png ADDED
pics/examples.png ADDED