Update README.md

README.md

@@ -12,3 +12,9 @@ tags:
 
 # Evaluation
 see <https://github.com/GermanT5/german-t5-eval>
+
+# Tips for training on GPUs
+This model is too big to fit on a normal 16GB GPU in FP32 mode.
+For various reasons, T5 models cannot be trained in FP16 mode.
+However, mixed precision training is not yet supported on many GPUs. 
+For example, it does not work on V100 GPUs. On A100, however, it does.