updates readme

README.md

@@ -1,3 +1,23 @@
----
-license: apache-2.0
----
+# Small BLOOM Model for Functional Testing
+## Description
+
+I've reduced the size [bloom](https://huggingface.co/bigscience/bloom) to roughly 0.5GB
+
+
+This repository hosts a significantly smaller version of the BLOOM model, designed primarily for functional testing purposes. It is an ideal choice for scenarios where computational efficiency and quick iterations are necessary, such as in development and testing environments.
+
+## Model Details
+
+The original BLOOM model has been scaled down with the following changes:
+
+- Number of Layers (n_layer): Reduced to 12 from the original 70 layers.
+- Hidden Size (hidden_size): Decreased to 512 from the original 14336.
+- Number of Attention Heads (n_head): Lowered to 8 from the original 112 heads.
+
+## Intended Use
+This model is suitable for functional testing and development purposes. It is not recommended for tasks that require high accuracy or complex language understanding and generation capabilities.
+
+## Disclaimer
+Please note that due to the significant reductions in size and complexity, this model does not retain the full capabilities of the original BLOOM model. Expect limitations in accuracy and depth of language understanding.
+
+crafted by Nasser Ali Alzahrani (@nassersala)

config.json

@@ -0,0 +1,25 @@
+{
+  "apply_residual_connection_post_layernorm": false,
+  "architectures": [
+    "BloomForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "attention_softmax_in_fp32": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "hidden_dropout": 0.0,
+  "hidden_size": 512,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "masked_softmax_fusion": true,
+  "model_type": "bloom",
+  "n_head": 8,
+  "n_layer": 12,
+  "pad_token_id": 3,
+  "pretraining_tp": 4,
+  "slow_but_exact": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.24.0",
+  "use_cache": true,
+  "vocab_size": 250880
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f4c412dcfe7f4fbe3a659603e7e97ec7778681652ad7d94b4ba5ff416afd9dd
+size 665171479