model v1.0

README.md

@@ -0,0 +1,133 @@
+------------
+## Arabic and English News Summarization NLP Model
+
+### About
+
+This model is for summarizing news stories in short highlights for both Arabic and English tasks.
+
+نموذج معرفي متخصص في تلخيص الأخبار العربية و الإنجليزية الى مجموعة من أهم النقاط
+
+### Fine-Tuning
+
+The model was finetuned using the [Arabic T5 Model](https://huggingface.co/bakrianoo/t5-arabic-large) which developed by [Abu Bakr Soliman](http://github.com/bakrianoo).
+
+The primary summarization model also developed by the same developer.
+
+### How to Use
+
+- You can use this [Colab Notebook](https://colab.research.google.com/drive/1DWND1CAfCXD4OxrfmLBEaKeXhjGmYkod?usp=sharing) to test the model
+
+1. Install [PyTorch](https://pytorch.org/)
+
+2. Install the following Python packages
+
+`$ pip3 install transformers==4.7.0 nltk==3.5 protobuf==3.15.3 sentencepiece==0.1.96`
+
+3. Run this code
+
+```python
+
+from transformers import AutoTokenizer, AutoModelWithLMHead
+import torch
+import nltk
+nltk.download('punkt')
+from nltk.tokenize import sent_tokenize
+
+device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
+m_name = "marefa-nlp/summarization-arabic-english-news"
+
+tokenizer = AutoTokenizer.from_pretrained(m_name)
+model = AutoModelWithLMHead.from_pretrained(m_name).to(device)
+
+def get_summary(text, tokenizer, model, device="cpu", num_beams=2):
+    if len(text.strip()) < 50:
+        return ["Please provide more longer text"]
+    
+    text = "summarize: <paragraph> " + " <paragraph> ".join([ s.strip() for s in sent_tokenize(text) if s.strip() != ""]) + " </s>"
+    text = text.strip().replace("\n","")
+    
+    tokenized_text = tokenizer.encode(text, return_tensors="pt").to(device)
+    
+    summary_ids = model.generate(
+            tokenized_text,
+            max_length=512,
+            num_beams=num_beams,
+            repetition_penalty=1.5, 
+            length_penalty=1.0, 
+            early_stopping=True
+    )
+    
+    output = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
+    return [ s.strip() for s in output.split("<hl>") if s.strip() != "" ]
+    
+## Prepare Samples
+
+samples = [
+    """
+        قال المدافع الإيطالي ليوناردو بونوتشي إن منتخب بلاده ليس خائفا من مواجهة نظيره الانجليزي على أرضه في المباراة النهائية في بطولة يورو 2020 لكرة القدم، في حين وصف المدافع الانجليزي جون ستونز المباراة المرتقبة بأنها ستكون "أكثر تميزا".
+        وسوف تقام المباراة في استاد ويمبلي، شمال غربي لندن، يوم الأحد.
+        وتسعى إيطاليا لإحراز اللقب الأوروبي للمرة الثانية بعد فوزها به أول مرة عام 1968.
+        ولم يفز الفريق الانجليزي بهذا اللقب القاري من قبل. والبطولة الرئيسية الوحيدة التي فازت بها انجلترا هي كأس العالم عام 1966 الذي أقيمت مباراته النهائية في استاد ويمبلي.
+    """,
+    
+    """
+        On a night fraught with tension, Italy clinched its first major title for 15 years with a penalty shootout win over England in the Euro 2020 final.
+        Luke Shaw's goal inside the opening two minutes gave England a lead it looked like it would hold onto all night, before a goalmouth scramble midway through the second half allowed Leonardo Bonucci to poke home an equalizer for Italy.
+        For the remainder of the match it felt as though extra-time and penalties were inevitable, as neither side seemed willing or brave enough to commit enough men forward to really trouble the opposing defenders.
+        England had suffered innumerable heartbreaks on penalties over the years and this time it was Italy's turn to inflict yet more pain on beleaguered English fans as Marcus Rashford, Jadon Sancho and Bukayo Saka all missed from the spot.
+    """,
+]
+
+## Get summariez
+
+print("Original Article:", samples[0])
+print("\n===========\nSummary: \n")
+hls = get_summary(samples[0], tokenizer, model, device)
+for hl in hls:
+    print("\t-", hl)
+    
+    
+print("Original Article:", samples[1])
+print("\n=========== \nSummary: \n")
+hls = get_summary(samples[1], tokenizer, model, device)
+for hl in hls:
+    print("\t-", hl)
+```
+
+Results
+
+```
+Original Article: 
+
+        قال المدافع الإيطالي ليوناردو بونوتشي إن منتخب بلاده ليس خائفا من مواجهة نظيره الانجليزي على أرضه في المباراة النهائية في بطولة يورو 2020 لكرة القدم، في حين وصف المدافع الانجليزي جون ستونز المباراة المرتقبة بأنها ستكون "أكثر تميزا".
+        وسوف تقام المباراة في استاد ويمبلي، شمال غربي لندن، يوم الأحد.
+        وتسعى إيطاليا لإحراز اللقب الأوروبي للمرة الثانية بعد فوزها به أول مرة عام 1968.
+        ولم يفز الفريق الانجليزي بهذا اللقب القاري من قبل. والبطولة الرئيسية الوحيدة التي فازت بها انجلترا هي كأس العالم عام 1966 الذي أقيمت مباراته النهائية في استاد ويمبلي.
+    
+
+===========
+Summary: 
+
+	- وسوف تواجه إيطاليا إنجلترا في بطولة يورو 2020 لكرة القدم يوم الأحد.
+	- ستقام المباراة في استاد ويمبلي، شمال غربي لندن، يوم الأحد.
+	- ولم يفز الفريق الانجليزي بهذا اللقب القاري قبل.
+	
+```
+
+```
+Original Article: 
+
+        On a night fraught with tension, Italy clinched its first major title for 15 years with a penalty shootout win over England in the Euro 2020 final.
+        Luke Shaw's goal inside the opening two minutes gave England a lead it looked like it would hold onto all night, before a goalmouth scramble midway through the second half allowed Leonardo Bonucci to poke home an equalizer for Italy.
+        For the remainder of the match it felt as though extra-time and penalties were inevitable, as neither side seemed willing or brave enough to commit enough men forward to really trouble the opposing defenders.
+        England had suffered innumerable heartbreaks on penalties over the years and this time it was Italy's turn to inflict yet more pain on beleaguered English fans as Marcus Rashford, Jadon Sancho and Bukayo Saka all missed from the spot.
+    
+
+=========== 
+Summary: 
+
+	- Italy beat England 1-0 in the Euro 2020 final.
+	- Luke Shaw's goal gave England a lead it looked like it would hold onto all night.
+	- Leonardo Bonucci scored the equalizer for Italy.
+	- Marcus Rashford, Jadon Sancho and Bukayo Saka all missed.
+```

config.json

@@ -0,0 +1,30 @@
+{
+  "_name_or_path": "./t5-full-models-12k/stat-checkpoint-6000",
+  "architectures": [
+    "T5ForConditionalGeneration"
+  ],
+  "d_ff": 2816,
+  "d_kv": 64,
+  "d_model": 1024,
+  "decoder_start_token_id": 0,
+  "dropout_rate": 0.1,
+  "eos_token_id": 1,
+  "feed_forward_proj": "gated-gelu",
+  "gradient_checkpointing": false,
+  "initializer_factor": 1.0,
+  "is_encoder_decoder": true,
+  "layer_norm_epsilon": 1e-06,
+  "model_type": "t5",
+  "num_decoder_layers": 24,
+  "num_heads": 16,
+  "num_layers": 24,
+  "output_past": true,
+  "pad_token_id": 0,
+  "relative_attention_num_buckets": 32,
+  "tie_word_embeddings": false,
+  "tokenizer_class": "T5Tokenizer",
+  "torch_dtype": "float32",
+  "transformers_version": "4.9.0.dev0",
+  "use_cache": true,
+  "vocab_size": 35000
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7d7d6bfb2959ba117cf44d11e0f7d33b175fce72b1f2f17458c4e150ccfeba57
+size 3156380325

scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e32bd296dbe0d71091a57a3210b6f53ad1fe87d520fb368e9cfdbc61eb5237bc
+size 623

special_tokens_map.json

@@ -0,0 +1 @@
+{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>"}

spiece.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b1638b19c09773fb382835ebed2a612ba15aceff181dd79703365aad4ae6821
+size 847294

tokenizer_config.json

@@ -0,0 +1 @@
+{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>", "extra_ids": 0, "additional_special_tokens": null, "sp_model_kwargs": {}, "special_tokens_map_file": "/root/.cache/huggingface/transformers/83fa3c1d6e6d11afc201e70615946ccb1b2464e8d577d134c3c97287f0ab5037.294ebaa4cd17bb284635004c92d2c4d522ec488c828dcce0c2471b6f28e3fe82", "name_or_path": "./t5-full-models-12k/stat-checkpoint-6000", "tokenizer_class": "T5Tokenizer"}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aec94cf0829b76ab2c3927d7eba16c7f880c0ab4079b02415b9e903b7c844883
+size 2735