ehsanaghaei commited on
Commit
2ff2376
·
1 Parent(s): 8736038

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -0
README.md CHANGED
@@ -1,3 +1,19 @@
1
  ---
2
  license: cc-by-nc-4.0
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-nc-4.0
3
+ language:
4
+ - en
5
+ tags:
6
+ - cybersecurity
7
  ---
8
+ This model represents an enhanced iteration of the [SecureBERT](https://huggingface.co/ehsanaghaei/SecureBERT) model, trained on a corpus eight times larger than its predecessor, over the course of 400 hours, leveraging the computational power of 8xA100 GPUs. The innovation, known as SecureBERT_Plus, brings forth an average improvment of 9% in the performance of the Masked Language Model (MLM) task. This advancement signifies a substantial stride towards achieving heightened proficiency in language understanding and representation learning within the cybersecurity knowledge.
9
+
10
+
11
+ SecureBERT is a domain-specific language model based on RoBERTa which is trained on a huge amount of cybersecurity data and fine-tuned/tweaked to understand/represent cybersecurity textual data.
12
+
13
+
14
+ Related cybersecurity language models:
15
+ [SecureGPT](https://huggingface.co/ehsanaghaei/SecureGPT)
16
+ [SecureDeBERTa](https://huggingface.co/ehsanaghaei/SecureDeBERTa)
17
+ [SecureBERT](https://huggingface.co/ehsanaghaei/SecureBERT)
18
+
19
+