Upload 11 files

README.md

@@ -1,166 +1,9 @@
 ---
-license: other
-license_name: cfai
-license_link: LICENSE
-language:
-- en
-metrics:
-- accuracy
-library_name: adapter-transformers
-pipeline_tag: text-generation
+tags:
+- autotrain
+- text-generation
+widget:
+- text: "I love AutoTrain because "
 ---
-# Model Card for Model ID
-If you're looking for a text generative model that is creative and generates human-like content and is memory-efficient this model might full-fill your needs give it a try!
 
-
-### Model Description
-NeXGen is a state-of-the-art text generative model designed to meet the demands of users seeking creativity and human-like content generation. This cutting-edge model is equipped with advanced capabilities that set it apart in the realm of natural language processing.
-One of NeXGen's standout features is its ability to generate highly creative and contextually relevant text. Whether you're looking to compose engaging stories, craft imaginative dialogues, or generate unique pieces of writing, NeXGen excels at producing content that mirrors the fluency and style of human expression. Its creative prowess extends to various genres and themes, making it a versatile tool for writers, content creators, and anyone in need of compelling textual output.
-Furthermore, NeXGen boasts an impressive understanding of context, allowing it to generate coherent and contextually appropriate responses. This contextual awareness enhances the model's ability to provide relevant information and maintain a natural flow in generated text, contributing to a more authentic and human-like reading experience.
-Users will find NeXGen to be an intuitive and user-friendly tool, allowing for seamless integration into various applications and platforms. Its user interface is designed for accessibility, ensuring that both novice and experienced users can harness the model's capabilities with ease.
-The model's architecture is underpinned by the latest advancements in natural language processing, leveraging sophisticated algorithms and vast datasets to achieve high-quality text generation. NeXGen's training regimen includes exposure to diverse linguistic patterns, enabling it to adapt to different writing styles and linguistic nuances.
-NeXGen is not only a powerful text generator but also a tool for enhancing productivity. Whether you need assistance in drafting creative content, brainstorming ideas, or generating textual prompts, NeXGen is equipped to facilitate the creative process and provide valuable support.
-In summary, NeXGen stands out as a go-to solution for those in search of a text generative model that excels in creativity, context awareness, and human-like content generation. Its user-friendly interface, advanced architecture, and versatility make it a valuable asset for a wide range of applications, offering users an innovative and efficient tool to meet their creative writing needs.
-
-
-
-- **Developed by:** [Sirclavin or (me)]
-- **Shared by:** [CrabfishAI]
-- **Model type:** [text generative model]
-- **Language(s) (NLP):** [English(EN)]
-- **License:** [CFAI]
-
-
-## Uses
-
-1. **Content Creation:**
-   - Generate creative writing, including stories, poetry, and fictional narratives.
-   - Produce marketing copy, ad content, and product descriptions with a natural and engaging tone.
-
-2. **Assistance in Writing:**
-   - Aid authors, journalists, and bloggers in brainstorming ideas and drafting articles.
-   - Provide assistance in generating outlines or summaries for writing projects.
-
-3. **Educational Support:**
-   - Assist students in generating ideas for essays, reports, or creative writing assignments.
-   - Offer language learning support by providing contextually relevant sentences and phrases.
-
-4. **Chatbot Development:**
-   - Power conversational agents and chatbots with human-like responses in customer service or information retrieval scenarios.
-
-5. **Prototyping and Idea Generation:**
-   - Facilitate brainstorming sessions by generating ideas and concepts for product development or problem-solving.
-
-6. **Social Media Content:**
-   - Generate engaging captions for social media posts, helping users maintain a consistent and appealing online presence.
-
-7. **Personal Assistant Applications:**
-   - Assist users in drafting emails, messages, or other forms of communication with a natural and personalized touch.
-
-8. **Entertainment and Gaming:**
-   - Enhance storytelling in video games by generating dynamic and contextually appropriate dialogues.
-   - Contribute to the creation of interactive fiction or game narratives.
-
-9. **Accessibility Features:**
-   - Provide support for individuals with disabilities, such as generating text for those who may have difficulty typing.
-
-10. **Innovative Writing Tools:**
-    - Integrate with writing platforms to offer advanced suggestions, corrections, and improvements to users' writing.
-
-11. **Research and Data Analysis:**
-    - Assist researchers in generating textual summaries or insights from large datasets.
-
-### Direct Use
-
-1. **Automated Email Drafting:**
-   - Quickly compose emails by providing key points, and let NeXGen generate a well-articulated message with appropriate language and tone.
-
-2. **Blog Post Generation:**
-   - Input a topic or key points, and NeXGen can assist in generating sections of a blog post or even an entire article.
-
-3. **Copywriting for Advertisements:**
-   - Generate creative and persuasive copy for advertisements, social media posts, or marketing materials.
-
-4. **Code Commenting:**
-   - Assist developers in generating clear and concise comments for their code to improve documentation and understanding.
-
-5. **Storyline Creation for Games:**
-   - Aid game developers in creating dynamic and engaging storylines or dialogues for characters within video games.
-
-6. **Learning Material Generation:**
-   - Develop study guides, flashcards, or educational content by providing key concepts to NeXGen.
-
-7. **Personal Journaling Assistance:**
-   - Generate prompts or suggestions for users looking to maintain a personal journal or diary.
-
-8. **In-app Chatbot Responses:**
-   - Power chatbots within applications with natural and context-aware responses for user interactions.
-
-9. **Social Media Status Updates:**
-   - Quickly generate interesting and varied status updates for platforms like Twitter or Facebook.
-
-10. **Scriptwriting Support:**
-    - Assist screenwriters in developing dialogue, scenes, or even plot points for film or television scripts.
-
-11. **Product Descriptions:**
-    - Generate compelling and informative product descriptions for e-commerce websites.
-
-12. **Idea Expansion:**
-    - Provide expanded ideas and details for creative projects, helping users flesh out their initial concepts.
-
-13. **Meeting Note Summaries:**
-    - Summarize meeting notes or discussions, condensing information into clear and coherent points.
-
-14. **Legal Document Drafting:**
-    - Assist legal professionals in generating preliminary drafts for contracts, agreements, or other legal documents.
-
-## How to get Started with the Model
-
-Use the code below to get started with the model:
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-model_name = "Sirclavin/NeXGen-based"
-model = AutoModelForCausalLM.from_pretrained(model_name)
-tokenizer = AutoTokenizer.from_pretrained(model_name)
-```
-
-And to use the model for text generation:
-```python
-from transformers import AutoTokenizer, AutoModelForCausalLM
-
-# Specify the model name from Hugging Face Model Hub
-model_name = "Sirclavin/NeXGen-based"
-tokenizer = AutoTokenizer.from_pretrained(model_name)
-model = AutoModelForCausalLM.from_pretrained(model_name)
-
-def generate_text(prompt, max_length=100, num_beams=5, no_repeat_ngram_size=2, top_k=50, top_p=0.95, temperature=0.7):
-    input_ids = tokenizer.encode(prompt, return_tensors="pt")
-
-    # Ensure attention_mask is provided
-    attention_mask = input_ids.ne(tokenizer.pad_token_id).float()
-
-    # Generate output text
-    output = model.generate(
-        input_ids,
-        max_length=max_length,
-        num_beams=num_beams,
-        no_repeat_ngram_size=no_repeat_ngram_size,
-        top_k=top_k,
-        top_p=top_p,
-        temperature=temperature,
-        attention_mask=attention_mask  # Pass attention_mask to the generation method
-    )
-
-    decoded_output = tokenizer.decode(output[0], skip_special_tokens=True)
-    return decoded_output
-
-# Example usage:
-prompt = "Your prompt here"
-generated_text = generate_text(prompt, max_length=200)
-
-print("Generated Text:")
-print(generated_text)
-```
-
-## Training Data
-the dataset used to train this model is unknown
+# Model Trained Using AutoTrain

added_tokens.json

@@ -0,0 +1,4 @@
+{
+  "<|pad|>": 50258,
+  "<|startoftext|>": 50257
+}

config.json

@@ -0,0 +1,20 @@
+{
+  "auto_mapping": null,
+  "base_model_name_or_path": "farnhua/gpt2-small-self_instruct_human_eval",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "revision": null,
+  "target_modules": [
+    "c_attn"
+  ],
+  "task_type": "CAUSAL_LM"
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60f3ea3765c1b02514bb4b684bd3d1f5004f1101b81bc7c09b3ba0df014ff825
+size 497805594

special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<|startoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|pad|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,38 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "50256": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50257": {
+      "content": "<|startoftext|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50258": {
+      "content": "<|pad|>",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|startoftext|>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "model_max_length": 1024,
+  "pad_token": "<|pad|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:451c10f29d391c5977d9401a129c75070c1f9e21c563dee1c87aee84480bdc81
+size 4472

training_params.json

@@ -0,0 +1,47 @@
+{
+    "model": "farnhua/gpt2-small-self_instruct_human_eval",
+    "data_path": "timdettmers/openassistant-guanaco",
+    "project_name": "NeXGen",
+    "train_split": "train",
+    "valid_split": null,
+    "text_column": "text",
+    "rejected_text_column": "rejected",
+    "token": null,
+    "lr": 0.0002,
+    "epochs": 3,
+    "batch_size": 2,
+    "warmup_ratio": 0.1,
+    "gradient_accumulation": 1,
+    "optimizer": "adamw_torch",
+    "scheduler": "linear",
+    "weight_decay": 0.0,
+    "max_grad_norm": 1.0,
+    "seed": 42,
+    "add_eos_token": false,
+    "block_size": -1,
+    "use_peft": true,
+    "lora_r": 16,
+    "lora_alpha": 32,
+    "lora_dropout": 0.05,
+    "logging_steps": -1,
+    "evaluation_strategy": "epoch",
+    "save_total_limit": 1,
+    "save_strategy": "epoch",
+    "auto_find_batch_size": false,
+    "fp16": false,
+    "push_to_hub": false,
+    "use_int8": false,
+    "model_max_length": 2048,
+    "repo_id": null,
+    "use_int4": true,
+    "trainer": "sft",
+    "target_modules": null,
+    "merge_adapter": false,
+    "username": null,
+    "use_flash_attention_2": false,
+    "log": "none",
+    "disable_gradient_checkpointing": false,
+    "model_ref": null,
+    "dpo_beta": 0.1,
+    "prompt_text_column": "prompt"
+}