---
license: apache-2.0
datasets:
- SRDdev/Youtube-Scripts
language:
- en
pipeline_tag: text-generation
widget:
- text: Introduction to Vertex AI Feature Store
  example_title: Example 1
- text: What are Kubeflow Components?
  exmaple_title: Example 2
tags:
- Text-Generation
---

# SCRIPTGPT

Pretrained model on the English language using a causal language modeling (CLM) objective. It was introduced in
[this paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)
and first released at [this page](https://openai.com/blog/better-language-models/).

## Model description
ScriptGPT is a language model trained on a dataset of 5,000 YouTube videos that explain artificial intelligence (AI) concepts.
ScriptGPT is a Causal language transformer. The model resembles the GPT2 architecture, 
the model is a Causal Language model meaning it predicts the probability of a sequence of words based on the preceding words in the sequence. 
It generates a probability distribution over the next word given the previous words, without incorporating future words.

The goal of ScriptGPT is to generate scripts for AI videos that are coherent, informative, and engaging. 
This can be useful for content creators who are looking for inspiration or who want to automate the process of generating video scripts. 
To use ScriptGPT, users can provide a prompt or a starting sentence, and the model will generate a sequence of words that follow the context and style of the training data.

The current model is the smallest one with 124 million parameters (ScriptGPT)

More models are coming soon...

## Intended uses
The intended uses of ScriptGPT include generating scripts for videos that explain artificial intelligence concepts, providing inspiration for content creators, and 
automating the process of generating video scripts. 


## How to use
You can use this model directly with a pipeline for text generation.

1. __Load Model__
```python
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("SRDdev/Script_GPT")
model = AutoModelForCausalLM.from_pretrained("SRDdev/Script_GPT")
```

2. __Pipeline__
```python
from transformers import pipeline
generator = pipeline('text-generation', model= model , tokenizer=tokenizer)

context = "Introduction to Vertex AI Feature Store"
length_to_generate = 200 

script = generator(context, max_length=length_to_generate, do_sample=True)[0]['generated_text']
```
<p style="opacity: 0.8">Keeping the context more technical and related to AI will generate better outputs</p>

## Limitations and bias
> The model is trained on Youtube Scripts and will work better for that. It may also generate random information and users should be aware of that and cross-validate the results.

The used is linked [here](https://www.kaggle.com/datasets/jfcaro/5000-transcripts-of-youtube-ai-related-videos)

## Citations
```
@model{
        Name=Shreyas Dixit
        framework=Pytorch
        Year=Jan 2023
        Pipeline=text-generation
        Github=https://github.com/SRDdev
        LinkedIn=https://www.linkedin.com/in/srddev
      }
```