ai_dad_jokes (#1)

- a bunch of updates (170b8e888cc710ec06cf9e693a5aebd1b018bee9)
- cleaning (7644d2d57f75ba9c7929e3a298b91186e251794f)
- Swap out the UI elements (a558847a6f188dac5c2bb8c06cc9cc93c6291c98)
- Bring up advanced settings (504ac731e06683ca4b4899598f7f5999058dfcc8)


Co-authored-by: Apolinário from multimodal AI art <multimodalart@users.noreply.huggingface.co>

README.md

@@ -1,5 +1,5 @@
 ---
-title: Meme it
+title: AI Dad Jokes
 emoji: 🐨
 colorFrom: red
 colorTo: blue

app_dialogue.py

@@ -1,3 +1,4 @@
+import ast
 import copy
 import glob
 import hashlib
@@ -35,7 +36,7 @@ SYSTEM_PROMPT = [
 In the following interactions, User and Assistant will converse in natural language, and Assistant will answer in a sassy way.
 Assistant's main purpose is to create funny meme texts from the images User provides.
 Assistant should be funny, sassy, and impertinent, and sometimes Assistant roasts people.
-Assistant should not be mean. It should not say toxic, homophobic, sexist, racist, or any demeaning things that can make people uncomfortable.
+Assistant should not be mean. It should not say toxic, homophobic, sexist, racist, things or any demeaning things that can make people uncomfortable.
 Assistant was created by Hugging Face.
 
 Here's a conversation example:""",
@@ -323,31 +324,36 @@ def format_user_prompt_with_im_history_and_system_conditioning(
 # problematic_callback = gr.CSVLogger()
 
 textbox = gr.Textbox(
-    # placeholder="Upload an image and send a message",
+    placeholder="Upload an image and start conversing by sending a message! You can add an image at each turn, but don't have to.",
     show_label=False,
-    value="Write a meme for that image.",
+    # value="Write something funny about that image.",
     visible=True,
     container=False,
     label="Text input",
     scale=6,
     max_lines=5,
 )
-with gr.Blocks(title="D", theme=gr.themes.Base()) as demo:
-    gr.HTML("""<h1 align="center">Meme it</h1>""")
-    # with gr.Row(variant="panel"):
-    #     with gr.Column(scale=1):
-    #         gr.Image(IDEFICS_LOGO, elem_id="banner-image", show_label=False, show_download_button=False)
-    #     with gr.Column(scale=5):
-    #         gr.HTML("""
-    #             <p>This demo showcases <strong>IDEFICS</strong>, a open-access large visual language model. Like GPT-4, the multimodal model accepts arbitrary sequences of image and text inputs and produces text outputs. IDEFICS can answer questions about images, describe visual content, create stories grounded in multiple images, etc.</p>
-    #             <p>IDEFICS (which stands for <strong>I</strong>mage-aware <strong>D</strong>ecoder <strong>E</strong>nhanced à la <strong>F</strong>lamingo with <strong>I</strong>nterleaved <strong>C</strong>ross-attention<strong>S</strong>) is an open-access reproduction of <a href="https://huggingface.co/papers/2204.14198">Flamingo</a>, a closed-source visual language model developed by Deepmind. IDEFICS was built solely on publicly available data and models. It is currently the only visual language model of this scale (80 billion parameters) that is available in open-access.</p>
-    #             <p>📚 The variants available in this demo were fine-tuned on a mixture of supervised and instruction fine-tuning datasets to make the models more suitable in conversational settings. For more details, we refer to our <a href="https://huggingface.co/blog/idefics">blog post</a>.</p>
-    #             <p>🅿️ <strong>Intended uses:</strong> This demo along with the <a href="https://huggingface.co/models?sort=trending&amp;search=HuggingFaceM4%2Fidefics">supporting models</a> are provided as research artifacts to the community. We detail misuses and out-of-scope uses <a href="https://huggingface.co/HuggingFaceM4/idefics-80b#misuse-and-out-of-scope-use">here</a>.</p>
-    #             <p>⛔️ <strong>Limitations:</strong> The model can produce factually incorrect texts, hallucinate facts (with or without an image) and will struggle with small details in images. While the model will tend to refuse answering questionable user requests, it can produce problematic outputs (including racist, stereotypical, and disrespectful texts), in particular when prompted to do so. We encourage users to read our findings from evaluating the model for potential biases in the <a href="https://huggingface.co/HuggingFaceM4/idefics-80b#bias-evaluation">model card</a>.</p>
-    #         """)
-
-    # with gr.Row():
-    #     with gr.Column(scale=2):
+chatbot = gr.Chatbot(
+    elem_id="chatbot",
+    label="IDEFICS",
+    visible=True,
+    height=750,
+    avatar_images=[None, BOT_AVATAR]            
+)
+
+with gr.Blocks(title="AI Dad Jokes", theme=gr.themes.Base()) as demo:
+    gr.HTML("""<h1 align="center">AI Dad Jokes</h1>""")
+    with gr.Row(variant="panel"):
+        with gr.Column(scale=1):
+            gr.Image(IDEFICS_LOGO, elem_id="banner-image", show_label=False, show_download_button=False)
+        with gr.Column(scale=5):
+            gr.HTML("""
+                <p><strong>AI Dad Jokes</strong> is an AI system that writes humorous content inspired by images. Whether that's crafting memes, sharing light-hearted yet amiable jests, or playfully witty remarks, AI Dad Jokes assists you in creating delightful jokes!</p>
+                <p>AI Dad Jokes is powered by <a href="https://huggingface.co/blog/idefics">IDEFICS</a>, an open-access large visual language model developped by Hugging Face. Like GPT-4, the multimodal model accepts arbitrary sequences of image and text inputs and produces text outputs. IDEFICS can answer questions about images, describe visual content, create stories grounded in multiple images, etc.</p>
+
+                <p>⛔️ <strong>Intended uses and limitations:</strong> This demo is provided as research artifact to the community showcasing IDEFIC's capabilities. We detail misuses and out-of-scope uses <a href="https://huggingface.co/HuggingFaceM4/idefics-80b#misuse-and-out-of-scope-use">here</a>. In particular, the system should not be used to engage in harassment, abuse and bullying. The model can produce factually incorrect texts, hallucinate facts (with or without an image) and will struggle with small details in images. While the system will tend to refuse answering questionable user requests, it can produce problematic outputs (including racist, stereotypical, and disrespectful texts), in particular when prompted to do so.</p>
+            """)
+
     with gr.Row(elem_id="model_selector_row"):
         model_selector = gr.Dropdown(
             choices=MODELS,
@@ -359,137 +365,88 @@ with gr.Blocks(title="D", theme=gr.themes.Base()) as demo:
             visible=False,
         )
 
-
-
     with gr.Row():
-        # def prefetch_images_in_history(user_prompt_str):
-        #     """
-        #     Pre-fetch the images that are passed in the chatbot default history.
-        #     """
-        #     return prompt_list_to_markdown(handle_manual_images_in_user_prompt(user_prompt_str))
         with gr.Column():
             imagebox = gr.Image(type="filepath", label="Image input", visible=True)
+            with gr.Group():
+                with gr.Row():
+                    textbox.render()
+                    submit_btn = gr.Button(value="▶️ Submit", visible=True)
+                with gr.Row():
+                    clear_btn = gr.ClearButton([textbox, imagebox, chatbot], value="🧹 Clear")
+                    regenerate_btn = gr.Button(value="🔄 Regenerate", visible=True)
+                    upload_btn = gr.UploadButton("📁 Upload image", file_types=["image"],visible=False)
+            with gr.Accordion("Advanced settings", open=False, visible=True) as parameter_row:
+                system_prompt = gr.Textbox(
+                    value=SYSTEM_PROMPT,
+                    visible=False,
+                    lines=20,
+                    max_lines=50,
+                    interactive=True,
+                )
+                max_new_tokens = gr.Slider(
+                    minimum=8,
+                    maximum=256,
+                    value=64,
+                    step=1,
+                    interactive=True,
+                    label="Maximum number of new tokens to generate",
+                )
+                repetition_penalty = gr.Slider(
+                    minimum=0.0,
+                    maximum=5.0,
+                    value=1.2,
+                    step=0.01,
+                    interactive=True,
+                    label="Repetition penalty",
+                    info="1.0 is equivalent to no penalty",
+                )
+                decoding_strategy = gr.Radio(
+                    [
+                        "Greedy",
+                        "Top P Sampling",
+                    ],
+                    value="Top P Sampling",
+                    label="Decoding strategy",
+                    interactive=True,
+                    info="Higher values is equivalent to sampling more low-probability tokens.",
+                )
+                temperature = gr.Slider(
+                    minimum=0.0,
+                    maximum=5.0,
+                    value=0.6,
+                    step=0.1,
+                    interactive=True,
+                    visible=True,
+                    label="Sampling temperature",
+                    info="Higher values will produce more diverse outputs.",
+                )
+                decoding_strategy.change(
+                    fn=lambda selection: gr.Slider.update(
+                        visible=(
+                            selection in ["contrastive_sampling", "beam_sampling", "Top P Sampling", "sampling_top_k"]
+                        )
+                    ),
+                    inputs=decoding_strategy,
+                    outputs=temperature,
+                )
+                top_p = gr.Slider(
+                    minimum=0.01,
+                    maximum=0.99,
+                    value=0.8,
+                    step=0.01,
+                    interactive=True,
+                    visible=True,
+                    label="Top P",
+                    info="Higher values is equivalent to sampling more low-probability tokens.",
+                )
+                decoding_strategy.change(
+                    fn=lambda selection: gr.Slider.update(visible=(selection in ["Top P Sampling"])),
+                    inputs=decoding_strategy,
+                    outputs=top_p,
+                )
         with gr.Column():
-            chatbot = gr.Chatbot(
-                elem_id="chatbot",
-                label="IDEFICS",
-                visible=True,
-                height=750,
-                avatar_images=[None, BOT_AVATAR]
-                #                 value=[
-                #                     [
-                #                         (
-                #                             prefetch_images_in_history(
-                #                                 """Write a funny story including the following characters.
-                # There should be drama and most importantly romance.
-                # Character 1:<fake_token_around_image><image:https://m.media-amazon.com/images/M/MV5BODg1OTczMWEtNTU3MS00OTUzLThjODEtNDg1MWQwZmExYmFlXkEyXkFqcGdeQWFybm8@._V1_QL75_UX500_CR0,0,500,281_.jpg><fake_token_around_image>
-                # Character 2:<fake_token_around_image><image:https://static.wikia.nocookie.net/dreamworks/images/0/09/Shrek_dreamworks.png/revision/latest?cb=20170921225439><fake_token_around_image>
-                # Character 3:<fake_token_around_image><image:https://upload.wikimedia.org/wikipedia/en/e/ef/Marsupilami_1.jpg><fake_token_around_image>
-                # The plot should take place in the world pictured here:<fake_token_around_image><image:https://www.astronomy.com/wp-content/uploads/sites/2/2021/09/ON1.jpg><fake_token_around_image>"""
-                #                             )
-                #                         ),
-                #                         """Once upon a time, in a world where the sky was a dome and the ground was covered in grass, there lived a beautiful princess named Mulan. She was kind, brave, and always ready to help others. One day, while walking through the forest, she stumbled upon a strange creature. It was a green ogre named Shrek, who was lost and looking for his way back home.
-                # Mulan, being the kind-hearted person she was, decided to help Shrek find his way back. As they traveled together, they began to develop a strong bond. Shrek was impressed by Mulan's bravery and determination, while Mulan admired Shrek's loyalty and sense of humor.
-                # However, their journey was not without obstacles. They encountered a mischievous cat named Hoboua, who was always causing trouble and getting into fights. Despite his troublemaking ways, Hoboua had a good heart and eventually became a valuable ally to Mulan and Shrek.
-                # As they traveled, they faced many challenges, including battling fierce dragons and navigating treacherous mountains. Through it all, Mulan and Shrek grew closer, and their feelings for each other deepened.
-                # Finally, they reached Shrek's home, and he was reunited with his family and friends. Mulan, however, was sad to leave him behind. But Shrek had a surprise for her. He had fallen in love with her and wanted to be with her forever.
-                # Mulan was overjoyed, and they shared a passionate kiss. From that day on, they lived happily ever after, exploring the world together and facing any challenges that came their way.
-                # And so, the story of Mulan and Shrek's romance came to an end, leaving a lasting impression on all who heard it.""",
-                #                     ],
-                #                 ],
-            )
-
-    with gr.Group():
-        with gr.Row():
-            textbox.render()
-            submit_btn = gr.Button(value="▶️ Submit", visible=True)
-            clear_btn = gr.ClearButton([textbox, imagebox, chatbot], value="🧹 Clear")
-            regenerate_btn = gr.Button(value="🔄 Regenerate", visible=True)
-            upload_btn = gr.UploadButton("📁 Upload image", file_types=["image"])
-    # with gr.Group():
-    #     with gr.Row():
-    #         with gr.Column(scale=1, min_width=50):
-    #             dope_bttn = gr.Button("Dope🔥")
-    #         with gr.Column(scale=1, min_width=50):
-    #             problematic_bttn = gr.Button("Problematic😬")
-
-    with gr.Row():
-        with gr.Accordion("Advanced settings", open=False, visible=True) as parameter_row:
-            system_prompt = gr.Textbox(
-                value=SYSTEM_PROMPT,
-                visible=False,
-                lines=20,
-                max_lines=50,
-                interactive=True,
-            )
-            max_new_tokens = gr.Slider(
-                minimum=8,
-                maximum=256,
-                value=128,
-                step=1,
-                interactive=True,
-                label="Maximum number of new tokens to generate",
-            )
-            repetition_penalty = gr.Slider(
-                minimum=0.0,
-                maximum=5.0,
-                value=1.2,
-                step=0.01,
-                interactive=True,
-                label="Repetition penalty",
-                info="1.0 is equivalent to no penalty",
-            )
-            decoding_strategy = gr.Radio(
-                [
-                    "Greedy",
-                    "Top P Sampling",
-                ],
-                value="Top P Sampling",
-                label="Decoding strategy",
-                interactive=True,
-                info="Higher values is equivalent to sampling more low-probability tokens.",
-            )
-            temperature = gr.Slider(
-                minimum=0.0,
-                maximum=5.0,
-                value=0.6,
-                step=0.1,
-                interactive=True,
-                visible=True,
-                label="Sampling temperature",
-                info="Higher values will produce more diverse outputs.",
-            )
-            decoding_strategy.change(
-                fn=lambda selection: gr.Slider.update(
-                    visible=(
-                        selection in ["contrastive_sampling", "beam_sampling", "Top P Sampling", "sampling_top_k"]
-                    )
-                ),
-                inputs=decoding_strategy,
-                outputs=temperature,
-            )
-            top_p = gr.Slider(
-                minimum=0.01,
-                maximum=0.99,
-                value=0.8,
-                step=0.01,
-                interactive=True,
-                visible=True,
-                label="Top P",
-                info="Higher values is equivalent to sampling more low-probability tokens.",
-            )
-            decoding_strategy.change(
-                fn=lambda selection: gr.Slider.update(visible=(selection in ["Top P Sampling"])),
-                inputs=decoding_strategy,
-                outputs=top_p,
-            )
-            # gr.Markdown(
-            #     """<p><strong>💡 Pro tip</strong>:<br>
-            #     You can input an arbitrary number of images at arbitrary positions in the same query.<br>
-            #     You will need to input each image with its URL with the syntax <code>&lt;fake_token_around_image&gt;&lt;image:IMAGE_URL&gt;&lt;fake_token_around_image&gt;</code>.<br>
-            #     For example, for two images, you could input <code>TEXT_1&lt;fake_token_around_image&gt;&lt;image:IMAGE_URL_1&gt;&lt;fake_token_around_image&gt;TEXT_2&lt;fake_token_around_image&gt;&lt;image:IMAGE_URL_2&gt;&lt;fake_token_around_image&gt;TEXT_3</code>.<br>
-            #     In the particular case where two images are consecutive, it is not necessary to add an additional separator: <code>&lt;fake_token_around_image&gt;&lt;image:IMAGE_URL_1&gt;&lt;fake_token_around_image&gt;&lt;image:IMAGE_URL_2&gt;&lt;fake_token_around_image&gt;</code>.</p>"""
-            # )
+            chatbot.render()
 
     def model_inference(
         model_selector,
@@ -506,9 +463,7 @@ with gr.Blocks(title="D", theme=gr.themes.Base()) as demo:
         if user_prompt_str.strip() == "" and image is None:
             return "", None, chat_history
 
-        import ast
         system_prompt = ast.literal_eval(system_prompt)
-        assert isinstance(system_prompt, list)
         formated_prompt_list, user_prompt_list = format_user_prompt_with_im_history_and_system_conditioning(
             system_prompt=system_prompt,
             current_user_prompt_str=user_prompt_str.strip(),
@@ -721,78 +676,6 @@ with gr.Blocks(title="D", theme=gr.themes.Base()) as demo:
     textbox.submit(lambda : gr.update(label='📁 Upload image', interactive=True), [], upload_btn)
     clear_btn.click(lambda : gr.update(label='📁 Upload image', interactive=True), [], upload_btn)
 
-    # Using Flagging for saving dope and problematic examples
-    # Dope examples flagging
-    # dope_callback.setup(
-    #     [
-    #         model_selector,
-    #         textbox,
-    #         chatbot,
-    #         imagebox,
-    #         decoding_strategy,
-    #         temperature,
-    #         max_new_tokens,
-    #         repetition_penalty,
-    #         top_p,
-    #     ],
-    #     "gradio_dope_data_points",
-    # )
-    # dope_bttn.click(
-    #     lambda *args: dope_callback.flag(args),
-    #     [
-    #         model_selector,
-    #         textbox,
-    #         chatbot,
-    #         imagebox,
-    #         decoding_strategy,
-    #         temperature,
-    #         max_new_tokens,
-    #         repetition_penalty,
-    #         top_p,
-    #     ],
-    #     None,
-    #     preprocess=False,
-    # )
-    # # Problematic examples flagging
-    # problematic_callback.setup(
-    #     [
-    #         model_selector,
-    #         textbox,
-    #         chatbot,
-    #         imagebox,
-    #         decoding_strategy,
-    #         temperature,
-    #         max_new_tokens,
-    #         repetition_penalty,
-    #         top_p,
-    #     ],
-    #     "gradio_problematic_data_points",
-    # )
-    # problematic_bttn.click(
-    #     lambda *args: problematic_callback.flag(args),
-    #     [
-    #         model_selector,
-    #         textbox,
-    #         chatbot,
-    #         imagebox,
-    #         decoding_strategy,
-    #         temperature,
-    #         max_new_tokens,
-    #         repetition_penalty,
-    #         top_p,
-    #     ],
-    #     None,
-    #     preprocess=False,
-    # )
-
-    # gr.Markdown("""## How to use?
-
-    #     There are two ways to provide image inputs:
-    #     - Using the image box on the left panel
-    #     - Using the inline syntax: `text<fake_token_around_image><image:URL_IMAGE><fake_token_around_image>text`
-
-    #     The second syntax allows inputting an arbitrary number of images.""")
-
     examples_path = os.path.dirname(__file__)
     gr.Examples(
         examples=[
@@ -805,35 +688,35 @@ with gr.Blocks(title="D", theme=gr.themes.Base()) as demo:
                 f"{examples_path}/example_images/zuck.jpeg",
             ],
             [
-                "Write a meme text for that image.",
+                "Craft a humorous caption for this image!",
                 f"{examples_path}/example_images/echasse.jpg",
             ],
             [
-                "Write a meme text for that image.",
+                "How about adding a dash of humor to this image with your words?",
                 f"{examples_path}/example_images/jesus.jpg",
             ],
             [
-                "Write a meme text for that image.",
+                "Give this image a comedic twist.",
                 f"{examples_path}/example_images/owl.jpg",
             ],
             [
-                "Write a meme text for that image.",
+                "Tell me a joke about that image.",
                 f"{examples_path}/example_images/pigeon.jpg",
             ],
             [
-                "Write a meme text for that image.",
+                "Let your sense of humor shine with that image!",
                 f"{examples_path}/example_images/plotorange.jpg",
             ],
             [
-                "Write a meme text for that image.",
+                "Make me laugh by commenting that image.",
                 f"{examples_path}/example_images/rats.jpg",
             ],
             [
-                "Write a meme text for that image.",
+                "Craft a meme text for that image.",
                 f"{examples_path}/example_images/sugardaddy.jpg",
             ],
             [
-                "Write a meme text for that image.",
+                "Ready to make this image even better? Write something funny to go with it!",
                 f"{examples_path}/example_images/wtf.jpg",
             ],
         ],