Update pipeline example

README.md

@@ -1,8 +1,7 @@
 ---
 language:
 - en
-pipeline_tag: image-to-text
-inference: false
+pipeline_tag: image-text-to-text
 arxiv: 2304.08485
 tags:
 - vision
@@ -51,30 +50,21 @@ Where `<prompt>` denotes the prompt asked by the user
 
 ```python
 from transformers import pipeline
-from PIL import Image    
-import requests
-
-model_id = "llava-hf/vip-llava-13b-hf"
-pipe = pipeline("image-to-text", model=model_id)
-url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/ai2d-demo.jpg"
-image = Image.open(requests.get(url, stream=True).raw)
 
-# Define a chat histiry and use `apply_chat_template` to get correctly formatted prompt
-# Each value in "content" has to be a list of dicts with types ("text", "image") 
-conversation = [
+pipe = pipeline("image-text-to-text", model="llava-hf/vip-llava-13b-hf")
+messages = [
     {
-
       "role": "user",
       "content": [
+          {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/ai2d-demo.jpg"},
           {"type": "text", "text": "What does the label 15 represent? (1) lava (2) core (3) tunnel (4) ash cloud"},
-          {"type": "image"},
         ],
     },
 ]
-prompt = processor.apply_chat_template(conversation, add_generation_prompt=True)
 
-outputs = pipe(image, prompt=prompt, generate_kwargs={"max_new_tokens": 200})
-print(outputs)
+out = pipe(text=messages, max_new_tokens=20)
+print(out)
+>>> [{'input_text': [{'role': 'user', 'content': [{'type': 'image', 'url': 'https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/ai2d-demo.jpg'}, {'type': 'text', 'text': 'What does the label 15 represent? (1) lava (2) core (3) tunnel (4) ash cloud'}]}], 'generated_text': 'Lava'}]
 ```
 
 ### Using pure `transformers`: