blacker521
commited on
Commit
•
2b095c5
1
Parent(s):
da44646
Update README.md
Browse files
README.md
CHANGED
@@ -5,4 +5,74 @@ language:
|
|
5 |
base_model:
|
6 |
- Qwen/Qwen2.5-1.5B-Instruct
|
7 |
pipeline_tag: text-generation
|
8 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
5 |
base_model:
|
6 |
- Qwen/Qwen2.5-1.5B-Instruct
|
7 |
pipeline_tag: text-generation
|
8 |
+
---
|
9 |
+
|
10 |
+
# NewsPicGen
|
11 |
+
|
12 |
+
NewsPicGen: News Picture Prompt Generation Model是一个中文新闻配图生成模型,使用[Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct)作为基座模型,使用SFT进行微调。
|
13 |
+
可以生成与新闻内容相关的高质量的中英双文配图prompt、中英双文关键字和绘画类型。直接通过Stable Diffusion生成配图,可根据绘画类型配置不同的绘图模板,生成多种风格的配图。
|
14 |
+
|
15 |
+
<p align="center">
|
16 |
+
🤗 <a href="https://huggingface.co/blacker521/NewsPicGen/">Hugging Face</a>   |   🤖 <a href="https://modelscope.cn/models/blacker521/NewsPicGen">ModelScope</a>
|
17 |
+
</p>
|
18 |
+
|
19 |
+
## 功能
|
20 |
+
- 生成与新闻内容相关的高质量的中英双文配图prompt、中英文关键字和绘画类型。
|
21 |
+
- 微调数据使用万级新闻数据,并采用多任务进行SFT微调,在生成绘画prompt的同时,对绘画类型(1.动物、2.人、3.人群、4.风景、5.建筑、6.科技产品、7.物品、8.其他)进行判断,强化模型输出效果。可以针对不同的绘画类型,配置不同的绘画模板。
|
22 |
+
- 支持JSON格式化输出,方便后续使用。
|
23 |
+
- 生成图片为漫画风格,对于新闻配图有较好的表现。
|
24 |
+
## 性能
|
25 |
+
|
26 |
+
- 使用QWen2.5-1.5B-Instruct作为基座模型,中等长度新闻生成绘画指令平均耗时500ms(A100-80G),配合[SGLang](https://github.com/modelscope/sglang)/[vllm](https://github.com/vllm-project/vllm)等框架可以更快的生成绘画指令。
|
27 |
+
|
28 |
+
## 快速开始
|
29 |
+
|
30 |
+
### 🤗 Hugging Face Transformers
|
31 |
+
|
32 |
+
使用Transformers生成绘画指令
|
33 |
+
|
34 |
+
```python
|
35 |
+
import json
|
36 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
37 |
+
|
38 |
+
model_name = "blacker521/NewsPicGen"
|
39 |
+
|
40 |
+
model = AutoModelForCausalLM.from_pretrained(
|
41 |
+
model_name,
|
42 |
+
torch_dtype="auto",
|
43 |
+
device_map="auto"
|
44 |
+
)
|
45 |
+
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
46 |
+
|
47 |
+
|
48 |
+
title = "孙颖莎谈大满贯最大的挑战"
|
49 |
+
content = "#孙颖莎希望找到赛场上拼搏的状态# 9月24日,是WTT中国大满贯2024倒计时2天,球员@孙颖莎 接受专访。孙颖莎在采访中谈及大满贯中最大的挑战,她表示大满贯已经是很顶尖的赛事水平了,所以每场球都会有挑战,希望自己能找到积极专注的在赛场上拼搏的状态。"
|
50 |
+
prompt = f'以下是一篇新闻,标题“{title}”。新闻内容:{content},请根据新闻内容生成绘画指令,图片要符合新闻内容,并且有创意。'
|
51 |
+
messages = [
|
52 |
+
{"role": "user", "content": prompt}
|
53 |
+
]
|
54 |
+
text = tokenizer.apply_chat_template(
|
55 |
+
messages,
|
56 |
+
tokenize=False,
|
57 |
+
add_generation_prompt=True
|
58 |
+
)
|
59 |
+
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
60 |
+
|
61 |
+
generated_ids = model.generate(
|
62 |
+
**model_inputs,
|
63 |
+
max_new_tokens=512
|
64 |
+
)
|
65 |
+
generated_ids = [
|
66 |
+
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
|
67 |
+
]
|
68 |
+
|
69 |
+
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
70 |
+
print(json.dumps(response, ensure_ascii=False))
|
71 |
+
# {
|
72 |
+
# "ch_keyword": "挑战大满贯,全力以赴",
|
73 |
+
# "ch_prompt": "画一个正在比赛中奋力拼搏的女子乒乓球运动员,她的面庞充满斗志和决心,手中握着乒乓球拍,眼睛紧盯着对手,背景为观众席上的欢呼声。",
|
74 |
+
# "en_keyword": "Challenging Grand Slam",
|
75 |
+
# "en_prompt": "Draw a female table tennis player in the middle of an intense match, her face filled with determination and resolve, holding a ping pong paddle in her hand, staring at her opponent closely, and the cheering from the audience in the background.",
|
76 |
+
# "type": "2"
|
77 |
+
# }
|
78 |
+
```
|