Update README.md

README.md

@@ -13,8 +13,8 @@ tags:
 
 ONNX/RKNN2部署Florence-2视觉多模态大模型! 
 
-- 推理速度(RKNN2)：RK3588推理一张512x512图片, 使用`<MORE_DETAILED_CAPTION>`指令, 总时间需要~2秒
-- 内存占用(RKNN2)：约1.9GB
+- 推理速度(RKNN2)：RK3588推理一张768x768图片, 使用`<MORE_DETAILED_CAPTION>`指令, 总时间需要~4.5秒
+- 内存占用(RKNN2)：约2GB
 
 ## 使用方法
 
@@ -54,9 +54,9 @@ python convert.py all
 注意: RKNN模型不支持运行时任意长度的输入, 所以需要提前确定好输入的shape. 之后可以修改convert.py中的`vision_size`, `vision_tokens`, `prompt_tokens`来修改输入的shape. 
 
 ## 存在的问题(rknn)
-- 已知转换vision encoder时输入分辨率>=640x640时会报错`Buffer overflow!`然后转换失败, 所以目前是使用512x512的输入, 推理质量会下降
-- 在同分辨率下推理精度相比onnxruntime有显著下降, 已确定问题出在vision encoder部分. 如果对精度要求高, 这部分可以换成onnxruntime
-- vision encoder推理需要1.5秒, 占比最大, 但其中大半时间都在做Transpose, 也许还可以优化
+- ~~已知转换vision encoder时输入分辨率>=640x640时会报错`Buffer overflow!`然后转换失败, 所以目前是使用512x512的输入, 推理质量会下降~~ (已解决)
+- ~~在同分辨率下推理精度相比onnxruntime有显著下降, 已确定问题出在vision encoder部分. 如果对精度要求高, 这部分可以换成onnxruntime~~ (已解决, 不过现在貌似仍然会丢失一点精度)
+- ~~vision encoder推理需要1.5秒, 占比最大, 但其中大半时间都在做Transpose, 也许还可以优化~~ (已优化)
 - decode阶段因为kvcache的长度不断变化, 貌似无法简单的使用NPU推理. 不过用onnxruntime应该也足够了
 
 ## 参考
@@ -71,8 +71,8 @@ python convert.py all
 
 ONNX/RKNN2 deployment for Florence-2 visual-language multimodal large model!
 
-- Inference speed (RKNN2): RK3588 inference with a 512x512 image, using the `<MORE_DETAILED_CAPTION>` instruction, takes ~2 seconds in total.
-- Memory usage (RKNN2): Approximately 1.9GB
+- Inference speed (RKNN2): RK3588 inference with a 768x768 image, using the `<MORE_DETAILED_CAPTION>` instruction, takes ~4.5 seconds in total.
+- Memory usage (RKNN2): Approximately 2GB
 
 ## Usage
 
@@ -113,9 +113,9 @@ Note: The RKNN model does not support arbitrary input lengths at runtime, so you
 
 ## Existing Issues (rknn)
 
-- It is known that converting the vision encoder with an input resolution >= 640x640 will result in a `Buffer overflow!` error and conversion failure. Therefore, the current input is 512x512, which will reduce inference quality.
-- Inference accuracy is significantly lower compared to onnxruntime at the same resolution. The problem has been identified in the vision encoder part. If high accuracy is required, this part can be replaced with onnxruntime.
-- Vision encoder inference takes 1.5 seconds, accounting for the largest proportion, but most of this time is spent on Transpose op, which may be further optimized.
+- ~~It is known that converting the vision encoder with an input resolution >= 640x640 will result in a `Buffer overflow!` error and conversion failure. Therefore, the current input is 512x512, which will reduce inference quality.~~ (Solved.)
+- ~~Inference accuracy is significantly lower compared to onnxruntime at the same resolution. The problem has been identified in the vision encoder part. If high accuracy is required, this part can be replaced with onnxruntime.~~ (Solved, although there is still some precision loss present.)
+- ~~Vision encoder inference takes 1.5 seconds, accounting for the largest proportion, but most of this time is spent on Transpose op, which may be further optimized.~~ (Solved.)
 - The decode phase seems unable to simply use NPU inference because the length of kvcache keeps changing. However, using onnxruntime should be sufficient.
 
 ## References