Update VLM example code in README #1466
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add pipe.start_chat() to VLM example. Without this, inference with several models results in empty outputs.
| @@ -133,6 +133,7 @@ from PIL import Image | |||
|
|
|||
| # Choose GPU instead of CPU in the line below to run the model on Intel integrated or discrete GPU | |||
| pipe = ov_genai.VLMPipeline("./InternVL2-1B", "CPU") | |||
| pipe.start_chat() | |||
|
|
|||
There was a problem hiding this comment.
BTW, below we have print(pipe.generate(prompt, image=image_data, max_new_tokens=100)) which returns VLMDecodedResults
should we fix it to extract texts[0] from this structure?
There was a problem hiding this comment.
Thanks, I added that to this PR. Apparently VLMDecodedResults has a .__str__ method that shows the text so it is not immediately noticeable that this generate method does not actually return the text; I agree it's better to be explicit about this in the example.
Explicitly print result.texts[0] in VLM example
ilya-lavrenov
approved these changes
Jan 6, 2025
|
build_jenkins |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add
pipe.start_chat()to VLM example. Without this, inference with several models results in empty outputs.This can be removed if this will be the default for VLM models, but at the moment, the most basic example should work with supported models.
Also changed printing the VLMDecodedResults to getting the generated text and printing that (see comment from Ilya).