Skip to content

Commit

Permalink
Merge pull request #6 from Lednik7/dev
Browse files Browse the repository at this point in the history
add load example, notes
  • Loading branch information
Lednik7 authored May 2, 2022
2 parents 1782501 + 0046faf commit e3122af
Showing 1 changed file with 26 additions and 1 deletion.
27 changes: 26 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,32 @@ probs = logits_per_image.softmax(dim=-1).detach().cpu().numpy()

print("Label probs:", probs) # prints: [[0.41456965 0.29270944 0.29272085]]
```
Enjoy the speed
#### Notes:
If you use:
```python3
clip.model.ResidualAttentionBlock.attention = attention
```
The results may not match. To avoid this, you must manually change the **logit_scale** parameter.\
For the example above:
```python3
onnx_model.logit_scale = 1600 # -> Label probs: [[0.99242663 0.00378558 0.00378777]]
```
**Enjoy the speed**

## Load saved model
Example for ViT-B/32 from Model Zoo
```python3
!wget https://clip-as-service.s3.us-east-2.amazonaws.com/models/onnx/ViT-B-32/visual.onnx
!wget https://clip-as-service.s3.us-east-2.amazonaws.com/models/onnx/ViT-B-32/textual.onnx
```
```python3
onnx_model = clip_onnx(None)
onnx_model.load_onnx(visual_path="visual.onnx",
textual_path="textual.onnx",
logit_scale=100.0000) # model.logit_scale.exp()
onnx_model.start_sessions(providers=["CPUExecutionProvider"])
```

## Model Zoo
Models of the original CLIP can be found on this [page](https://github.com/jina-ai/clip-as-service/blob/main/server/clip_server/model/clip_onnx.py).\
They are not part of this library but should work correctly.
Expand Down

0 comments on commit e3122af

Please sign in to comment.