feat: Generate embeddings for images

[cecheta]

Required by #748

Purpose

• This PR fetches embeddings from Computer Vision when an image is uploaded and advanced image processing is enabled
• Adds functional and unit tests
  • Note that the unit tests rely on pytest-httpserver
  • To enable this, the fixtures for enabling HTTPS for the mock server have been moved to the top-level conftest.py, so they are in the scope of all tests (unit + functional)
• Created client to call computer vision to generate embeddings, as this is not supported yet by the official SDK
• Add config validation before saving config file, so that advanced image processing cannot be enabled for non-image types (e.g. .txt)
• Added computer vision key/endpoint to env vars

Does this introduce a breaking change?

• Yes
• No

Testing:

• Deployed to Azure in keys + rbac
• Uploaded images, verified embeddings in logs

[github-actions]

Coverage Report •

File	Stmts	Miss	Cover	Missing
code/backend/batch/utilities/helpers
azure_computer_vision_client.py	47	0	100%
env_helper.py	133	10	92%	221, 226–227, 230–232, 244, 248–250
code/backend/batch/utilities/helpers/config
config_helper.py	134	0	100%
code/backend/batch/utilities/helpers/embedders
embedder_factory.py	10	3	70%	12–13, 15
push_embedder.py	51	0	100%
TOTAL	2383	673	71%

Tests	Skipped	Failures	Errors	Time
194	0 💤	0 ❌	0 🔥	11.979s ⏱️

[gaurarpit]

Looks great.. some comments

[gaurarpit]

Looks great. Nothing major, some minor ones. Ship-It.

                              ______________________________________________
                          .-'                     _                        '.
                        .'                       |-'                        |
                      .'                         |                          |
                   _.'               p         _\_/_         p              |
                _.'                  |       .'  |  '.       |              |
           __..'                     |      /    |    \      |              |
     ___..'                         .T\    ======+======    /T.             |
  ;;;\::::                        .' | \  /      |      \  / | '.           |
  ;;;|::::                      .'   |  \/       |       \/  |   '.         |
  ;;;/::::                    .'     |   \       |        \  |     '.       |
        ''.__               .'       |    \      |         \ |       '.     |
             ''._          <_________|_____>_____|__________>|_________>    |
                 '._     (___________|___________|___________|___________)  |
                    '.    \;;;o;;;;o;;;;;o;;;;;o;;;;;o;;;;;o;;;;;o;;;;/   |
                      '.~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~   |
                        '. ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~  |
                          '-.______________________________________________.'

[gaurarpit]

tests are failing.. looking into them

[gaurarpit]

tests are failing.. looking into them

fixed.

[superhindupur]

Regarding the use of a mock http server for unit tests:

I understand the point about not wanting to tie our mocks tightly to the underlying implementation (http requests vs sdk), but couldn't we resolve this issue by just using an interface and implementation? Mock the interface behavior in the tests, and the implementation details will remain hidden.
Also, mocks in general tend to be tied to underlying implementation. Why the special consideration for this case?

I would vote for keeping unit tests free of http servers - and reserve this type of testing for our functional tests. Consistency helps readers reason about the code base. I'd argue that in this case readability is more important than the possible rework of the test mocks if/when we switch the underlying calls from http to sdk.

[adamdougal]

Bhavana and I caught up on a call. Decided to keep the tests as they are, but added a comment to the test class explaining the rationale.