feat(ml): ML on Rockchip NPUs

[yoni13]

Goals: ML on Rockchip NPUs.
Testing on board: #13243 (reply in thread)

TODO:

• It's works on my OrangePi 3B (RK3566).
• Build Docker images.
• Build models for more SOCs.
• Able to set thread numbers for each model type (e.g. visual -> 2 threads) via environment variables.
• NPU core masks for RK3576/3588
• Decide model path (immich-app/ViT-B-32__openai/textual/rk3566.rknn) .
• Export script that accepts CLI arguments for the Immich model name and SoC, then exports the model.
• Able to maximize NPU usage by using rknnpool.py
• Documentation.
• Write tests.
• Model on Huggingface.
• Make Docker images work out of the box (without downloading models manually).

Nice to have:

• Test on PC and other arm-based boards to ensure it doesn't break something.
• Rebase my commits (sorry for ugly commit messages).
• Test if it's working on RK3588 (I don't have one).
• Support more models.

#13243

[yoni13]

Docker launch command:

docker run --security-opt systempaths=unconfined --security-opt apparmor=unconfined --device /dev/dri --device /dev/dma_heap --device /dev/rga --device /dev/mpp_service -v /cache:/cache:ro  -p 3004:3003 -v /sys/kernel/debug/:/sys/kernel/debug/:ro --name rknnimmich_name -d rknnimmich

and it works (if you download model to cache ofc)

ViT-B-32 and buffalo_l are loaded with two threads rerunning jobs: 2.7G, and peak 3.5G RAM. (its like running 4 models at the same time)
Update: this statistic is before we only load onnx when required, will update mem usage when I got time

[yoni13]

Most issues mentioned in the review should be fixed now.

[yoni13]

Wrote a matrix build script to see what model is not supported, all of them are nlib and XLM.
https://github.com/yoni13/immich_to_rknn2/actions/runs/12892750773

[mertalev]

Wrote a matrix build script to see what model is not supported, all of them are nlib and XLM. yoni13/immich_to_rknn2/actions/runs/12892750773

Oh wow, that is pretty cool!

I have an RK3588, so I can do some testing with this PR soon.

[yoni13]

3000 photos, Everything default, rerun smart search, face detection&recognition
RK3566 NPU:

• 50mins
• 65 c

RK3566 CPU:

• 4hr 37min
• 76c

GPU?
/dev/mali0 on RK3566 doesn't even work on Armbian/DietPi