Skip to content

@ModelCloud ModelCloud.ai

Change the repository type filter

All

    Repositories list

    2 repositories

    • GPTQModel

      Public
      Production ready LLM model compression/quantization toolkit with accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.
      Python
      •
      Apache License 2.0
      •31•192•7•1•Updated Jan 10, 2025Jan 10, 2025

    • Device-SMI

      Public
      Self-contained Python lib with zero-dependencies that give you a unified device properties for gpu, cpu, and npu. No more calling separate tools such as nvidia-smi or /proc/cpuinfo and parsing it yourself.
      devicecpugpusminpuxpu
      Python
      •
      Apache License 2.0
      •1•9•1•2•Updated Jan 10, 2025Jan 10, 2025