Lab 11: KServe Inference with HAMi DRA GPU Sharing
Deploy a KServe Standard vLLM service and run two Predictor replicas on one NVIDIA GPU through native HAMi DRA claims.
3 docs tagged with "vllm"
View all tagsLab 16: Serve Models from a KitOps ModelKit on HAMi
Package a model as a KitOps ModelKit, pull it from Jozu Hub with an initContainer, and serve it locally with SGLang (and optionally vLLM) on HAMi GPU shares.
Lab 6: Run vLLM on HAMi GPU Shares
Install HAMi on a GPU cluster and schedule vLLM inference services with GPU partitioning.
