Lab 15: Run SGLang on HAMi GPU Shares
Install HAMi on a GPU cluster and schedule SGLang inference services with GPU partitioning.
3 docs tagged with "inference"
View all tagsLab 16: Serve Models from a KitOps ModelKit on HAMi
Package a model as a KitOps ModelKit, pull it from Jozu Hub with an initContainer, and serve it locally with SGLang (and optionally vLLM) on HAMi GPU shares.
Lab 6: Run vLLM on HAMi GPU Shares
Install HAMi on a GPU cluster and schedule vLLM inference services with GPU partitioning.
