What is HAMi?

HAMi (Heterogeneous AI Computing Virtualization Middleware) is an open source platform for managing heterogeneous AI computing devices in Kubernetes clusters. Formerly known as k8s-vGPU-scheduler, HAMi enables device sharing across multiple containers and workloads.

HAMi is a Cloud Native Computing Foundation Sandbox project and is listed in both the CNCF Landscape and CNAI Landscape.

Key Features

Device Sharing

• Multi-device Support: Compatible with various heterogeneous AI computing devices (GPUs, NPUs, etc.)
• Shared Access: Multiple containers can simultaneously share devices for improved resource utilization

Memory Management

• Hard Limits: Enforce strict memory limits inside containers to prevent resource conflicts
• Dynamic Allocation: Allocate device memory on-demand based on workload requirements
• Flexible Units: Specify memory allocation in MB or as a percentage of total device memory

Device Specification

• Type Selection: Request specific types of heterogeneous AI computing devices
• UUID Targeting: Specify exact devices using device UUID for precise allocation

Ease of Use

• Transparent to Workloads: No code changes required inside containers
• Simple Deployment: Install and uninstall using Helm with minimal configuration

Open Governance

• Community Driven: Jointly initiated by organizations across internet, finance, manufacturing, and cloud services
• Neutral Development: Managed as an open source project with CNCF

What's Next

Here are some recommended next steps:

• Learn about HAMi's architecture
• Install HAMi in your Kubernetes cluster