~/ nmadiraju.com Amazon EKS · SageMaker HyperPod on EKS
arrow_back Back to nmadiraju
Visual Walkthrough · AI/ML on EKS

How SageMaker HyperPod Works with Amazon EKS

Follow atlas-pretrain, two weeks on 16 GPU nodes, through a failed GPU, a restart and a shared cluster, then check yourself with 8 questions.

1Visual Walkthrough

A SageMaker HyperPod cluster of 16 ml.p5.48xlarge nodes orchestrated by an EKS cluster A HyperPod cluster, ml-hp, with instance group train-p5 of 16 ml.p5.48xlarge nodes, each with 8 GPUs, the device plugins and the health-monitoring agent; below it the SageMaker HyperPod service. Left: two teams, FSx for Lustre and an S3 bucket. Right: EKS cluster ml-eks with the PyTorchJob atlas-pretrain, Kueue task governance and an InferenceEndpointConfig. The walkthrough shows a GPU failure on plain nodes, deep health checks, automatic node replacement, job auto-resume and checkpoints, team quotas with preemption, and serving the model. team research atlas-pretrain · 70B 16 nodes · 2 weeks team applied 1-node experiments FSx for Lustre S3 atlas-data dataset · copies device plugins NVIDIA GPU + EFA · DaemonSets health-monitoring agent DaemonSet · aws-hyperpod SageMaker HyperPod · cluster ml-hp NodeRecovery: Automatic · deep health checks on EKS CLUSTER ml-eks · ORCHESTRATOR API server and scheduler PyTorchJob atlas-pretrain Kueue · task governance research: quota 16 × ml.p5.48xlarge applied: borrows idle nodes InferenceEndpointConfig atlas-chat model: s3://atlas-data/final/

    2Quiz to check understanding

    Check yourself