# Qwen2.5-0.5B on AMD

A 0.5B dense chat model on one AMD Instinct MI325X on a bring-your-own cluster.

Source: /recipes/qwen2.5-0.5b/

<!-- vale write-good.Passive = NO -->
A 0.5B dense chat model on one AMD Instinct MI325X on a bring-your-own
cluster: one `Standalone` engine, no cache, weights pulled straight from
Hugging Face. The model is deliberately small - the point of this run is the
AMD serving path, not the model. The class claims the GPU through DRA
(`gpu.amd.com`); the serving stack installs the AMD GPU operator in DRA mode
when a class claims that driver, against the amdgpu kernel driver the node
image already carries. The engine is AMD's ROCm vLLM.

The manifests are the ones validated end to end on an 8x MI325X bare metal
node (each GPU has just under 256 GiB of HBM3e), registered here as a
bring-your-own cluster - `source: Existing` is the one cloud source that
installs the AMD stack today. Bring the cluster with the amdgpu driver on its
GPU nodes, label the pool's nodes `modelplane.ai/pool=gpu-mi325x`, and apply
the platform side first, then the ML side.

## Validated deployments

{{< validated-deployments >}}

## Platform

{{< manifests "recipes/qwen2.5-0.5b/inference-class.yaml" >}}

{{< manifests "recipes/qwen2.5-0.5b/inference-cluster.yaml" >}}

## Deployment

{{< manifests "recipes/qwen2.5-0.5b/model-deployment.yaml" >}}

{{< manifests "recipes/qwen2.5-0.5b/model-service.yaml" >}}
<!-- vale write-good.Passive = YES -->