Using GPUs with FKS

Fly Kubernetes is in beta and not recommended for critical production usage. To report issues or provide feedback, email us at

GPUs are available on Fly Kubernetes. Information about our GPUs can be found in our Fly GPUs documentation.

GPUs are consumed by requesting a GPU resource, similarly to requesting cpu or memory. There is a custom resource<gpu type> that is used to request GPUs. Note that:

  • The available GPU types are: a10, l40s, a100-pcie-40gb and a100-sxm4-80gb. Check the documentation for which regions they are available in.
  • Pods are deployed in the same region as your cluster. You can place your Pod in a particular region by adding a <region> annotation to your Pod’s metadata.
  • GPU resource requests should only specify the limits section.
  • You can specify CPU and memory resources alongside GPU resources. The minimum number of cores supported is 2 and the minimum amount of memory is 4096 MiB. The VMs deployed will always be performance Machines.
  • The valid number of GPUs you can request are: 1, 2, 4 and 8.
  • You can only request one type of GPU at a time

Below is an example of a Pod with a GPU:

apiVersion: v1
kind: Pod
  name: ollama
  annotations: ams # optional
    - name: ollama
      image: ollama/ollama:latest

This will deploy a GPU Machine with the size a100-80gb with 1 GPU core in the region ams.