Deploy DeepSeek-R1 Models on Kubernetes with Amazon EKS

Поделиться
HTML-код
  • Опубликовано: 30 янв 2025

Комментарии •

  • @CarlosSantanaDev
    @CarlosSantanaDev 11 минут назад

    Great demo folks!

  • @Alex-p1e7t
    @Alex-p1e7t 37 минут назад

    I tried this so many times following your blog but the nodes does not come up at all.
    Nodes are not there so the pods are always pending. I waited for more than hour.
    The deployment and everything is there but no nodes. All though the node pools are created.

  • @vgrin545
    @vgrin545 2 часа назад

    what about container rightsizing recommendations?

    • @ContainersfromtheCouch
      @ContainersfromtheCouch  Час назад

      We've included some starter recommendations, check the manifests folder in the GitHub repo linked in the description. Auto Mode (Karpenter) handles the sizing automatically based on these manifests.

    • @vgrin545
      @vgrin545 Час назад

      @@ContainersfromtheCouch Karpenter provides node sizing.

    • @devilopstalks
      @devilopstalks Час назад

      @@vgrin545you are right, in order for the node to be right sized the resources should be right profiled. There are tools like Kubecost or StormeForge that provides AI capabilities to do that right sizing for you. Other than that, is up to you to do load testing and benchmarking and tweak the parameters to see how much CPU, memory and GPU cores you want to define, there is no magic solution other than companies that provide it as a SaaS like the ones I mentioned above ☝️

    • @ContainersfromtheCouch
      @ContainersfromtheCouch  Час назад

      @vgrin545 You're right. Container sizing like CPU and memory limits are also in that manifest. We don't have automatic vertical right sizing in the sample.

    • @vgrin545
      @vgrin545 Час назад

      @@ContainersfromtheCouch You do not need VPS. This Model is supposed to provide container rightsizing based on provided CPU/Memory Requests/Limits. We need to avoid overprovisioning.