SageMaker JumpStart: deploy Hugging Face models in minutes!
HTML-код
- Опубликовано: 20 окт 2024
- Experimenting with the latest and greatest models doesn't have to be difficult. With SageMaker JumpStart, you can easily access and experiment with cutting-edge large language models without the hassle of setting up complex infrastructure or writing deployment code. All it takes is a single click. In this particular video, I walk you through the process of deploying and testing the Mistral AI 7B model as an example.
⭐️⭐️⭐️ Don't forget to subscribe to be notified of future videos ⭐️⭐️⭐️
To get started, you simply need to navigate to the SageMaker JumpStart website and locate the Mistral AI 7B model. Once you find it, you can click on the model to select it. This will initiate the setup process, which takes care of all the required infrastructure for you. Once the setup is complete, SageMaker JumpStart provides a sample notebook and you can start testing the model immediately!
If you want to experiment with the latest state-of-the-art models like the Mistral AI 7B model, SageMaker JumpStart provides a hassle-free way to do so. Try it out and explore the possibilities of cutting-edge AI models with just one click!
Amazon SageMaker JumpStart: aws.amazon.com...
Follow me on Medium at / julsimon or Substack at julsimon.subst....
Showing the 280 reference models is very useful - it helps focus more than the 350k models on HF. I think these reference models and maybe top three types of easy access models mean alot to most due to trade offs between speed, MoE accuracy of desired role, recency, and performance. I've been favoring GPT, Mistral, and Llama lately and its great to see a quick start for these. Thanks for demonstrating the SageMaker connection!
Yes, definitely a good place to start, and once you've found an architecture that works for you, you can check out the countless fine-tuned variants on the HF hub. Getting close to 500k models :)
THank you for this tutorial!!! Never knew it was so simple
Glad it was helpful!
Thank you for the tutorials, Julien!
My pleasure!
Hello Julian, While I'm setting up for deploying any model, it says the instance limit is 0 , could you please help me with that
Contact AWS support through the console and increase your service limit.
Well explained Julien. Thanks.
Glad you liked it!
The interface and everything has changed since this video. Can you provide an updated video that walks through the process of loading a module from huggingface into stagemaker jumpstart?
Hi, you don't load models from Hugging Face. The models are already in AWS. The UI has evolved but the workflow is still the same : open Jumpstart, select a model, click on deploy, open the sample notebook.
That's great! The main challenge i am facing in germany is to find models that support / "understand" german alongside english and can be deployed to EU AWS - Regions due to privacy and EU regulation and safety concerns with company data.
I can find some of these models through the hugging face platform. but those are often not easily deployable to sagemager or if, then there's no capable enough AWS EU Region Server that allows this model to run properly.
Would be really grateful for a tutorial or resources on how to get those "language modified" models on a private inference endpoint in EU Region.
Hi, SageMaker works exactly the same in all AWS regions, you shouldn't see any difference or restriction. Or are you talking about GPU availability?
Please post here if you need help : discuss.huggingface.co/c/sagemaker/17
Is possible to finetune the models which are available in Jumpstart? If yes please share the insights.
Some models allow it, some don't, see docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-foundation-models-use-studio-updated.html#jumpstart-foundation-models-use-studio-updated-fine-tune
Hi Julie, I got the error which said "ClientError: An error occurred (ValidationException) when calling the CreateModel operation: Caller is not subscribed to the marketplace offering." Do you know how to fix it? Thanks!
Looks like you need to deploy a model listed on the AWS marketplace. You need to suscribe to the model first on the AWS marketplace, and then deploy it with the notebook.
Thank u Julien appreciate this so much right now
thanks!
Thank you for this. Please I will like to know how can I query this endpoint from a web service? or if there is any guide you can point me to.
Hi, the endpoint is a web service. You can invoke it either with the SageMaker SDK predict() API, or with any HTTP-based library. Each model in Jumpstart has a sample notebook, start from there.
I dont see the mistral model on SageMaker, what's wrong?
It's not on Jumpstart, but you can deploy it easily with the SageMaker SDK,, just like any hub model. Go to huggingface.co/mistralai/Mistral-7B-v0.1, click on "Deploy", select "Amazon SageMaker" and run the generated code.
@@juliensimonfr Ah thanks for the tip, going that way for testing. Enjoy your coffee!
OK but how do I use any model on hugging face I want? Who wants to deploy a model that doesn't have any value prop over GPT4 or Claude (e.g. uncensored)?
Not sure what the second statement means, but you can pretty much deploy any Hugging Face model on Sagemaker. Go to the model page, click on "Deploy", select "SageMaker", copy paste the deployment code snippet and run it in your AWS account.
1:58 you say “on the hub, we have …” what do you mean by “the hub”? I am new to Hugging Face so not familiar with that term.
The Hugging Face hub at huggingface.co
That's great!
Agreed. Thanks for watching :)