WallarooAI
WallarooAI
  • Видео 108
  • Просмотров 7 370
Building Custom LLMs for Production Inference Endpoints
In this session we dive into the details for how to build, deploy, and optimize custom Large Language Models (LLMs) for production inference environments.
This session will cover the key steps for Custom LLMs (LLama), focusing on:
- Why custom LLM’s?
- Inference Performance Optimization
- Bias and Harmful language Detection
Learn More:
- Wallaroo LLM Operations Docs: docs.wallaroo.ai/wallaroo-llm/
- Wallaroo Community Edition: portal.wallaroo.community/
Просмотров: 35

Видео

Humans Talking About AI | Wallaroo.AI Podcast - Ep. 5 Efficient AI For the Future15:05Humans Talking About AI | Wallaroo.AI Podcast - Ep. 5 Efficient AI For the Future
Humans Talking About AI | Wallaroo.AI Podcast - Ep. 5 Efficient AI For the Future
Просмотров 8День назад
In this episode of Humans Talking About AI, Steven Spellicy (COO, Wallaroo.AI) chats with Chris McDermott, VP of Research & Development Engineering at Wallaroo.AI, about the growing infrastructure demands of artificial intelligence. From the scarcity of GPUs to energy and water consumption, we explore the challenges companies face in balancing AI innovation with sustainability. Hear about the s...
Forecasting Demand at Multiple Granular Levels with Wallaroo6:57Forecasting Demand at Multiple Granular Levels with Wallaroo
Forecasting Demand at Multiple Granular Levels with Wallaroo
Просмотров 221 день назад
In this video you will learn how to deploy, serve and orchestrate a demand forecasting model. This video will show how to easily integrate your model inference with your existing tech stack and automate the model inference process with Wallaroo orchestration. Learn More Free Wallaroo Community Edition - hubs.la/Q02nf2vF0 Forecasting Tutorial: hubs.la/Q02T_7d10
Deploying LLMs With Wallaroo and Ampere on Oracle Cloud Infrastructure3:22Deploying LLMs With Wallaroo and Ampere on Oracle Cloud Infrastructure
Deploying LLMs With Wallaroo and Ampere on Oracle Cloud Infrastructure
Просмотров 30Месяц назад
In this video we show how we can use Wallaroo installed on Oracle Cloud Infrastructure (OCI) with Ampere hardware to deploy a quantized version of Llama 3.0. The video shows automated model packaging to optimize for the Ampere hardware as well as managing and monitoring LLMs for performance and potential toxicity with Wallaroo LLM Listeners. Try for yourself using the plans below. Wallaroo Infe...
Edge AI Inference Lifecycle Pt 4: Edge In-Line Model Updates with Wallaroo1:17Edge AI Inference Lifecycle Pt 4: Edge In-Line Model Updates with Wallaroo
Edge AI Inference Lifecycle Pt 4: Edge In-Line Model Updates with Wallaroo
Просмотров 10Месяц назад
Edge AI Inference Lifecycle Part 4: This video is part of a series of shorts. Part 4 shows updating a computer vision (CV) model to an edge endpoint in-line with no production downtime. Try these for yourself with the following resources. Edge AI Inference Tutorials: hubs.la/Q02z04X80 Free Community Edition: hubs.ly/Q02vLZC40 Azure Marketplace Community Edition: hubs.la/Q02zlrqg0
Edge AI Inference Lifecycle Pt 3: Optimizing Models Using Shadow Deploy with Wallaroo1:19Edge AI Inference Lifecycle Pt 3: Optimizing Models Using Shadow Deploy with Wallaroo
Edge AI Inference Lifecycle Pt 3: Optimizing Models Using Shadow Deploy with Wallaroo
Просмотров 11Месяц назад
Edge AI Inference Lifecycle Part 3: This video is part of a series of shorts. Part 3 shows optimizing models on an edge endpoint using the Shadow Deploy method. Try these for yourself with the following resources. Edge AI Inference Tutorials: hubs.la/Q02z04X80 Free Community Edition: hubs.ly/Q02vLZC40 Azure Marketplace Community Edition: hubs.la/Q02zlrqg0
Edge AI Inference Lifecycle Pt 2: Centralized Model Observability with Wallaroo2:39Edge AI Inference Lifecycle Pt 2: Centralized Model Observability with Wallaroo
Edge AI Inference Lifecycle Pt 2: Centralized Model Observability with Wallaroo
Просмотров 9Месяц назад
Edge AI Inference Lifecycle Part 2: This video is part of a series of shorts. Part 2 shows centralized model observability computer vision on a remote edge endpoint. Try these for yourself with the following resources. Edge AI Inference Tutorials: hubs.la/Q02z04X80 Free Community Edition: hubs.ly/Q02vLZC40 Azure Marketplace Community Edition: hubs.la/Q02zlrqg0
Edge AI Inference Lifecycle Pt 1: Deploying Models to the Edge with Wallaroo2:49Edge AI Inference Lifecycle Pt 1: Deploying Models to the Edge with Wallaroo
Edge AI Inference Lifecycle Pt 1: Deploying Models to the Edge with Wallaroo
Просмотров 16Месяц назад
Edge AI Inference Lifecycle Part 1: This video is part of a series of shorts. Part 1 shows deploying a computer vision (CV) model to an edge endpoint. Try these for yourself with the following resources. Edge AI Inference Tutorials: hubs.la/Q02z04X80 Free Community Edition: hubs.ly/Q02vLZC40 Azure Marketplace Community Edition: hubs.la/Q02zlrqg0
Wallaroo Edge AI inference lifecycle (deployment, monitoring and updates)11:03Wallaroo Edge AI inference lifecycle (deployment, monitoring and updates)
Wallaroo Edge AI inference lifecycle (deployment, monitoring and updates)
Просмотров 43Месяц назад
In this video you will learn how to - Deploy a computer vision model to the cloud and to the edge. - Centrally manage and administer our deployments. - Retrieve and view model logs - Automatically detect drift for model deployments. - Perform in-line model updates. Try for yourself: Free Community Edition: hubs.la/Q02yhJnc0 Deploy and Infer Edge Models Tutorial: hubs.la/Q02PV5SN0
Optimized Inference on Intel CPU & GPU with OpenVino and Wallaroo3:47Optimized Inference on Intel CPU & GPU with OpenVino and Wallaroo
Optimized Inference on Intel CPU & GPU with OpenVino and Wallaroo
Просмотров 1363 месяца назад
This video shows how maximized performance and efficiency with Wallaroo’s Platform optimized for Intel Architecture, leveraging the inference serving capabilities of OpenVINO Toolkit. This optimization ensures that Wallaroo’s Platform fully utilizes the underlying intel hardware, resulting in faster processing times, reduced latency, and increased throughput for AI workloads in the cloud or at ...
Deploying LLM Inference Endpoints & Optimizing Output with RAG in Wallaroo2:49Deploying LLM Inference Endpoints & Optimizing Output with RAG in Wallaroo
Deploying LLM Inference Endpoints & Optimizing Output with RAG in Wallaroo
Просмотров 673 месяца назад
In this short video we'll look at how we can address LLM hallucinations by deploying a RAG pipeline with an open source LLM in Wallaroo.
Unveiling Tomorrow's Retail: Exploring the Role of AI in Shaping the Industry59:37Unveiling Tomorrow's Retail: Exploring the Role of AI in Shaping the Industry
Unveiling Tomorrow's Retail: Exploring the Role of AI in Shaping the Industry
Просмотров 2223 месяца назад
In this webinar, we delve into the pivotal role of (AI) in shaping the future of retail. From intelligent automation to predictive analytics, AI technologies are revolutionizing every aspect of the retail value chain. Through discussions and real-world examples, attendees will gain valuable insights into the transformative potential of AI in driving revenue growth, enhancing customer engagement...
Monitoring LLM Inference Endpoints with Wallaroo LLM Listeners3:12Monitoring LLM Inference Endpoints with Wallaroo LLM Listeners
Monitoring LLM Inference Endpoints with Wallaroo LLM Listeners
Просмотров 1763 месяца назад
In this short video we go over Monitoring LLMs with Wallaroo LLM Listeners by monitoring the input and generated text by our LLMs in an automated way. We use Open AI GPT 3.5 to summarize our texts and set a threshold score for how well the model performs using Wallaroo LLM Listeners before publishing the generated text. We also show LLM monitoring for drift using Assays in Wallaroo.
Revolutionizing Healthcare & Life Sciences with AI at the Edge1:01:48Revolutionizing Healthcare & Life Sciences with AI at the Edge
Revolutionizing Healthcare & Life Sciences with AI at the Edge
Просмотров 463 месяца назад
This session will offer a deep dive into the latest innovations and opportunities that AI at the edge presents for the healthcare and life sciences industries. Our experts will share cutting-edge solutions and real-world applications that are enabling healthcare providers to enhance patient outcomes, streamline operations, and reduce costs.
Deploying and Monitoring LLM Inference Endpoints in Wallaroo58:29Deploying and Monitoring LLM Inference Endpoints in Wallaroo
Deploying and Monitoring LLM Inference Endpoints in Wallaroo
Просмотров 923 месяца назад
In this session we dive into deploying LLMs to Production Inference Endpoints and then putting in place automated monitoring metrics and alerts to help track model performance and suppress potential output issues such as toxicity. We also cover the process of optimizing LLMs using RAG, for relevant, accurate, and useful outputs. You will leave this session with a comprehensive understanding abo...
Ethical AI, Training on Youtube Content, & CoSAI - Humans Talking About AI Pod Episode #319:16Ethical AI, Training on Youtube Content, & CoSAI - Humans Talking About AI Pod Episode #3
Ethical AI, Training on Youtube Content, & CoSAI - Humans Talking About AI Pod Episode #3
Просмотров 203 месяца назад
Ethical AI, Training on RUclips Content, & CoSAI - Humans Talking About AI Pod Episode #3
AMD Acquires Silo AI - Humans Talking About AI - Episode 215:10AMD Acquires Silo AI - Humans Talking About AI - Episode 2
AMD Acquires Silo AI - Humans Talking About AI - Episode 2
Просмотров 1213 месяца назад
AMD Acquires Silo AI - Humans Talking About AI - Episode 2
Building Lean & Nimble AI Teams - Humans Talking About AI - Episode 115:31Building Lean & Nimble AI Teams - Humans Talking About AI - Episode 1
Building Lean & Nimble AI Teams - Humans Talking About AI - Episode 1
Просмотров 254 месяца назад
Building Lean & Nimble AI Teams - Humans Talking About AI - Episode 1
Navigating the Terrain: Challenges and Strategies in Deploying LLMs in Production.1:00:09Navigating the Terrain: Challenges and Strategies in Deploying LLMs in Production.
Navigating the Terrain: Challenges and Strategies in Deploying LLMs in Production.
Просмотров 574 месяца назад
Navigating the Terrain: Challenges and Strategies in Deploying LLMs in Production.
Production AI: Automated Monitoring of LLMs in Wallaroo4:52Production AI: Automated Monitoring of LLMs in Wallaroo
Production AI: Automated Monitoring of LLMs in Wallaroo
Просмотров 234 месяца назад
Production AI: Automated Monitoring of LLMs in Wallaroo
Production AI: Deploying LLMs to Production in Wallaroo2:22Production AI: Deploying LLMs to Production in Wallaroo
Production AI: Deploying LLMs to Production in Wallaroo
Просмотров 374 месяца назад
Production AI: Deploying LLMs to Production in Wallaroo
Production AI: Deploying and Monitoring LLMs in Wallaroo7:21Production AI: Deploying and Monitoring LLMs in Wallaroo
Production AI: Deploying and Monitoring LLMs in Wallaroo
Просмотров 684 месяца назад
Production AI: Deploying and Monitoring LLMs in Wallaroo
Edge AI Inference Endpoint: Inline Model Updates in Wallaroo4:50Edge AI Inference Endpoint: Inline Model Updates in Wallaroo
Edge AI Inference Endpoint: Inline Model Updates in Wallaroo
Просмотров 3125 месяцев назад
Edge AI Inference Endpoint: Inline Model Updates in Wallaroo
Beyond Edge AI: Deployment Manage, Observe, Update44:18Beyond Edge AI: Deployment Manage, Observe, Update
Beyond Edge AI: Deployment Manage, Observe, Update
Просмотров 325 месяцев назад
Beyond Edge AI: Deployment Manage, Observe, Update
Edge AI Inference Endpoint Part 3: Monitoring Edge Inference Models for Data Drift in Wallaroo2:41Edge AI Inference Endpoint Part 3: Monitoring Edge Inference Models for Data Drift in Wallaroo
Edge AI Inference Endpoint Part 3: Monitoring Edge Inference Models for Data Drift in Wallaroo
Просмотров 225 месяцев назад
Edge AI Inference Endpoint Part 3: Monitoring Edge Inference Models for Data Drift in Wallaroo
Edge AI Inference Endpoint Part 2: Monitoring Edge Inference Logs in Wallaroo0:42Edge AI Inference Endpoint Part 2: Monitoring Edge Inference Logs in Wallaroo
Edge AI Inference Endpoint Part 2: Monitoring Edge Inference Logs in Wallaroo
Просмотров 275 месяцев назад
Edge AI Inference Endpoint Part 2: Monitoring Edge Inference Logs in Wallaroo
Edge AI Inference Endpoint Part 1: Deploy and Serve Models to the Edge in Wallaroo2:00Edge AI Inference Endpoint Part 1: Deploy and Serve Models to the Edge in Wallaroo
Edge AI Inference Endpoint Part 1: Deploy and Serve Models to the Edge in Wallaroo
Просмотров 225 месяцев назад
Edge AI Inference Endpoint Part 1: Deploy and Serve Models to the Edge in Wallaroo
Production AI: Anomaly Detection with Wallaroo2:43Production AI: Anomaly Detection with Wallaroo
Production AI: Anomaly Detection with Wallaroo
Просмотров 337 месяцев назад
Production AI: Anomaly Detection with Wallaroo
Production AI: Model Hot Swap in Production with Wallaroo1:54Production AI: Model Hot Swap in Production with Wallaroo
Production AI: Model Hot Swap in Production with Wallaroo
Просмотров 297 месяцев назад
Production AI: Model Hot Swap in Production with Wallaroo
Production AI: Custom Inference Pipelines with Wallaroo2:39Production AI: Custom Inference Pipelines with Wallaroo
Production AI: Custom Inference Pipelines with Wallaroo
Просмотров 267 месяцев назад
Production AI: Custom Inference Pipelines with Wallaroo

Комментарии

  • @CommenterIndeed
    @CommenterIndeed 3 месяца назад

    To train a doctor, send him/her through university, medical school, etc. How much does it cost to train one doctor? Now times that by millions of doctors in the world. Software can be used by the world. Software is updated regularly as new drugs and new diagnostic techniques are developed. NO more cramming, with short term memories that ONLY lasts 7 days after the examination. Make sense? If there are so much resources and energy, then invest that in innovation = more worth while. Make sense?

  • @CommenterIndeed
    @CommenterIndeed 3 месяца назад

    Ai = programmed by humans. Future = all routine work are done by Ai -> much more efficient and cost effective.

  • @chrisbrien105
    @chrisbrien105 3 месяца назад

    LLM are not AI, so no worries

  • @colinclarke5386
    @colinclarke5386 7 месяцев назад

    Promo'SM

  • @RonnieSwafford-l9d
    @RonnieSwafford-l9d Год назад

    I love it! "We want to minimize the angry, mean grandma...". This is how I like to talk about AI! The potential is infinite and the future is limitless.