ChatGPT from Scratch: How to Train an Enterprise AI Assistant • Phil Winder • GOTO 2023

This presentation was recorded at GOTO Copenhagen 2023. #GOTOcon #GOTOcph
gotocph.com
Phil Winder - CEO of Winder.AI, Author of "Reinforcement Learning" ‪@philwinder263‬
RESOURCES
/ drphilwinder
/ drphilwinder
github.com/philwinder
winder.ai
winder.ai/blog
Links
arxiv.org/abs/2203.02155
arxiv.org/pdf/2201.05601
arxiv.org/abs/2305.11206
arxiv.org/pdf/2210.15424
arxiv.org/pdf/2306.01116
arxiv.org/pdf/1904.09483
arxiv.org/pdf/2306.01116
arxiv.org/pdf/2309.11295
github.com/Zjh-819/LLMDataHub
arxiv.org/abs/2210.15424v1
github.com/huggingface/peft
github.com/TimDettmers/bitsan...
github.com/AutoGPTQ/AutoGPTQ
mr-ranedeer.com
llm-attacks.org
winder.ai/using-reinforcement...
github.com/huggingface/text-g...
github.com/vllm-project/vllm
developer.nvidia.com/triton-i...
www.langchain.com
arstechnica.com/information-t...
ABSTRACT
In today's fast-paced business environment, the demand for intelligent enterprise assistants that can help optimize workflow, handle customer queries, and even automate tasks is at an all-time high. But how do you go about creating a powerful and reliable conversational agent like ChatGPT? This tutorial aims to answer exactly that question.
We invite developers, data scientists, and tech enthusiasts to a rapid tutorial on building Large Language Models (LLMs) tailored for enterprise applications. We will delve into the architecture, training methodologies, data pipeline construction, and optimization techniques required for creating a state-of-the-art enterprise assistant. Participants will gain experience with LLM with a comprehensive LLM walkthrough along with the foundational knowledge required to build, fine-tune, and deploy LLMs in an enterprise setting.
Key Takeaways:
• A Brief History of LLMs: Tracing the evolutionary journey of Large Language Models from their rudimentary forms to cutting-edge architectures like GPT-4, and understanding their impact on the field of Natural Language Processing (NLP).
• Understanding the core architecture and components of Large Language Models like ChatGPT.
• Techniques for curating and pre-processing domain-specific datasets that result in a highly specialized and efficient LLM.
• Strategies for efficient and cost-effective model training, from fine-tuning pre-trained models to training from scratch.
• Deployment considerations for LLMs, including the use of cloud-based services like AWS and Azure.
• Security and ethical considerations in deploying LLMs in a business environment, including data privacy and model interpretability.
By the end of the tutorial, attendees will have a working knowledge of LLMs and the confidence to prototype intelligent conversational agents for their organizations. Whether you are a novice exploring the world of NLP and machine learning or an experienced developer looking to upskill, this tutorial has something for everyone. Come join us as we demystify the intricacies of developing enterprise-grade LLMs! [...]
TIMECODES
00:00 Intro
01:33 1. History
06:01 2. Core architecture & components
12:33 3. Data preparation
20:36 4. Modelling & training
26:20 5. Deployment
30:22 6. Extras
36:53 7. Demo
48:05 Outro
Download slides and read the full abstract here:
gotocph.com/2023/sessions/2906
RECOMMENDED BOOKS
Phil Winder • Reinforcement Learning • amzn.to/3t1S1VZ
Holden Karau, Trevor Grant, Boris Lublinsky, Richard Liu & Ilan Filonenko • Kubeflow for Machine Learning • amzn.to/3JVngcx
Kelleher & Tierney • Data Science (The MIT Press Essential Knowledge series) • amzn.to/3AQmIRg
Lakshmanan, Robinson & Munn • Machine Learning Design Patterns • amzn.to/2ZD7t0x
Lakshmanan, Görner & Gillard • Practical Machine Learning for Computer Vision • amzn.to/3m9HNjP
Aurélien Géron • Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow • amzn.to/2XZaQy8
/ gotocon
/ goto-
/ goto_con
/ gotoconferences
#ChatGPT #LLM #AIAssistant #LargeLanguageModels #AITutorial #AIDemo #NaturalLanguageProcessing #NLP #Huggingface #ReinforcementLearning #PhilWinder
Looking for a unique learning experience?
Attend the next GOTO conference near you! Get your ticket at gotopia.tech
Sign up for updates and specials at gotopia.tech/newsletter
SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily.
ruclips.net/user/GotoConf...

[@cyberchad]

Me on my way to add chatgpt clone on my resume

[@The.24_7.Operator]

same

[@stevenhe3462]

I do think one can melt an egg, but not with a regular oven. Lava should be expected.