Optimizing and Fine-Tuning Large Language Models on Intel Edge Developer Cloud with OpenVINO
In this video, we delve into the intricacies of running and optimizing large language models (LLMs) using Intel Edge Developer Cloud and Intel OpenVINO. Here's what we cover:
1. Environment Setup
Platform: Intel Edge Developer Cloud
Kernel: Python 3.10 (OpenVINO Notebooks 2024.1.0)
CPU: Intel(R) Xeon(R) Gold 5218R CPU @ 2.10GHz
2. Model Selection and Configuration
Inference Model: neural-chat-7b-v3-1
Fine-Tuning Model: Meta-LLama-2-7b-hf
Dataset: Subset of "wikitext-2-raw-v1" from Hugging Face datasets library
3. Model Optimization with OpenVINO
FP16 Conversion: Enhancing performance with half-precision floa...