Видео

You're doing a fantastic job! I have a quick question: My OKX wallet holds some USDT, and I have the seed phrase. (alarm fetch churn bridge exercise tape speak race clerk couch crater letter). What's the best way to send them to Binance?

Great work on this project! I’ve loved the simplicity + perf when using it.

Nice presentation! This feature may significantly reduce prefill recomputation in the long contexts.

And can it run out of the box or difficult to execute it on amd?

So at the end is vllm running on amd gpu good?

when trying to run EAGLE model in openai api_server, how should you format the script? Since EAGLE doesn't use a draft model in its set up, I'm not sure how to make it work

12:12

which chat ui is being used in the end?

Thank you for the video. Do you expect it to work with Multimodal Models like llama-3.2-vision and Pixtral?

7:16 으앙 찾던 내용이다 아키텍처 아저씨 최고 최고 최고

45:04 6버전 이야기

7:08

Hi, great video, I would like to know if it is possible to use e5-mistral-7b-instruct in VLLM for embedding and completions with only one instance of VLLM?

Excellent progress and very informative. Thank you Neural Magic and team from your innovation and fantastic contributions.

Great session - thanks for posting!

28:12 AWQ?

I have a question!. I'm serving multimodal on vLLM, quantized (InternVL2) on L4 , it takes ~5-6 secs to complete a request, so when multiple request hit at a time, it takes much time ~30 secs to complete the requests. how to handle it like multiple requests also gets completed in ~5 secs. I have less understanding in batch_requesting and all.

Say code opensource but where?

Please share the source code

Please share the source code of quantization

Can Deepsparse run on Raspberry Pi AI kit for even faster fps?

mam my question is that if i have trained a model on simple yolov8 from ultralytics i got the best.pt file as my trained model can i directly remove unwanted weights from that from your technique or i have to complete train the model through your technique to get that right?

mam my question is that if i have trained a model on simple yolov8 from ultralytics i got the best.pt file as my trained model can i directly remove unwanted weights from that from your technique or i have to complete train the model through your technique to get that right?

HI! We are creating a system that classifies tomato ripeness levels using image processing in CNN architecture with YOLOv8 model. We are using Raspberry Pi 4 OS with 4GB RAM and we have encountered problem - the system has 2-3 minutes delay/lag in classifying the ripeness level. Would you happen to have any recommendation/suggestion sir on this problem?

Link to collab is not available now

Can you make a tutorial of practically pruning some GAN model like GFPFGAN model ?

Could you clarify if these pruning strategies are post training or training aware. It seems like progressive sparsification is training aware but from what i recall woodfisher approach is post training and require some fine tuning at the end ?

In the recipes , its shown we do distillation to recover accuracy and then followed by quantization . Curios do you observe degradaiton in quantization step , why not distill or fine tune post quantization ? Also with this recipes , does training become more expensive compared to base dense models ? are there any data comparing the training cost and time

Wow, very insightful interview!

Any C++ API for this?

Hi, I am really interested in improving inference time for YOLO models in CPU. Here are some questions I have: (1) After using a SparseZoo recipe to apply to our data, using "sparseml.ultralytics.train", what is the format of the generated weights? (2) Moreover, is it possible to import the generated weights for the sparsified model in OpenCV using: cv2.dnn.readNetFromONNX('yolov8n_sparsified.onnx')? (3) As far as I see in all the Neural Magic repositories, all of them have Apache License 2.0. Is this correct? (4) Are there any commercial restrictions of using cv2.dnn.readNetFromONNX('yolov8n_sparsified.onnx')? Many thanks in advance

Many thanks for this video. I have several questions: (1) After "applying to our data" using a SparseZoo recipe to apply to our data, using "sparseml.ultralytics.train", what is the format of the generated weights? (2) Moreover, have you tried to import the generated weights for the sparsified model in OpenCV using: cv2.dnn.readNetFromONNX('yolov8n_sparsified.onnx') (3) And finally, I have a question with the license. As far as I see in all the Neural Magic repositories, all of them have Apache License 2.0. Is this correct? (4) So are there any commercial restrictions of using cv2.dnn.readNetFromONNX('yolov8n_sparsified.onnx')? Many thanks!!!

@Neural Magic You are building amazing tools but the demos aren't good, I don't like these PPT explanations, show us the practical demos and all the different tool and methods you have. I still can't figure out how do I train my yolov5 with custom dataset using sparcing. I still can't figure out what is deep sparce, what is sparifying, sparceML, sparceZoo...and... I just want to know if I have to train my custom dataset with yolov5 with your sparcing method. I should see complete code examples for that, and what if I have already trained weights, how can we apply pruning and quantization. It's very confusing as none of your article explain things completely. There should have been hands on videos or articles. As a beginner it is difficult to understand. By the way you should try Roboflow style demos, those are great.

Hi there, What are the hardware specifications of GPU to run yolov8 model , I have Nvidia GT 730 and while running the model it is giving me "cuda: no kernel image is available for execution on the device."

hello , i followed all steps and i now have best_pruned.onnx model file, deep sparse is giving me the ability to only test on an image how can i deploy my model on a camera live stream or a video file ? Thank you

29:28 This truly is a game changer!

This is truly ground breaking.... you guys are doing phenomenal work...

Amazing stuff!

LLMs running on CPUs is ground breaking. You guys are doing amazing work !

Hi, do you guys have any ML intern position

Is it possible to write a yolov5 object recognition application using neural magic on windows?

Has anyone, outside of the NeuralMagic team, pruned their own model with SparseML and can confirm these claims?

Hello @Neural Magic can i follow the same steps for training the model in windows??

this is great work. Is there any research on the ideal sparse training to dense training ratios?

What about LLM's and/or Stable Diffusion models? Can your techniques be used width GPT's?

Very well done!

Congrats on making it to EMNLP 2022! Also, for those who can't take pineapple pizzas, you have no idea what you are missing out.

we need yolo v7 deep sparse. is that too much to ask for?

When big NLP models like GPT-2, M2M100 etc.? 😞😭

WHEN YOLOV7 SPARSE COMING