Видео

Hey Elvis! This is a great overview of the paper. We work on structured generation at .txt and took the claims in this paper very seriously. We recently took a good look at the data and code behind this paper and published a response. We found several issues in the paper regarding the claim that "structured generation constraints significantly impact LLM performance across various tasks". Here's a quick summary of our findings: 1. The paper itself finds that structured generation has superior performance on a number of classification tasks (thanks for calling this out in the video!). 2. The prompts used for unstructured (NL) generation are markedly different than the ones used for structured generation, so the comparisons are not apples-to-apples to begin with. 3. The structured generation prompts do not provide the model with adequate information to solve the task, this leads to particularly poor performance for the ‘json-mode’ examples. 4. The real meat of the paper is actually about parsing the results of one LLM with a second LLM. The authors refer to this as the “Perfect Text Parser”. 5. The paper confuses structured generation with JSON-mode, although independent runs of these evals show that “JSON-mode” yields better results than unstructured generation. If you'd allow it I'd love to post a link to the response, but don't want spam your comments section! 😂

How much is this the underlying model being really good and how much of it is a really well done chain of thought prompt?

GitHub have api very well documentade.

I am using DeepSeek 2.5 since a while. In many tasks this LLM beats ChatGpt, Google Gemini and Claude Sonnet. It is slower but I like the usefulness of the responses. I assume that at DeepSeek smart people are developing useful models which are working well with less advanced compute. Banning modern NPU/GPU chips from China represents a clear incentive to develop LLMs which are running with less NPU/GPU requirements. That will pay off soon for the Chinese AI developers while US providers like OpenAI and Microsoft will be drowning in their compute cost.

Not sure about the API version, but the chat version sucks.

Can you make a video comparing it with another model in a environment

Same deepSeek, same problem, giving different result

Peter has 5 candles that are all the same length. He lights them all at the same time. After a while, he blows out the candles one after the other. Which of the five candles was the first one he has blown out? Here is a figure of the five candles after they have been blown out. The number of = represents the length of the candle. Respond with the label of the candle that has been blown out first by Peter. 1. ==== 2. ======= 3. ======== 4. = 5. ==

Okay, let's see. I have this problem about Peter and his candles. He has five candles, all the same length, and he lights them all at the same time. After some time, he blows them out one after the other, and then there's a figure showing the lengths of the candles after they've been blown out. The lengths are represented by the number of equal signs. So, candle 1 has four equals, candle 2 has seven, candle 3 has eight, candle 4 has one, and candle 5 has two. First, I need to understand what's going on here. All candles are the same length initially, and they're all lit at the same time. That means they should be burning at the same rate, assuming they're made of the same material and are in the same environment. But then Peter blows them out one after the other, and we see their lengths after that. I think the key here is to figure out which candle was blown out first based on how much it has burned. Since they all started at the same length and were burning at the same rate, the one that was blown out first should have burned the least, right? Because it had less time to burn before being extinguished. Looking at the lengths after being blown out: candle 3 is the longest with eight equals, then candle 2 with seven, candle 5 with two, candle 1 with four, and candle 4 with one equal sign. Wait, actually, candle 4 is the shortest with just one equal sign, and candle 3 is the longest. So, if the one that was blown out first has the least time to burn, it should be the one with the least length burned off. But I need to think carefully here. The number of equals represents the length remaining, not the length burned. Wait a minute, if the equals represent the length remaining, then the candle with more equals means it has burned less, and the one with fewer equals has burned more. So, the candle that was blown out first would have had the least time to burn, meaning it has the most length remaining, which is candle 3 with eight equals. But, let's double-check. If all candles were lit at the same time and then blown out one after the other, the first one blown out would have burned for the shortest time, so it should have the most remaining length. Conversely, the last one blown out would have burned the longest, having the least remaining length. Given that, candle 3 has the most remaining length, so it should be the first one blown out. But, there's a twist. The problem says "Which of the five candles was the first one he has blown out?" and provides the lengths after being blown out. Wait, perhaps I should consider the burning rates differently. Maybe the candles have different burning rates even if they are the same length? But the problem says they are all the same length and presumably made of the same material, so they should burn at the same rate. Alternatively, maybe the number of equals represents the length burned, not the remaining length. But that doesn't make sense because it says "the number of = represents the length of the candle." Hmm, this is confusing. Let me read the problem again. "Here is a figure of the five candles after they have been blown out. The number of = represents the length of the candle." So, the equals represent the length remaining after being blown out. Therefore, the candle with the most equals has the most length remaining, meaning it burned the least, which would be the first one blown out. Therefore, candle 3 with eight equals is the first one blown out. But let's think about this again. If all candles are burning at the same rate, and the first one is blown out, it has less time to burn, so more length remains. The next one is blown out a bit later, so it has burned a little more, and so on. So, the order from most remaining length to least remaining length should correspond to the order in which they were blown out, from first to last. So, candle 3: eight equals (most remaining, first blown out) Then candle 2: seven equals Then candle 1: four equals Then candle 5: two equals Then candle 4: one equal (least remaining, last blown out) Yes, that seems logical. But perhaps there is another way to look at it. Maybe considering the time each candle was burning before being blown out. Assuming they all started at the same time, and were blown out one after the other, the first blown out has the shortest burning time, hence most remaining length. Yes, that aligns with what I thought earlier. So, the answer should be candle 3. But the problem is presented in a way that might be trying to trick me, so I need to be sure. Another way to approach this is to consider the total burn time. Let’s say each candle burns at a rate of 1 unit per minute. They all start burning at time t=0. Suppose Peter blows out the first candle at time t1, the second at t2, and so on, with t1 < t2 < t3 < t4 < t5. Then, the remaining length of each candle would be original length minus burn rate times the time it was burning before being blown out. Since original lengths are the same and burn rate is the same, remaining length is original length minus t1, original length minus t2, etc. So, the candle with the most remaining length has the smallest t, meaning it was blown out first. Yes, this confirms the earlier conclusion. Therefore, candle 3, with eight equals, is the first one blown out. Peter lights all five candles at the same time. Since they are the same length and burn at the same rate, the candle that was blown out first would have burned for the shortest time and thus has the most remaining length. The figure shows that candle 3 has the most remaining length with eight "=" symbols. Therefore, candle 3 was the first one Peter blew out. Answer: Candle 3

The first candle blown out should have been the longest one. The LLM was incorrect saying it would be the shortest.

The subject is so wide that they deserve their own pre-print service.

❤

Learn how to build with AI in my new courses here: dair-ai.thinkific.com/ Use code BLACKFRIDAY to get a 35% off. The offer expires 11/29.

no answer how to enable Claude. dislike

This is a really well presented and concise video! Excited to see what OpenAI and Anthropic roll out in the coming months, we've reached the point where AI is actually useful and capable

Thanks Elvis for sharing your thoughts on this new development.

aaaaaaaannnnndddd THEY STILL CAN'T SEE (socialism/equality) YET???

Learn how to build with AI in my new courses here: dair-ai.thinkific.com/ Use code RUclips20 to get an extra 20% off. Limited-time offer!

Cool

You should mention the name of the website.

Learn how to build with AI in my new courses here: dair-ai.thinkific.com/ Use code RUclips20 to get an extra 20% off. Limited-time offer!

Thanks for this course, even with a newbie like me, it was very usueful

Good walkthrough and some interesting thoughts shared, specifically about in addition to covering more programming languages, covering more frameworks like next js. What i would like to add is: is there a way the code llm can be provided the latest documentations (the version specific documentation) of any framework, that is rapidly under development, (example: langchain) so that the llm can leverage it to better assist the developer at debugging and developing....

Learn how to build with AI in my new courses here: dair-ai.thinkific.com/ Use code RUclips20 to get an extra 20% off. Limited-time offer!

Claude and Chat-GPT is very sensitive and politically correct. I asked Claude if a certain gun's original magazine was better quality than a different reputable brand and it didn't answer me as it could cause harmful misuse? What the hell even is it on about? Grok on the other than directly answered me. Unfortunately depending on the topic you have to use different AI unless Grok becomes best at everything you use AI for.

NotebookLM made me fall in love with teaching again!

Eventually the people who authors PDFs should just go straight to make them AI friendly and AI first. For people, they should consume the content in more friendly format such as podcast, voice summary etc.

Whats llm do you use to resume article? the same question to use in vídeo

for deeper and better analysis of pdf's I recommend Google's "NotebookLM". And it's free.

how to use

Grata sorpresa ver que sabes hablar en español ¡Saludos desde Chile! a video for the analysis tool would be great

wait a long time for Haiku 3.5 model ..apparently the price hmm better switch to gpt 4o mini

But ChatGPT can do the same. Do you think the accuracy on Claude is higher?

Can it understand the relevant topics of the PDF, how they relate, then generate a JSON file so that we can use it with React Flow?

Learn how to build with AI in my new courses here: dair-ai.thinkific.com/ Use code RUclips20 to get an extra 20% off. Limited-time offer!

Thanx!!

Can I use the sonnet model with the gitcopilot individual plan?

Thank you for the thorough explanation and for walking through multiple examples. Your effort is greatly appreciated!

Enjoying hearing from a Belizean and Caribbean man on a technical topic.

wait, some people said its mean 3x cheaper than sonnet ?

Nonsense price. A bad decision of the maeketing department.

Thanks man this clears up alot i main use the Haiku for some of my little projects. Please can you also cover the catching, that will be great

Microsoft may have squeezed them somewhere (for Copilot adoption), hence the need to reprice their cheaper model.

If that is the price i’ll just use mistral large 2

They can prince it however they want. And when noone shows up, theyll change to the market levels, having wasted alot of customer goodwill.

There's no sense to increase 4x ...

Learn how to build with AI in my new courses here: dair-ai.thinkific.com/ Use code RUclips20 to get an extra 20% off. Limited-time offer!

but i dont see this feature in my extension ?! i see only old coplilot

Layman's terms/peasant terms: for example can scrape an entire website like property taxes map keys in my area and find houses paid off with at least 2 acres and names then put those into a neat list?

if i didn't have such Trust issues with Google... i would consider using this. BUT! since Google will use your data and not tell you about it.. will keep all my research off of Google's grubby little paws