It's funny how this feature has been in Gemini for months, but suddenly everyone's buzzing about it because OpenAI released it. No shade, just an observation! 😉
this is the best feature since I started working with chatgpt 3.5 a year ago... it brings some reason and sanity to the prompting abracadabra and the way openai solved it and presented is beautiful
I haven't really understoo the buzz around this at all, I've been getting 99% consistent JSON outputs from LLama and GPT for months. This basically boils down to a template in the system prompt, and a simple json cleaning script - Most of the failures I was getting before this were to do with random commas and unclosed brackets, sometimes a plain text "Sure I can do that for yo" at the begining, but this was algorithmically easy to fix. Very weird, maybe I'm missing something but seems excessive to reduce the output creativity of an LLM by fine tuning it to json only.
doing it at the model level paves the way for this to be used more with things like their own internal "thinking steps" (a bit similar to Antrhopic's Ant thinking) which would be very helpful for ideas like q* and strawberry etc. I agree that most of this could be done before with prompts and tricking function calling etc
About the zero-retention comment at the end of the clip: OpenAI states that they do not train on any content send via the API. Zero-retention is a use case for sensitive data that requires additionally a BAA, the signing of the data processing agreement and the use of specific endpoints, as you point out correctly. PII or information covered by HIIPA are typical uses here. Otherwise all data is kept for 30 days in case of legal stuff, safety issues, account abuse etc... Hope this helps!
🎯 Key points for quick navigation: 00:00:00 *🆕 Overview of New JSON Structured Outputs* - OpenAI has introduced guaranteed JSON responses in their API. - Previously, consistent JSON responses required multiple attempts and external tools. - Now, JSON responses are assured, though accuracy may still vary. 00:02:29 *🛠️ Setting Up Pydantic and Zod for Data Models* - The video demonstrates setting up data models using Pydantic (Python) and Zod (JavaScript). - Pydantic and Zod are used to define and constrain data types for better accuracy. - This section shows practical implementation for extracting and categorizing data from articles. 03:27:00 *🧩 Extracting and Categorizing Data from Articles* - A historical example of extracting data from articles to build knowledge graphs. - Demonstrates named entity recognition (NER) and its limitations. - Focuses on improving accuracy for product and organization identification using the new API features. 08:23:00 *📊 Demonstrating Model Outputs and Accuracy* - The practical example shows running OpenAI's model on various articles. - Results include extracting names, products, organizations, and summaries from the text. - Highlights both successful extractions and areas where the model might struggle. 12:20:00 *🌐 Future Use Cases and Cost Considerations* - Discusses potential use cases such as sentiment analysis and automated reporting. - Notes the reduction in API costs and the importance of understanding data retention policies. - Emphasizes creative applications of the new structured output capabilities. Made with HARPA AI
With future increased compute power and reduced token cost ----> automatic property/knowledge graphs produced on every document as standard beginning context ----> for all agentic actions and reasoning tasks. Ie hybrid GraphRag today, Langgraph property graphs, tree of thought reasoning paths at forest level ---> for all data.
It is true orchestration frameworks have done things similar but they couldn't do it at the model level before, which is the change here. Now that said maybe for many case that doesn't matter, the old way was fine etc.
So can we parse the candidate resumes, pre-define the keys(name,skills,experience etc) and make the LLM process , store it in respective keys..so this feature supports this use case right?..I was hard-coding using regex and try to generalize the template (which was definitely painful and like brute force thing) 😶🌫😶
How close are we to being able to merge the knowledge and sentiment extracted from simultaneous conversations with millions of people around the world, for a collective human and digital intelligence?
If you have the data and the money this could be done now. Already big companies do this on their customers. Sentiment doesn't really get a lot of intelligence though.
@samwitteveenai You pointed out that the structured output guarantees correct json format, but not correct entity extraction. That means a human in the loop still needs to be part of the system. Good.
@@samwitteveenaiI am a retired librarian, and don't have a lot of money. I do have a Oneplus 11 smartphone with 16 GB of RAM, which could be part of a global platform for collective intelligence. My on-device digital agent could visualize the graph it has constructed of our conversation for a sanity check before submitting it to the shared graph. This would also give me a chance to prevent sharing something that I want to keep private. Just thinking out loud here.
@@johnkintree763 I agree things like this are very interesting and the tech is well along the way there. The biggest challenge against this kind of thing is privacy. Even anonymizing data gets massive push back and sometimes rightly so .
Hey Sam I've followed your work since the Langsmith first beta. I want to talk to you, what's your best channel to have a meeting, LinkedIn? Cheers from down under!
It's funny how this feature has been in Gemini for months, but suddenly everyone's buzzing about it because OpenAI released it. No shade, just an observation! 😉
gemini version coming with images to JSON as well
I wasn't aware, but use both. Thank you for pointing that out, so cool!
Exactly. I was also wondering the same
This requires an industry-wide update for error and authentication management for all LLMs and authentication management agents for all LLMs.
this is the best feature since I started working with chatgpt 3.5 a year ago... it brings some reason and sanity to the prompting abracadabra
and the way openai solved it and presented is beautiful
There has been a way to get stuctured outputs for a while right?
I haven't really understoo the buzz around this at all, I've been getting 99% consistent JSON outputs from LLama and GPT for months. This basically boils down to a template in the system prompt, and a simple json cleaning script - Most of the failures I was getting before this were to do with random commas and unclosed brackets, sometimes a plain text "Sure I can do that for yo" at the begining, but this was algorithmically easy to fix. Very weird, maybe I'm missing something but seems excessive to reduce the output creativity of an LLM by fine tuning it to json only.
doing it at the model level paves the way for this to be used more with things like their own internal "thinking steps" (a bit similar to Antrhopic's Ant thinking) which would be very helpful for ideas like q* and strawberry etc. I agree that most of this could be done before with prompts and tricking function calling etc
About the zero-retention comment at the end of the clip:
OpenAI states that they do not train on any content send via the API. Zero-retention is a use case for sensitive data that requires additionally a BAA, the signing of the data processing agreement and the use of specific endpoints, as you point out correctly. PII or information covered by HIIPA are typical uses here.
Otherwise all data is kept for 30 days in case of legal stuff, safety issues, account abuse etc...
Hope this helps!
🎯 Key points for quick navigation:
00:00:00 *🆕 Overview of New JSON Structured Outputs*
- OpenAI has introduced guaranteed JSON responses in their API.
- Previously, consistent JSON responses required multiple attempts and external tools.
- Now, JSON responses are assured, though accuracy may still vary.
00:02:29 *🛠️ Setting Up Pydantic and Zod for Data Models*
- The video demonstrates setting up data models using Pydantic (Python) and Zod (JavaScript).
- Pydantic and Zod are used to define and constrain data types for better accuracy.
- This section shows practical implementation for extracting and categorizing data from articles.
03:27:00 *🧩 Extracting and Categorizing Data from Articles*
- A historical example of extracting data from articles to build knowledge graphs.
- Demonstrates named entity recognition (NER) and its limitations.
- Focuses on improving accuracy for product and organization identification using the new API features.
08:23:00 *📊 Demonstrating Model Outputs and Accuracy*
- The practical example shows running OpenAI's model on various articles.
- Results include extracting names, products, organizations, and summaries from the text.
- Highlights both successful extractions and areas where the model might struggle.
12:20:00 *🌐 Future Use Cases and Cost Considerations*
- Discusses potential use cases such as sentiment analysis and automated reporting.
- Notes the reduction in API costs and the importance of understanding data retention policies.
- Emphasizes creative applications of the new structured output capabilities.
Made with HARPA AI
With future increased compute power and reduced token cost ----> automatic property/knowledge graphs produced on every document as standard beginning context ----> for all agentic actions and reasoning tasks. Ie hybrid GraphRag today, Langgraph property graphs, tree of thought reasoning paths at forest level ---> for all data.
Love your videos, they contain so interesting details.
Can we make the quality of the respond better with prompting
Good question. Certainly the prompt will have some impact of the quality of the output.
Would that Marvin do it even in 2023 already? Like Apple re-develop new features by looking what great exists in its App Store
It is true orchestration frameworks have done things similar but they couldn't do it at the model level before, which is the change here. Now that said maybe for many case that doesn't matter, the old way was fine etc.
So can we parse the candidate resumes, pre-define the keys(name,skills,experience etc) and make the LLM process , store it in respective keys..so this feature supports this use case right?..I was hard-coding using regex and try to generalize the template (which was definitely painful and like brute force thing) 😶🌫😶
yes can totally do something like this.
I'm trying like heck to find that ZDR statement anywhere on the open ai site and failing. if you have a link that would be greatly appreciated.
Here you go platform.openai.com/docs/models/how-we-use-your-data
How close are we to being able to merge the knowledge and sentiment extracted from simultaneous conversations with millions of people around the world, for a collective human and digital intelligence?
If you have the data and the money this could be done now. Already big companies do this on their customers. Sentiment doesn't really get a lot of intelligence though.
@samwitteveenai You pointed out that the structured output guarantees correct json format, but not correct entity extraction. That means a human in the loop still needs to be part of the system. Good.
@@samwitteveenaiI am a retired librarian, and don't have a lot of money. I do have a Oneplus 11 smartphone with 16 GB of RAM, which could be part of a global platform for collective intelligence.
My on-device digital agent could visualize the graph it has constructed of our conversation for a sanity check before submitting it to the shared graph. This would also give me a chance to prevent sharing something that I want to keep private. Just thinking out loud here.
@@johnkintree763 I agree things like this are very interesting and the tech is well along the way there. The biggest challenge against this kind of thing is privacy. Even anonymizing data gets massive push back and sometimes rightly so .
thanks:)
4o-mini tends to misbehave. 4o works fine for normal use case but too expensive.
I'll make a video of how to do it with Gemini Flash which is dirt cheap
Hey Sam I've followed your work since the Langsmith first beta. I want to talk to you, what's your best channel to have a meeting, LinkedIn? Cheers from down under!
LinkedIn is easiest