Chat with Your SQL Data Using ChatGPT
HTML-код
- Опубликовано: 10 апр 2023
- In this video, we will use Open AI to chat with our SQL/Tabular data without writing any SQL query. We can ask any question about our database using conversation and the results will be retrieved and explained again without any coding.
☎️ Do you need any career or technical help? Book a call with me: calendly.com/mg_cafe
Part 1 - Chat With Your Data:
• Use Open AI (ChatGPT) ...
Part 2 - Chat With Your Data:
• Use ChatGPT On Your Ow...
Open AI+ Azure = Revolutionize the Way You Do Business
• Open AI+ Azure = Revol...
ChatGPT Came To Azure:
• ChatGPT Came To Azure!
Repo used in this video:
github.com/MG-Microsoft/ChatG...
*******************
LET'S CONNECT!
*******************
Join Discord Channel: / discord
✅ You can contact me at:
LinkedIn: / mohammad-ghodratigohar
Email: mo.ghodrati95@gmail.com
Twitter: / mg_cafe01
🔔 Subscribe for more cloud computing, data, and AI analytics videos
by clicking on the subscribe button so you don't miss anything.
#ChatGPT #OpenAI #SQL #GPT3 #GPT4 #machinelearning #cloudcomputing #chatbot #virtualassistant #customerservice #developers #technews #innovation #Microsoft #CognitiveServices #BotService #naturalanguageprocessing #NLP #conversationalAI #videotutorial #tutorial #learningAI #AIchatbot #AIassistant #AzureAPI #AzureTools #MG #Azure #LangChain 
Наука
Really cool to see MG. However, I was hoping this was going to be more of an expansion on Part 2 and showing how to hook up a SQL server to Azure Cognitive Search. Would love to see a video on that :)
This is gonna change the whole job of a data analyst to another LEVEL... Will try this and get back with doubts...👍 cool video
That’s exactly what I was looking for thanks so much
This is amazing content MG. Thanks alot for sharing this!
Hi "MG", it would be good to see a film about basic Azure ML configurations. The problem I see is that Azure ML provides so many possibilities for teams working on a business solution that it can be confusing for beginners. Basically, Microsoft does not introduce a "standard"; you can use everything that is available in any way, for example, Databricks cluster, MLflow, bamboolib - a GUI for pandas - here you go, GitLab plus Databricks - no problem, ONNX, Feast, OpenAI - why not. In my opinion, this is a big drawback because at the "start," I have to choose tools myself (know the pros and cons of these solutions), and I have to have a solution architecture established. For example, should I use the Run class and methods (log, log_accuracy_table, log_list, log_predictions) or maybe MLflow?
Hey bro !, Amazing video and very clear explanation. Very clear and comprehensive. I tried and understood the works. Thank you.
Hi MG, Can you please make a tutorial on how to connect it with SQL server as well.
I think I'm echoing what others are asking for: I have SQL 2016,2019 servers with multiple DBs. I would like an interface application similar to what you're using, I think. That interface can process the schemas of the databases and then help me form new queries and retune existing queries as required.
As I see it, your python interface application is producing a local .db file as source material for your questions. I'm hoping the interface can leave the data in place -due to its size. I imagine with the schemas, ChatGPT or some other AI engine can loosely understand the goals and produce, test, refine queries for me to QA and work with.
Perhaps this is not possible yet?
I suppose another issue is privacy and how I can use the engine without exposing my private data.
Amazing. The code worked perfectly!
Excellent video MG. Love the content. Only one query. If the data inside the table changes while we are conversing with ChatGPT, does the change reflect in the model's output ?
Hi MG, wondering if you have tested sequence of prompts (sql) one feed to others as part of analysis.
Hi MG! The video was really helpful. Can I know in details about the same scenario if we have the column's content of the data table in the jsonb that is "key-value pair" format?
@MG, how can you connect to existing server? Having SSMS, server name and lots of database in it. Without using/creating sqlite, can i directly connect langchain to existing SSMS? Thanks
Excellent video!!! Is it possible to add memory to the agent so that it remembers the questions and reduce the number of tokens used in them?
Amazing, Thanks MG.
Great Video, Is there a way to implement CRUD operations in the DB using Langchain and GPT? If the user types "I want to add a new artist", then the app should ask for the necessary details. Once the user enters the necessary details, it should create a new record in the Artist table.
This I great thanks for sharing. Will try it and see if I can have it generate sql query to build complex report joining those tables
This looks great. I would assume you do this thru Azure OpenAI too? Also, it would be great to see ChatGPT interact with tabular data in form of a CSV, pandas or pyspark dataframe
wow, this is really cool which will really help many of our scenario. Please share more of these kind of videos. Here you've linked with local SQLite, is there a way we can connect to external source in SQL Serverless in Synapse? (which has like GBs of data)
Hi! How can I retrain the model as to reduce the error in the translation natural language to query? Retraining is I guess almost absurd, but would you have any suggestion on how to correct these queries? For instance, I sometimes ask for an average and the query doesn't add a GROUP BY cluase.
You can run a stored procedure that brings back only the data you want and just place the data in a collections object.
Brilliant! I love it.
How do i setup set-up MySQL in Visual Studio Code. Thank you for the video.
Hi @MG, thanks a million for the wonderful video. I am trying this on my SQL DB and I get the token limit error. Can you please share any solution on how to handle the issue when applying it on a large DB?
Love your videos.
Hi MG! Nice video, that;s what I was looking for. Just one question, I am working in a company that make intranet system, so we have our own database and I guess servers. Can you please tell me what's the cost to setup this sort of system? Do you need to pay just openAI ? Would be great if you include pricing you need to set up the system! Btw +1 Subscribed ! :)
Thanks a lot MG!... Do you know how to connect Mongo DB with ChatGPT?
does this sql tables have to be relational tables? Can I use nosql for langchain?
Best intro ever!
Really great video..! Question: I have an Azure SQL database. I had to use SQL Alchemy to connect to it from OpenAI, and it is working fine, albeit a bit slow. Is SQL Alchemy still the only way to connect from OpenAI or is there a way to connect directly? Thank you!
Great video. Can I teach chatgpt to generate code in some custom script which it didn't see before?
Would we get better outputs if i move my RDBMS data to a db like Elastic Search?
Speaking of talking with you DB. What are you chances of you making a video on using the AskYourDatabase plugin for ChatpgtPlus.
very nice, thank you
Hi, Can we connect this to a MySQL database directly?
Also can OpenAI help us in modifying the data?
missing something..is there a sql agent loaded locally that you interact with which in turn accesses db?
Do conv go outside local, to openai servers or are all this local
(if not I dont like chatgpt learning my data and database structure)
Hi Thank you for the video. Can you please explain something on Database testing using OpenAI normal Language
can you do it for gpt4all local installation to communicate with an sql like the example? if yes how ?
Is it possible to get pie charts/bar charts using the chatbot? Basically, it should query data from SQL and plot graphs
Hi MG, Can we try something in the reverse direction. Example, given a query with lots of joins and aliases, can OpenAI figure out all the columns (both alias and real names) used in the query?
Want to know which tables and columns are used in the query.
is there a way to connect it to a SQL Server Management Studio database?
Hey, I have a question the gpt model has token problems you know we can use a limited number of tokens I mean we can not train a model with a large amount of data. Just we will need to divide it into smaller chunks. So does sql solve this issues I mean if I have a table of thousands of data and I ask question gpt to tell me about specific entry in database does I will get sane error or I can do that . Please answer . Thanks in advance
Can you maybe do a video where you show one of these models one can download, that is very good in SQL query creations and stuff like the video, that can also talk to a MS SQL database that is big. In other words, do something similar but not using ChatGPT where you have to pay for every large promnpt/result? This would be awesome.
When I use Langchain to connect my sql database to Openai api do my private data becomes accessible to OpenAI? I am asking this in context to data security. Suppose I am building a chatbot for my client and use this architecture then is there a possibility that the data is exposed to OpenAI no matter they use it or not to train their models?
Please how can we connect with data on AWS dynamodb?
Do we have an example with Angular?
Thankyou MG, for the amazing video. A couple of questions
1. It appears that you need to send a list of all the table names to chatgpt for each query which will cost money. Is there a way to avoid sending it each time let ChatGPT remember at least for that session if not permanent
2. If I migrate my RDBMS data into a vector database, would it be an overkill? any chatGPT cost minimization pro/cons?
is there a way to connect it to a PosgreSQl DB?
what is the privacy that we get from this. lets say we use this for company data then is this query / database accessible to others?
Hi, thanks for cool stuff. The shown example has tables with names that match the purpose, the same applies to the columns in the tables. In reality, tables can have names that are meaningful to the programmer, but that do not tell what data is in the table. The same applies to columns. Do you have a suggestion on how to install and in which place a translator that would instruct the AI what the purpose of the table is (what the data in it describes) and the same for each column in the corresponding table.
I am also looking for an answer. Great question! The solution can unlock many possibilities. I am a newbie here but I read there could be several langchain agents that can work together. In that case, we may feed all the logic in a vector db. Fire the LLM model, ask it to read the definition from a vector db and LLM will generate the sql query , and finally hit the sql db and return result to the LLM model. Not sure, but just an educated guess. waiting for an expert to reply. :)
how can I achieve this is MSSQL database
how can you connect to own sql server?
The video I amazing but I have a question about chat GDP can replace the developers.
I know chatgpt doesn't have direct access to the database, it just generates sql query based on the input given, but does it save this information. Can I integrate this with organizational data. Does it raise privacy concerns ?
is it possible to add memory to the agent?
How would you go about building nuance so that GPT knows which tables to pull to answer the question? Say if my database is a fake bike database with 100+ tables and I ask show me all customers who bought mountain bikes, how would you tell GPT that it should actually use type = Mountain Bike and to use the right table?
Can you please make a video on a multiple PDF Chatbot where the answers contain texts,images,tables from the pdfs and also tell the PDFs from where the answer is taken(citation)
Can we connect this to KUSTO databases ?
Great video thanks. Question: What if your SQL schema has table names that are not intuitive. My ERP uses table names such as ABXXXXX for the payables...etc.
I would think if they have unintuitive names. Then the person has to type those unintuitive names in his input query as well.
I wonder whether the model can use views since with a view you can rename the weird named columns
Hi MG! Nice video! Could you please help me with multiple tables? the problem is I am not able to read the data from multiple tables so please help me.
Wow that’s an intro ☝️
Say, where did you get these spiffy jumpers?
So I wanted to know how one can make AI response into json format . So let say I have an AI assistant which the user can ask it to help him to figure out some item for a party . I want the result to be like {
“Ai_message”: “something here”,
“Items”: [ #list of items here ]
}
And the Ai will get the results from SQLite database using Django framework or just python
This is interesting, but when I actually implemented this on an active MySQL database that has any complexity to it, I was very disappointed in the output. This is still in its infancy unless you have a very simple database and need very little information out of it. From a cost perspective, it was definitely not worth it.
Amazing!! I have a Doubt...
Let's say I have created an environment where a user can log in, enter their query into the chat box, and ChatGPT will fetch the results from the database and provide them to the user, as shown in your video. Let's assume that my database contains income data, and since ChatGPT has access to data from all over the world, I use the income data from my database to fetch and do some analysis or reporting based on the tax policies of a particular country.
The concern here is whether the data in the database can be leaked or put at risk in the open world While using ChatGPT.??
Same doubt!! @MG please explainnnnnnnnnnnnnn
Getting the following after cloning the repository locally "
1 validation error for SQLDatabaseToolkit
llm field required (type=value_error.missing)",
what i am missing here? This issue was raising by the interpreter at "----> 2 toolkit = SQLDatabaseToolkit(db=db)" line
It was resolved by enhancing the following statement;
"
dburi = SQLDatabase.from_uri("sqlite:///Chinook.db")
llm = OpenAI(temperature=0)
toolkit = SQLDatabaseToolkit(db=dburi, llm=llm)
agent_executor = create_sql_agent(
llm=llm,
toolkit=toolkit,
verbose=True
)
"
@@adilmajeed8439
KeyError: 'tools'
I'm getting this error after applying the above solution
@@adilmajeed8439 Thanks, solved by your reply and saved me some time.
How to connect the sql server to chatgpt,
This is safe for our database? have risk to drop o changes database? How connect my db of Databricks to OpenAI?
I believe that chatGPT doesn directly interact with your database. It just checks the meta information of your database, makes a query, and returns the query to you. From there you run the query on your device/server. In the example case, I believe the LangChain package is doing it for us. Correct me if I'm wrong though.
I tried using this in a large database schema and I always go over the limit of 4096 tokens for any request. I guess this works in a small schema like this and where tables and columns have very descriptive names, but this is not the case for an enterprise database.
A data dictionary / catalogue which explains the tables and columns can be used to speed it up and reduce errors, I think.
Use the embeddings api to narrow down the most relevant data. Search for youtube videos that explain embedding.
@@creneemugo94freecodecamp recently released a whole tutorial on vector embedding...fyi
Hi MG,
Is there any way to do this in dotnet (c#) with SQL Server. May be using Azure Cognitive Services with database.
I tried a lot but did not find anything.
Thanks
Yes please I would like to know too.
Hi MG I want to do natural language prompts against IoT data that's stored in timescaledb (postgres)
did you figure it out?
Hi there, Is that Possible using PHP and MySQL?
Microsoft does not support MySQL, and honestly, I don't know why. For example, Dataset only supports SQL Server. I have to load data using CSV or JSON files.
hahaha same question, leaving this reply here in case someone answers.
Can you please help me from out of this below error:
ValidationError: 1 validation error for SQLDatabaseToolkit
llm
field required (type=value_error.missing)
Change to this:
llm = OpenAI(temperature=0)
db = SQLDatabase.from_uri("sqlite:///./Chinook.db")
toolkit = SQLDatabaseToolkit(db=db, llm=llm)
agent_executor = create_sql_agent(
llm=llm,
toolkit=toolkit,
verbose=True
)
instead of
db = SQLDatabase.from_uri("sqlite:///./Chinook.db")
toolkit = SQLDatabaseToolkit(db=db)
agent_executor = create_sql_agent(
llm=OpenAI(temperature=0),
toolkit=toolkit,
verbose=True
)
@@vasubabujinagam5977 god bless u bro I was so annoyed and this worked
Is anyone familiar with Chris Harrison and the graph he achieved representing the dense cross references of the Bible? (I would include a link here but some people are weary of such clickables, but if you Google it it comes right up)... I am looking to achieve a similar output, from an input of 2 million words directed to a single project, anyone know how this could be achieved?
How I can create multiple chats like the ChatGPT where every Chat have own chat history? Please anyone who can help me.
3:32 , But you shouldn't miss the starting, trust me. 🤣🤣
Hello "MG" , great thanks !!!!......please make a video for connecting database (any database ) to huggingface model using langchain or anything .....THANK YOU 😊
without OPENAI_KEY
I can't see this working .. There's no way any company will ever open its data to Chatgpt
i worry about him
Why I'm getting this error after run this code:
db = SQLDatabase.from_uri("sqlite:///./Chinook.db")
toolkit = SQLDatabaseToolkit(db=db)
agent_executor = create_sql_agent(
llm=OpenAI(temperature=0),
toolkit=toolkit,
verbose=True
)
ValidationError: 1 validation error for SQLDatabaseToolkit
llm
field required (type=value_error.missing)
me too 😭
This sure looks pretty neat, but I'm running into an error. Ubuntu 22.04, Python 3.10.6, langchain 0.0.198
In cell 8 of the notebook:
agent_executor.run("Describe the playlisttrack table")
Error in on_chain_start callback: 'name'
---------------------------------------------------------------------------
AuthenticationError Traceback (most recent call last)
Cell In[8], line 1
----> 1 agent_executor.run("Describe the playlisttrack table")
...and then many, many more lines in the error message. Anyone know what's going on here?
Hello "MG" , great thanks !!!!......please make a video for connecting database (any database ) to huggingface model using langchain or anything .....THANK YOU 😊
without OPENAI_KEY