Sample chat inputs and responses, along with my scoring that you can choose to work on top of, or not, if you want to take part: github.com/Sentdex/chatbotrnd/blob/master/chat_question_examples.py
Hey Sentdex, these videos are actually exactly what I'm looking for. The problem with teaching stuff (especially in the programming world) is that things are always way too sterile. Like, OK OK OK, we've had enough clean dataset machine learning. There is a lack of R&D videos in ML. More videos like this help in two main ways: 1) we get to see how great programmers problem solve 2) it makes it less demoralizing seeing someone else also struggle and allows us to cultivate a mindset of pushing through hurdles. Keep it up!!
Some of the videos I make (different account) are like this and that get absolutely no traction. Sterile videos get views even though I want to make detailed videos, mistakes and all.
I love watching these kinds of videos, it gives so much insight into these kinds of projects. Also for some of my more casual projects, these are insanely motivating, and they help really help me wanting to code more just for fun. Great work! Keep it up!
A legendary Jedi Master in sci-fi movie series Star Wars, Obi-Wan Kenobi was a noble man and gifted in the ways of the Force. He trained Anakin Skywalker, served as a general in the Republic Army during the Clone Wars, and guided Luke Skywalker as a mentor.
I came in confused. When the video was done I couldn't even leave, because I'm now completely lost. I'm basically just here to listen to Sentdex have fun. Living vicariously through you, Sentdex.
Someone has probably commented this before but it's a Star Wars meme. Obi Wan Kenobi drops from rafters in front of General Greivous. Surrounded by Greivous's troops says: "Hello there!" General Greivous retorts with "General Kenobi" and ensues to draw his light sabres to fight. Also, could you split strings by punctuation and then check for each string in the list if the string occurs more than once in the list? 42:41 - "Sorting dics" Intentionally misquoting Sentdex 53:00 - Possible reason why the answer isn't changing is because you're not retraining the models? I was doing some Data Science stuff and one of the projects the tutor was speaking about doing was creating a Spam Filter. He was using Pandas DataFrames so that might be a more efficient method of sorting answers.
sentdex you are awesome. I discovered your CNN serie with the cat vs dog classification. Now I'm obsessed with Computer Vision and I can finally say that my math bachelor is useful, FINALLY !
hey sentdex, Here's an Idea that i thought and sorry if it's wrong what if there's 2 chatbot models chating with each other and every time they reply to ones sentence/question there's an Understanding score that measures how good that reply was means how good the other chatbot was able to understand that reply. It kinda like GAN's we two neural network competes with each other to get better. Coz we humans are like that the more understandable ones question/sentence is the better we are able to reply to them. Again sorry if it sounds totally foolish
Great video, love the chatbot.One thing i noticed is that when the answer attended to be short one for some of your questions, the long answers will not be good ones. So maybe penalize it against its length in such scenario, especially when the question is not too common (didn't occur in the dataset) or if the model is overfitting on some of the common words like 'living' for example. i guess there are tons of things one could try and test on this amazing chatbot!!!! we are waiting and hungry for more videos
This content is great!! Thanks so much for all your videos! Just clicked the notifications button :D Oh and instead of having the if statement for checking if I_count is less than zero, you can just do max(0, i_count-1) woo
Sentdex, you provide some of the best material for python (I watch a lot of python videos online, so trust me --- other viewers) Keep up the good work :)
I tried to make the same chatbot also using Seq2Seq using the Cornell corpus dataset but google colab ran out of RAM :( Edit: Also my model had a 1 layer encoder and a 1 layer decoder each with 500 neurons LOL
1:03:15 "como estas" which means "how are you", and the answer "Gracias por favor" means "thank you please", which is not ok, and it does not have any meaning in spanish. Anyway, great video
I'm noob coder but I had question can't it keep learning with new question and answers or we have to train it all from scratch everytime we want to add even more answers ?
Have you thought about using something like mechanical Turk to help score responses in some way? Or a tutorial on some other data labelling solutions out there, could be an interesting series too.
I am building chat bot too. what i find most curious is encoding/decoding representations of meaning into words or sentences. I'm assuming that the encoding and decoding for the model you are using were trained from scratch with the rest of the model. I wonder what happens if the encoding/decoding is trained separately, perhaps on a dictionary. I guess this falls under the discussion of whether its best to create models that do end-to-end learning tasks, or connect separate modules trained for sub-tasks.
32:18 I guess you're not yet a "power use" of Sublime Text, I would have selected "ans" do Ctrl+D twice to select the other two occurrences and type answer 34:39 To precise a space for the first split argument is facultative since it's its default value. 38:10 instead of list comprehension you can fix it with a set comprehension. Well not, you can use set(words) on line 19 like for word in set(words) Btw I was surprised by results because in your test it doesn't print "up" twice before it shows the list and looking at your code I don't get why it's filtered, but once fixed it should give the result 3 for total repetitions. 44:27 In the sorted function call you add the argument reverse=True 49:37 I faced this issue and found a solution, you can do from collections import most_common and do print(most_common(my_dict, 5)) to get the top 5 values, if you want the index of the best value (when not sorted) you can convert your dict to list of tuples with list(my_dict) use numpy.argmax() on it and use numpy slicing to filter the second column (of values)
If the scoring is so important and complex, why not turn it into a machine learning task? You could great a dataset of acceptable sentences and unacceptable sentences and get a model to predict the probability that each sentence is either acceptable or unacceptable, or something
awesome fanatstic superlastic video dude you are my inspiration to make tutorials in programming and new innovative projects ...may god bless you.........................................................................
Amazing video *as always*, What do you think about Evolved Transformer Networks ? Google developed Meena on it apparently and seems very promising. Good luck and happy coding !
love from india :3 great work and effort, i wanna be like you - *could potentially make something that can be world changing* "ahaha nice responses cool, hey guys its me!"
When you want to make a new file in linux you don't have to open it with nano. You can just do `touch filename.something` or even `sublime filename.something` though I'm not sure about the second
Just wondering if it would be possible to give the chat-bot a sense of 'self'? i.e. based on the answers it has previously delivered. It would make the bot much more convincing if its answers were consistent.
Hi Sentdex, Here are some recent paper that you may find helpful regarding non-greedy decoding. - (nucleus / top-p sampling) arxiv.org/pdf/1904.09751.pdf - (self-terminating training) arxiv.org/pdf/2002.02492.pdf tl;dr: Beam search (along with similar maximization schemes) does NOT seem to work very well during decoding, provided that you've trained using maximum likelihood (MLE). This is even true for extremely expressive generative models such as GPT-2, whose training was based on MLE and did not incorporate penalty against repetition. The second paper recommends a different learning objective that incorporates such penalties and seem to work well for RNN-type models. Instead, consider using either top-k sampling or nucleus sampling during decoding, as they are agnostic to model architecture (i.e., can be adapted to RNNs or transformers, and can be used regardless of the learning objective(s)). It is also easier to implement since you do not need to re-train on 400mil pairs using fancy objectives. They have shown great promise in recent packages like Microsoft's DialoGPT (dialog version of GPT-2), which was also trained on Reddit convos. Thanks for your videos -- you make some great content.
I think if it is a chat bot functioning like siri, you need to add a random function after getting the results, other than that, you will need to supervise some stuff. Just a humble opinion😁
@sentdex am I the only one that can’t launch the video from an iPhone ? I am locked in the « something went wrong tap to retry » loop ! It’s been like this for the last 2 days !
I just waited for this. I had stucked on my project of chatbot using tf . I took a course on Udemy to build chatbot , the author did it in TF1 whereas I would like to do in TF2 with all attention mechanism . But I continued with the course on building chatbot, at one point of time , I had error like Module not found , since he used TF1.4 I couldn't install that version, since no such version for now, atleast to mimic wht he's doing. So wht to do? How can I build chatbot using Seq2Seq model using TF2 with attention mechanisms. (Note: Not a NMT model to translate, but a chatbot model to give response to the user queries)
Do you like Sentdex R&D video style? > Yes, please do more! > Hello General Sentdex > Yes, this is exactly what I'm looking for > Да, продолжай их делать! > Yes, sorry for my English level
Okay idk if this is the genuine question to ask here, but anyways um going to ask: I have ubuntu 18.04 installed on my PC but can't install my gpu driver nvidia GEFORCE(MX-150) when i run command like nvidia-smi , it shows "NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running" help me if u can please @sentdex
I think there are a ton of STEM related responses because those kinds of peoole never shut up about it on Reddit! They don't necessarily have a disproportionate representation in Reddit users. Just a disproportionate representation in responses.
Hey thanks a lot for the video... I have a doubt in case of huge dataset how will you train the model bcoz I'm using 1050TI always my gpu crashes when either dataset is big or nn is big so can you tell what will you do at that time. If you're using Colab can you do a tutorial on that and how to access files from it
i wanna implement a chatbot which is self learning and generative chatbot . for the HR Policies , i have pdf files of HR Policies . How can i implement it, please give link and ideas. i don't have data in question answers form. if i'll upload any document then it will automatically train based on data and give reply.
![KSI - Thick Of It (feat. Trippie Redd) [Official Music Video]](http://i.ytimg.com/vi/At8v_Yc044Y/mqdefault.jpg)
Sample chat inputs and responses, along with my scoring that you can choose to work on top of, or not, if you want to take part: github.com/Sentdex/chatbotrnd/blob/master/chat_question_examples.py
Long form is interesting. There are videos out there that are 4 to 8 hours long of straight coding. I consume them! Improves my coding skills
sentdex is there a problem the video? Its always crashing at 1:37. Couldn’t watch it 😩.
Someone can teach own chat AI using conversation of people in social networks.
@Matteo Trace definitely, been using flixzone for since march myself :D
Hey Sentdex, these videos are actually exactly what I'm looking for. The problem with teaching stuff (especially in the programming world) is that things are always way too sterile. Like, OK OK OK, we've had enough clean dataset machine learning. There is a lack of R&D videos in ML. More videos like this help in two main ways: 1) we get to see how great programmers problem solve 2) it makes it less demoralizing seeing someone else also struggle and allows us to cultivate a mindset of pushing through hurdles.
Keep it up!!
Jacques Thibodeau this comment is underrated
Some of the videos I make (different account) are like this and that get absolutely no traction. Sterile videos get views even though I want to make detailed videos, mistakes and all.
@@mechanicalmonk2020 Make those videos, get your channel going, and afterwards you can upload research videos
I was wondering what Snowden is up to in quarantine
He has always been in quarantine
@@randallwalkerdiaz1002 yes because he leaked too much
He does look like Snowden lol
I definitely enjoy this format because it allows me to learn how you think through problems!
Please do more videos like that. You are completely right about me sitting at home and begging for content.
Sentdex: "Are you a machine?"
Machine: "No, I'm dad!"
🤣🤣🤣🤣🤣🤣🤣
Reddit's always gonna Reddit.
10:38 “I’m a software engineer. I’m a software”
It has become self-aware, you accomplished more than you ever could have imagined.
@Arbaazbaby way to kill the joke
I love watching these kinds of videos, it gives so much insight into these kinds of projects.
Also for some of my more casual projects, these are insanely motivating, and they help really help me wanting to code more just for fun.
Great work! Keep it up!
A legendary Jedi Master in sci-fi movie series Star Wars, Obi-Wan Kenobi was a noble man and gifted in the ways of the Force. He trained Anakin Skywalker, served as a general in the Republic Army during the Clone Wars, and guided Luke Skywalker as a mentor.
I just see notification from sentdex and I watch it on the spot 😂😂Great content man!!
Thanks and thanks.
Haven't had time to watch any recent videos. But it's great they are here during this time.
7:22 “I’ve been constantly running train” Chad sound byte
You know you have found the right chatbot when you type "hello" and it responds: general kenobi
I came in confused. When the video was done I couldn't even leave, because I'm now completely lost.
I'm basically just here to listen to Sentdex have fun. Living vicariously through you, Sentdex.
Someone has probably commented this before but it's a Star Wars meme.
Obi Wan Kenobi drops from rafters in front of General Greivous. Surrounded by Greivous's troops says: "Hello there!"
General Greivous retorts with "General Kenobi" and ensues to draw his light sabres to fight.
Also, could you split strings by punctuation and then check for each string in the list if the string occurs more than once in the list?
42:41 - "Sorting dics"
Intentionally misquoting Sentdex
53:00 - Possible reason why the answer isn't changing is because you're not retraining the models?
I was doing some Data Science stuff and one of the projects the tutor was speaking about doing was creating a Spam Filter. He was using Pandas DataFrames so that might be a more efficient method of sorting answers.
sentdex you are awesome. I discovered your CNN serie with the cat vs dog classification. Now I'm obsessed with Computer Vision and I can finally say that my math bachelor is useful, FINALLY !
My concentration always goes towards that EEG helmet behind you!!! That's what my master's thesis was about but, here I'm hanging on NLP.
hey sentdex, Here's an Idea that i thought and sorry if it's wrong what if there's 2 chatbot models chating with each other and every time they reply to ones sentence/question there's an Understanding score that measures how good that reply was means how good the other chatbot was able to understand that reply. It kinda like GAN's we two neural network competes with each other to get better. Coz we humans are like that the more understandable ones question/sentence is the better we are able to reply to them. Again sorry if it sounds totally foolish
Great video, love the chatbot.One thing i noticed is that when the answer attended to be short one for some of your questions, the long answers will not be good ones. So maybe penalize it against its length in such scenario, especially when the question is not too common (didn't occur in the dataset) or if the model is overfitting on some of the common words like 'living' for example. i guess there are tons of things one could try and test on this amazing chatbot!!!! we are waiting and hungry for more videos
Great job! I loved it
Great video :) ,
if you love working with bots, I recommend the rasa library is great and has a pretty powerful documentation.
This content is great!! Thanks so much for all your videos! Just clicked the notifications button :D Oh and instead of having the if statement for checking if I_count is less than zero, you can just do max(0, i_count-1) woo
"I'm a software engineer. I'm a software" it knows too much!
You are making the world better place. Thank you !!!!!!
Sentdex, you provide some of the best material for python (I watch a lot of python videos online, so trust me --- other viewers)
Keep up the good work :)
Thanks!
THIS MAN HAS EVERY TUTORIAL I WANT
Did I like this video?
> No, I loved this video. More plz.
I tried to make the same chatbot also using Seq2Seq using the Cornell corpus dataset but google colab ran out of RAM :(
Edit: Also my model had a 1 layer encoder and a 1 layer decoder each with 500 neurons LOL
Great video, very interesting.
"I'm a meth addict". WTF the source is Reddit, right? 😅
I would love to see a vid about the preparation of the input and the structure of the network along with labeling.
Is it just me or the sound is really low?
Might be lower than normal, but I can turn it up to be obnoxiously loud still.
yes it is
Niceee. Quick question, when will neural network from scratch videos be made? Great work anyway!
Keep doing with videos like this.
It's awesome and let us focus on something.
Greetings from Italy!
Be safe.
Thank you, you too!
Nice Video! I like that format
Here for the Algorithm now, I'll be back for the video later!
sentdex you are a genius man
really....
Your latest content is amazing🙌
@Sentdex you are a great man, I have learnt alot from You :) lot of love from Pakistan .Stay save.
You too!
1:03:15 "como estas" which means "how are you", and the answer "Gracias por favor" means "thank you please", which is not ok, and it does not have any meaning in spanish. Anyway, great video
I can vouch for how good spacy is. You can pick and choose what features you want to use. Highly recommend
I'm noob coder but I had question can't it keep learning with new question and answers or we have to train it all from scratch everytime we want to add even more answers ?
Have you thought about using something like mechanical Turk to help score responses in some way? Or a tutorial on some other data labelling solutions out there, could be an interesting series too.
I am building chat bot too. what i find most curious is encoding/decoding representations of meaning into words or sentences. I'm assuming that the encoding and decoding for the model you are using were trained from scratch with the rest of the model. I wonder what happens if the encoding/decoding is trained separately, perhaps on a dictionary. I guess this falls under the discussion of whether its best to create models that do end-to-end learning tasks, or connect separate modules trained for sub-tasks.
"Yes, I am satoshi nakamoto"
"Yes, bitcoin cash is bitcoin"
The second one scares me
Wtf is the source of the data
Sentdex: What's your favourite color?
Bot: Black, Black, Black, Black, Black, Black, Black, Black
32:18 I guess you're not yet a "power use" of Sublime Text, I would have selected "ans" do Ctrl+D twice to select the other two occurrences and type answer
34:39 To precise a space for the first split argument is facultative since it's its default value.
38:10 instead of list comprehension you can fix it with a set comprehension. Well not, you can use set(words) on line 19 like for word in set(words)
Btw I was surprised by results because in your test it doesn't print "up" twice before it shows the list and looking at your code I don't get why it's filtered, but once fixed it should give the result 3 for total repetitions.
44:27 In the sorted function call you add the argument reverse=True
49:37 I faced this issue and found a solution, you can do from collections import most_common and do print(most_common(my_dict, 5)) to get the top 5 values, if you want the index of the best value (when not sorted) you can convert your dict to list of tuples with list(my_dict) use numpy.argmax() on it and use numpy slicing to filter the second column (of values)
You're awesome dude...!!
Thank you
Love the style!
Hi, but can you again make video on how to make Charcot from scratch with tensorflow 2
If the scoring is so important and complex, why not turn it into a machine learning task? You could great a dataset of acceptable sentences and unacceptable sentences and get a model to predict the probability that each sentence is either acceptable or unacceptable, or something
Hi which OS are you using and which screen recording app you are using. Thanks for great videos by the way :)
Ubuntu for OS, OBS to record.
@@sentdex thanks 👍🏻
awesome fanatstic superlastic video dude you are my inspiration to make tutorials in programming and new innovative projects ...may god bless you.........................................................................
Amazing video *as always*, What do you think about Evolved Transformer Networks ? Google developed Meena on it apparently and seems very promising. Good luck and happy coding !
I have read the paper its a crazy model
Haven't looked into them yet, planning to probably after NNFS. We'll see :)
love from india :3
great work and effort, i wanna be like you -
*could potentially make something that can be world changing*
"ahaha nice responses cool, hey guys its me!"
Question: wouldnt it better for a score to use only multiplications and divisions because of the impact of the single parameters?
Very cool bro
have you tried ChatterBot module but with much larger training data like those you have?
omg ...!!! YAAASSS been waiting for this ... but holy shit u crazy with those 2 RTX 8000 .... meanwhile me out here using a GTX 1050
love you sir......thanks a lot!!!
pep8 is in tears
Once you finish experimentation, would you publish your ideas and experiment results
Most likely, yeah
So the libcublas does loads successfully. After so many hours on non-gpu system I thought it's just written to throw warning 🤷🏻♂️
Thanks for this upload :)
When you want to make a new file in linux you don't have to open it with nano.
You can just do `touch filename.something` or even `sublime filename.something` though I'm not sure about the second
Oh right, I like to forget about touch, thanks!
Can't believe Sentdex didn't know General Kenobi :")
Never really got deep into star wars, my apologies :D
1:05:45 the moment you fail a turing test.
Just a quick question. Does your new book use NumPy, or everything from scratch using Python only?
Both. We first show how to do it in raw python, then make use of Numpy since raw python can get quite silly overall.
I thought sentdex was so good he didn't need Stack....but we're all human
Just a regular person like you :D
@@sentdex 👍👍I love your videos man
Just wondering if it would be possible to give the chat-bot a sense of 'self'? i.e. based on the answers it has previously delivered. It would make the bot much more convincing if its answers were consistent.
It is difficult to create chatbot model for Multi turn conversations, I would love to see how chatbots handle multi turn conversations in future
Hi Sentdex,
Here are some recent paper that you may find helpful regarding non-greedy decoding.
- (nucleus / top-p sampling) arxiv.org/pdf/1904.09751.pdf
- (self-terminating training) arxiv.org/pdf/2002.02492.pdf
tl;dr: Beam search (along with similar maximization schemes) does NOT seem to work very well during decoding, provided that you've trained using maximum likelihood (MLE).
This is even true for extremely expressive generative models such as GPT-2, whose training was based on MLE and did not incorporate penalty against repetition. The second paper recommends a different learning objective that incorporates such penalties and seem to work well for RNN-type models.
Instead, consider using either top-k sampling or nucleus sampling during decoding, as they are agnostic to model architecture (i.e., can be adapted to RNNs or transformers, and can be used regardless of the learning objective(s)). It is also easier to implement since you do not need to re-train on 400mil pairs using fancy objectives. They have shown great promise in recent packages like Microsoft's DialoGPT (dialog version of GPT-2), which was also trained on Reddit convos.
Thanks for your videos -- you make some great content.
I think if it is a chat bot functioning like siri, you need to add a random function after getting the results, other than that, you will need to supervise some stuff. Just a humble opinion😁
Can you explain the media pipe hand-tracking because i need it in my graduation project
17:50 general kenobi
Are you still using the 2015 comment corpus, or have you updated the dataset?
Updated, using like 400 million pairs now. He tends to think it's 2019 or 2018 ATM.
We are here. From notifications!
Welcome!
How are you coping with the lockdown harrison? Things must seem very ordinary.
For translation try deepL... It is amazing!
@sentdex am I the only one that can’t launch the video from an iPhone ? I am locked in the « something went wrong tap to retry » loop ! It’s been like this for the last 2 days !
Hello can I define a temporary dict and check if the word is already there for repetition?
I just waited for this. I had stucked on my project of chatbot using tf . I took a course on Udemy to build chatbot , the author did it in TF1 whereas I would like to do in TF2 with all attention mechanism . But I continued with the course on building chatbot, at one point of time , I had error like Module not found , since he used TF1.4 I couldn't install that version, since no such version for now, atleast to mimic wht he's doing. So wht to do? How can I build chatbot using Seq2Seq model using TF2 with attention mechanisms. (Note: Not a NMT model to translate, but a chatbot model to give response to the user queries)
I've trained a chatbot on 3M pairs, but my responses are nowhere as good as the ones yours outputs. How many pairs did you train on?
How can I train my chatbot with multilanguage???
Every video of you I saw includes spamming the console. I lowkey enjoy its suffering
Why "Brave" browser against chrome/FF ???
Dex. Please show us how to bring machine learning inside an application. Any application
Do you like Sentdex R&D video style?
> Yes, please do more!
> Hello General Sentdex
> Yes, this is exactly what I'm looking for
> Да, продолжай их делать!
> Yes, sorry for my English level
Is there any chance of open sourcing Daniel's ensemble code? :)
There's a chance. No ETA on that though :D
28:10 do you even touch? ;P
touch test.py
Do you work on a standing desk?
Yes
it says 2 minutes ago uploaded but some comments says 1 day ago .
They are channel members. Video is probably uploaded yesteday but it was private.
I like sentdex 💓☺
Why you prefer Brave?
Good browser where my data isn't the business model.
@@sentdex Yes I agree, I use it too!
Okay idk if this is the genuine question to ask here, but anyways um going to ask:
I have ubuntu 18.04 installed on my PC but can't install my gpu driver nvidia GEFORCE(MX-150) when i run command like nvidia-smi , it shows "NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running" help me if u can please @sentdex
I think there are a ton of STEM related responses because those kinds of peoole never shut up about it on Reddit! They don't necessarily have a disproportionate representation in Reddit users. Just a disproportionate representation in responses.
Hey thanks a lot for the video... I have a doubt in case of huge dataset how will you train the model bcoz I'm using 1050TI always my gpu crashes when either dataset is big or nn is big so can you tell what will you do at that time. If you're using Colab can you do a tutorial on that and how to access files from it
It depends on the model, but you can try reducing the batch size
What does R&D stand for in this case?
Research and development
i wanna implement a chatbot which is self learning and generative chatbot .
for the HR Policies , i have pdf files of HR Policies .
How can i implement it, please give link and ideas.
i don't have data in question answers form.
if i'll upload any document then it will automatically train based on data and give reply.
please give reply on my mail
Are you a genius
Doubtful.