Markov chain - next word Prediction - Python Code
HTML-код
- Опубликовано: 21 сен 2018
- Using machine learning auto suggest user what should be next word, just like in swift keyboards. Code is explained and uploaded on Github. English version of video also uploaded. Markov Chains is being used to achieve the functionality.
Github Code location:
github.com/sandeepbhutani304/...
Dear brother,
please make a video to create step by step
1. creating corpus
2. input text
3. tokenizing
4. train
4. then show prediction result
Hello my dear friend, this algorithm doesn't work in my language. In Azerbaijani language, the predictions is like ? or ! or sme kind of other types of letters. Please help out
@Practical Matter after typing the command( python markov-nextwordpred.py) on anaconda prompt this message appear (Usage: start typing.. program will suggest words. Press tab to chose the first suggestion or keep typing) but when i type for example (I)and i press space on the cmd appear (NONE) there is no suggestions :( it was earlier working now after some days its not showing above problem
Hi.. Just check the txt file
@@SandeepBhutani thanks its working
ML is subject to alteration by third party interference and this is finally validation of what's going on with g-board APK nonetheless, Python markov chain exampling here is very much appreciated as well as it's pretty much not known to actually be so prone to such being other than what's from what a person is thinking of as the whole machinelearning part of smartphone keyboards using that to adjust to someones'[personal] preferences yet the way that it has no security protocols in place for anyone using the g-board application at all.. is there an Markov chain that is for something that can ENSURE the words generated are something like the words in suggestion candidates to be secure and then [read-only] in fact?
hello i m running it on pycharm it shows that start typing and press tab
but when i enter text it doesnt predict next word
Please follow step by step. Make sure that you have text file in place with some text
hello sir ,
i am getting error in python file
print(str(c.decode('utf-8')), end='', flush=True)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte\
plz help....
While opening file use rb as second parameter
You can also mention errors='ignore' while opening the file
@@SandeepBhutani Hello, where to write errors='ignore'
While opening the file
@@SandeepBhutani I am using mac and it says no module found msvcrt. I have searched everywhere and I can't correct it.
hello thanks for this i'm not used to phython but ive installed anaconda the code and the variables are okey in spyder but at anaconda prombt there is no suggestions pleaz could you tell me why ?
Where exactly you are stuck?
@@SandeepBhutani after typing the command( python markov-nextwordpred.py) on anaconda prompt this message appear (Usage: start typing.. program will suggest words. Press tab to chose the first suggestion or keep typing) but when i type for example (I)and i press space on the cmd appear (NONE) there is no suggestions :(
@@aynmedoria3492 It mean your program is running correctly. NONE means there is nothing in database (test.txt file shown in video is acting as database). Add something to that file, and program will start suggesting next words based on contents of this file
thanks for replying :) actually the database is full it's not empty :( but there's no suggestions and in spyder the variables are okey i can see what i have in the database ..... does the anaconda require any configuration before starting to use it in this case ? if it's no than please could u see what's the problem : (
@@aynmedoria3492 To make things simple, simply create a file test_text.txt (if not already exists) where the markov_nextwordpred.py file is stored. Write some sentences on your own in test_text.txt file and run the program. It should work.
The program isn't working with another data set . I did try renaming it as you mentioned.
any help would be appreciated.
If you can tell exact issue then I can help
@@SandeepBhutani i added a wiki dump text to your text_text file and it stopped showing predictions
@@SandeepBhutani I too faced the exact issue when I started executing the code no errors with my text database file. No suggestions or predictions were displayed. Can you help me out?
I will check it.. Though, FYI I am using same in live chatbot
@@SandeepBhutani could you please tell me how you expanded the database
Or could you provide a link for the updated database ?
It would be a big help
can this be used in suggesting Indian languages without the use of intlk?
like I want the text data to be in the Indian language and can it predict the next word...?
Should be working.. Any issue facing using nltk?
@@SandeepBhutani yes.. Inltk has less number of functionalities to autocorrect the words.. Do we have some other way to overcome this issues??
Nltk does not autocorrect.. Are you facing issues in tokenization?
hello i am getting an error at code.. print(str(c.decode('utf-8')), end = ' ', flush=True) help me out
What is the error?
@@SandeepBhutani invalid syntax
With invalid syntax it also shows an arrow ^ which shows location of error. U can try downloading and executing the code directly from github.. Or paste complete error here as it is
@@SandeepBhutani runfile('C:/Users/Admin/Desktop/predictive.py', wdir='C:/Users/Admin/Desktop')
File "C:/Users/Admin/Desktop/predictive.py", line 85
print(str(c.decode('utf-8')), end = ' ', flush=True)
^
SyntaxError: invalid syntax
error i am getting
You did it good. But sorry to say, your explanation is very bad. Please try to explain you code fluently with basics i.e what we had use and why we are using it ....and so on.
Hi.. Check the other video for theory