WordNet - Natural Language Processing With Python and NLTK p.10
HTML-код
- Опубликовано: 5 фев 2025
- Part of the NLTK Corpora is WordNet. I wouldn't totally classify WordNet as a Corpora, if anything it is really a giant Lexicon, but, either way, it is super useful. With WordNet we can do things like look up words and their meaning according to their parts of speech, we can find synonyms, antonyms, and even examples of the word in use.
Playlist link: • Natural Language Proce...
sample code: pythonprogrammi...
hkinsley.com
/ sentdex
sentdex.com
seaofbtc.com
I wrote my own wordnet about 3 to 4 years ago used an online synonym resource with python. It wasn't very successful as it had to pull from an online resource per every comparison, but I'm glad theres a great offline resource like this out there!
Ah man, I love this video. Just what I need to start my thesis. Subbed and Liked.
You earn a like every time I wantch your videos
This is the Best Lesson from the start, Continuing the next videos. Would be great if you can make video specific on NLG.
"A ship is more similar to a cactus than a cat. Makes total sense." 😆
very well explained video, thanks a lot for such content.
Dear Diary, today I learned that a ship is more similar to a cactus than to a cat.
what about "cat" as in " catamaran (dual hulled boat) ?
The way you explained help me understand quickly. Thanks.
keep up the good work man. you help a lot. thanks.
I'm a little bit confused here. After obtaining the synonym using wordnet.synset, why do we have to go into lemma and choose the first element of the lemma to obtain the word? It does not feel very intuitive. Can you please explain the structures obtained from using wordnet.synset? And also, why do we need to reference 0 when we are getting the antonym? We did not do that when we append the synonym list.
Same here, I don't quite understand what is lemma() and name() object doing here. And what kind of data type of lemma() is? Is it an Array or something?
l.antonyms() is a list with only 1 element.
you can try l.antonyms()[1] and it will throw out an IndexError: list index out of range.
I am not fully understood either. Just try to explain what I know. :-)
Thank you, bhai.
based on what I have searched, lemmas are kinda like synsets, so in my case im thinking like the lemmas are like synsets of the first element in our synsets. I don't know pelase correct me. That's just my understanding
Hi,
First, I would like to say thank you for this channel, really I love it.
I did compare between two words total and amount and that is what I got as result
Synset('sum.n.05') Synset('sum.n.01')
0.181818181818
I think it is not that accurate in some cases Coz Total and amount should have same meaning and the percentage should be high, where as I got this result
Synset('sum.n.01') Synset('sum.n.01')
1.0 when I compared the amount with sum
Shouldn't the comparison be between 'total.n.01' and 'amount.n.01' ? The above looks like it's comparing two different senses of sum with each other, which would have different meanings (like the meaning of 'bull' in regular English vs in the stock market, as was explained in one of the first videos of this series).
This video was pretty brief, not boring and informative. - Thanks
Further I had a question.
What if my scenario is that I have to search fir key words.
Say I have to look for key words like 'Sales Document' / 'Purchase Documents' in another document
Now they can also be written as 'Sales should be documented' or 'company selling should be written in the text files'. (For Sales Document keyword) Is there an approach here or will I have to build something?
Hello Sentdex,
I have hospital reviews on which I have to do sentiment analysis. SO do you think I should have some kind healthcare corpora to analyse or I can use one the corporas shown in your videos if not could you help me with the approach that I should take. Thanks!!
Can you do a video for SentiWordnet>??
thank you for this tutorial!
No sé como llegué aquí pero estuvo muy interesante, muchas gracias
Can I add words or synsets of my own to wordnet? Is there any way to edit or customize wordnet adjsting as per our requirements?
I did not get the concept of lemmas as used in the context here. what does it essentially mean? That portion has a rather non intuitive syntax which I had to copy carefully - not something thats easy to figure out yourself.
Further, is it possible to compare similarity between blocks of text: like one sentence / paragraph against another?
Hey.. I have gained a lot of knowledge from your python, machine learning and natural language text processing videos..Thank you..We are doing a project on building ontology for knowledge oriented search engine. In this process, we have taken a text document, applied POS tagging to it and have applied topic modeling lda method to obtain concepts. We need to extract relationships between these concepts. We are stuck! Don't know how to do! It would be great if you could help us on this..Thanks
I have a question. Why did we use [0] in "antonyms.append(l.antonyms()[0].name())" ? Does that mean we're only taking the first element of the antonyms list?
Great video sir.
Can I use wordnet and python to carry out Word Sense Induction, if so is it possible for you to post a video on it?
Is there a way to show how the database came to decide ship is closer to cactus than cat?
How can I search for a "word" which is made up of two words. e.g. "steering wheel"?
It is listed on its own in the online website for wordnet but leaving a space between the two words when querying it causes an error.
Thanks! :)
Thank you very much. My question is: how we use wordnet to disambiguate words
Hey buddy, like the comparing of words, can I compare sentences and get the similarity percentage ??
How can I use my data set containing 30,000 product names and cluster them into groups based on similarites based on their product names. Example I would group "pepsi" and "pepsi plus" into one group.
I need help.What should i type in python to import arabic wordnet?
This lecture was difficult to understand from the point of view of syntax of Python constructions, I had to watch it several times. Comments explaining what is our goal in each case would facilite understanding.
Agree, I kind of confused about the data type those object / method output.
For example, lemma()
If you didn't watch the whole series it could certainly be confusing. But if you had it's the same approach used in all 9 previous videos.
Really nice video man. Could you give me some insight of how choose best words for make my image ML consider? I am doing it with my base of followers instagram feed photos?
As I need make my training first I must consider with groups I must train before take a look on all this photos.
I know I could make this with comments but I would like to consider the real post first. I think it's more relevant. Could you help me with that?
sir,,,I am a beginner in nltk. can you tell me what is the difference between synsets and lemmas.
Is there a way that we might obtain a more accurate understanding of the meaning by looking at the word in the context of the text? If so, how would we go about implementing that?
Thanks a lot for your videos.. it is really helpful to me... Just I want to know that How Similarity, Synonyms & antonyms will be helpful in my classification model? I am unable to figure out uses of it...
Hi Harrison. Thanks for the video. I was wondering if there is a way I can change the structure of the sentence with nltk? So instead of saying: "Peter kissed Sarah", you could say "Sarah kissed Peter". I understand that people use nltk to change words to substitute synonyms in place of existing words based on the wup_similarity score but overall the text should remind fairly the same unless nltk can be used to change the structure of the sentences hence I fail to see how people can use it to automatically change an essay / news article and make it totally dissimilar to the one that has already been written.
What if you do pos tagging and swap the two proper nouns around?
Great, just a question could you please provide some examples about sentence similarity.
+Mohammed Abujayyab Try gensim
Could you show how to retrieve hierarchy (ancestors and children) of a word (I guess this is hypernym). Thank you for the video.
+sentdex Your videos are awesome! I'm especially amazed that you make such complex concepts and such difficult steps so easy to understand and try
I had a question,
In the wup_similarity function, what does the number stand for? For instance, in "ship.n.01" what does 01 stand for? Does it mean all the meanings? Or is it only pointing to a particular meaning? And if it points to a particular meaning, how do I determine which meaning do I use in the statement?
"Computers will read a few articles and switch words arounf and post an article and claim it as their own"
Humans already did this, it's called journalism.
Could you make a video utilizing the TIMIT database to train an LSTM neural network to do voice to text?
How do I find out the semantic orientation of an adjective i.e. whether the adjective is positive , negative or neutral using NLTK.
is it possible to add new words into wordnet? can u show a demo on that
+Revathy Ramanujam You'd probably need to make your own local copy and add there, never needed to do that, so never tried.
thanku you so muchhhh !...............excellent explanation
+aakash kag Awesome, happy to share!
+sentdex Hi ,
Thank you for valuable knowledge . I am using it from India :). I would need to know if I can use this for accessing hypernyms if yes how. Please assist.
+sentdex Hi ,
Thank you for valuable knowledge . I am using it from India :). I would need to know if I can use this for accessing hypernyms if yes how. Please assist.
how can I get all the verbs from brown corpus ?
You are awesome!! great stuff there man!! Thanks a lot!!! :)
not clear with what exactly lemmas are?
Love your tutorials!
Can you please make a news bot kind of a thing sometime? It would be really cool!
Thanks for all these awesome videos!
How exactly is word similarity calculated? Incidentaly, I compared ship/kitten (to see the difference between kitten/cat) and I got the exactly same number (0.38095238095238093) as you did for cactus
wordnet not working on my python 3.7.2 idle same for lemmatization
it was great thank you
If you try, you can do a good impression of chris griffin. btw, loving your tutorials. many thanks.
Quem acompanha do Brasil (ou precisa desenvolver em Português) existe a wordnetPT (wordnet.pt/),. É necessário baixar o arquivo sql, executar para obter um banco de dados e criar as funções para extrair as informações. Obviamente, ela é de grande ajuda.
Love your videos but watching you use idle instead of jupyter notebook gives me the same feeling as when my best friend doesn't like my favorite movie. Like how could you not love what I love?
For real though, don't you find it easier to insert new cells and run portions of code at a time compared to commenting out stuff and running the whole script?
+Lance Dacey I've tried to put my finger on what's up. Maybe I am broken, who knows. I think it is in part an objective and end-result in mind. Many people use these modules to "tinker" with the data to do research/analysis for themselves or at least fundamentally add value personally to their analysis. I use these modules with programs and websites that just simply run the code, unlike what IPython will do and very much like a general interpreter would do.
Still though, one could argue that IPython would make the initial development time faster, and one might be right. I've wanted to like and use IPython for years now. I keep trying, and maybe one day.
@@sentdex fast forward three years: I started using Jupyter Lab because of your Pandas tutorial series and am hooked
To clarify, he is using the word "good" as "good" not "good"
AWESOME!!!
can someone explain the difference between lemmas and synset?
I mean wrt to the 'syns[0].lemma()[0].name()'
Would you say the word-net is a real dictionary just like human learns English?
Hi, there is wordnet for Spanish? Thanks you! I love this channel!
I am not sure, could try googling that if you haven't.
How about Hypernyms/Hyponyms?
please add more codes in wsd
I am being confused between line 1 and line 3 as follows. Because synonyums and antonums are completely exclusive:
------
1. for l in syn.lemmas():
2. synonyms.append(l.name())
3. if l.antonyms():
------
Is there anyone who can give me an explain?
Many thanks.
Lemmas in nltk have different methods .
w = wordnet.synset("good.n.01").lemmas()[0]
print(w.__doc__)
Lemma methods:
Lemmas have the following methods for retrieving related Lemmas. They
correspond to the names for the pointer symbols defined here:
wordnet.princeton.edu/man/wninput.5WN.html#sect3
These methods all return lists of Lemmas:
- antonyms
- hypernyms, instance_hypernyms
- hyponyms, instance_hyponyms
- member_holonyms, substance_holonyms, part_holonyms
- member_meronyms, substance_meronyms, part_meronyms
- topic_domains, region_domains, usage_domains
- attributes
- derivationally_related_forms
- entailments
- causes
- also_sees
- verb_groups
- similar_tos
- pertainyms
yup , me too also
very informative video. Can you or any one let me know how to access Hindi WordNet in python.
can you make a tutorial about how to install NLTK wordnet in python ! ! ?
+Reda Ayala install nltk, then do nltk.download, and it's one of the options. If you just install everything, you will have wordnet installed.
thank you , i checked only wordnet and it's working , thnx for help ))
what is difference between lemmas and synset?
as far as my understandind synset contains list of possible meaning of a given word
and each element in a synset has different lemmas each lemma being a list of words with same meaning
I've tried to compare "good" with "bad" i have got surprised result 0.6666666666666666 which is not good any clarification !!
I need connect with you if you don't mind please
I forgot the drink some coffee step and didnt manage to achieve shit after that point
wup_similarity is throwing error
3 cats make a ship!
why "nice" and "good" has only 11% similarity??
I created a NLTK gist which replaces all words in the string with their synonyms.
gist.github.com/Ghost---Shadow/c361f2d6b4501f40648b
I tried a lot of things to make it reliable but it still mostly prints nonsense. What should I do?
lol ship and cat are 32% similar, i can't think of a single way they're similar
They can often have similar colors. :D
@@sentdex true dat, i guess the similarity algorithm is pretty genius!
Why do you give so much of add , i don't see anyother youtuber to give so much add , this tutorial is not worth much