I can understand word embedding as the process of passing the digital form of the word through the embedding network to get the word embedding. But is sentence embedding a combination of word embedding?
Absolutely! They can be used for classification, generation, clustering, basically any ML model that you'd like to train. Embeddings can be used as a preprocessing method, where instead of text, now you have vectors, and you train models on those vectors.
@@SerranoAcademy Thanks for this wonderful video! Just to make sure, though. If one creates embeddings using cohere embeddings say, then they can be used in any model say BERT or GPT-J, somehow? Or are we talking about NLP tasks?
Does this mean, we can start the process in say, Swahili and then convert to English, so as not to lose the particular characteristics of Swahili which happens when one starts with English?
Great question! There are many ways to calculate them, some with neural networks. Check out Word2Vec to learn them (a future video will come out on that, stay tuned!) The idea is that when two words appear in similar sentences a lot, the model will slowly join them, and when they don't, the model will slowly separate them. In that way, similar words end up close to each other. A neural network, for example, can be trained to predict the neighbours of a word, and the last layer of the neural network can be used for the embedding.
Hey thanks for the video, im looking to do this exercise with around 13k to 15k news articles. Is there an efficient way of doing it with cohere?
I can understand word embedding as the process of passing the digital form of the word through the embedding network to get the word embedding. But is sentence embedding a combination of word embedding?
If one is given embeddings, can one go ahead and use those embeddings for any NLP model?
Absolutely! They can be used for classification, generation, clustering, basically any ML model that you'd like to train. Embeddings can be used as a preprocessing method, where instead of text, now you have vectors, and you train models on those vectors.
@@SerranoAcademy Thanks for this wonderful video! Just to make sure, though. If one creates embeddings using cohere embeddings say, then they can be used in any model say BERT or GPT-J, somehow? Or are we talking about NLP tasks?
Does this mean, we can start the process in say, Swahili and then convert to English, so as not to lose the particular characteristics of Swahili which happens when one starts with English?
curious, how are the embeddings actually calculated?
Great question! There are many ways to calculate them, some with neural networks. Check out Word2Vec to learn them (a future video will come out on that, stay tuned!)
The idea is that when two words appear in similar sentences a lot, the model will slowly join them, and when they don't, the model will slowly separate them. In that way, similar words end up close to each other. A neural network, for example, can be trained to predict the neighbours of a word, and the last layer of the neural network can be used for the embedding.
i assumed if world cup is in qatar, it makes sense to add arabic translation unless there are issues in arabic support
good