from the last 3-4 hrs i am trying to find a step-by-step proper material on how to fine to BERT with your dataset , finally found it , thank for making this video.
Really really amazing Pritish. This video is not like those boring lecture videos. The animations are amazing. your explanation is clear with goof pronunciation. Amazing. Keep it up. I hope you continue posting these type of videos. ❤❤❤❤
i have some text files in which there are elements and their values but the pattern in which the text is displayed in the file are different from file to file. Is it possible to train Bert on these files so that when I ask it to extract only the element names and their corresponding values it will do that regardless of the text pattern?
Can you please explain how I can retrain the same model (after exporting) with new data. Basically, I want to train the same model in stages. Please help.
so we don't need to freeze any layer of the pretrained model? i have a problem this is with VIT my image shape is 24x24 but the pretrained model input shape is 224x224 it is possible to fix that? and the learning parameter are 8900000 and i want to fine tune it on my dataset
recreated this in pycharm, when i want to use the model (i saved it first) i get this error: TypeError: No common supertype of TensorSpec(shape=(None, None), dtype=tf.int32, name=None) and TensorSpec(shape=(None, 78), dtype=tf.int64, name='input_ids_attention_mask'). is there a way to fix this without retraining the model?
I'd like to ask, a paper I am trying to use for another dataset said they had optimal performance at epochs=50, however at epochs=3, it's already getting decent performance. May I ask why this is? Also, do you run bert in inference mode?
Excellent video. I got one error while running the code. inputs = tokenizer(['Hello world', 'Hi how are you'], padding=True, truncation=True, return_tensors='tf') inputs For this line I got the following error: TypeError Traceback (most recent call last) Cell In[50], line 1 ----> 1 inputs = tokenizer(['Hello world', 'Hi how are you'], padding=True, truncation=True, 2 return_tensors='tf') 3 inputs TypeError: 'BertTokenizer' object is not callable Can you please help?
You can serialize a custom Keras model like our BERTForClassification using the model.save('filename') method. This will save the entire model, including the architecture, weights, and optimizer, to a file. You can then load the saved model using the tf.keras.models.load_model('filename') method. If the model.save() method doesn't work for you, you can use model.save_weights() instead. This method saves only the weights of the model to a file, so you will need to define the model architecture exactly as it was at the time of saving the weights in order to load the saved weights correctly. model = ... model.load_weights('/path/to/file')
@@PritishMishra This doesn't work. This is my code: classifier.save(save_path) classifier_2 = tf.keras.models.load_model(save_path) I get this error: TypeError: No common supertype of TensorSpec(shape=(None, None), dtype=tf.int32, name=None) and TensorSpec(shape=(None, 27), dtype=tf.int64, name='input_ids/attention_mask').
@@PritishMishra Yes, upon further research I came upon this solution too. The code that works looks like this: classifier.save_weights(save_path) classifier_2 = BERTForClassification(bert_model=transformers_model, num_classes=no_classes) classifier_2.load_weights(save_path) example_input = "just some dummy input string" encoded_example = tokenizer.encode(example_input, padding=True, truncation=True, return_tensors="tf") _ = classifier_2(encoded_example) The input is needed, because the bert input layer has a dynamic size, and gets built after running an input through it. Otherwise, other functions regarding the model (such as classifier_2.summary()) would return an error. Thanks for your help!
@@WoWmastersonTuralyon Thank you very much for the answer, but I don't understand why the answer that the model gives me when I do classifier_2(encoded_input) is: and not the category my input should be in
hi bro i facing error I'm using hugging face dataset if sentiment analysis. where in the dataset contain sentiment column and data type of sentiment column is string how to convert into integer and label number 1,2,3 in your case automatically convert into int64 please guide me I'm stock from last 7 days thanks for your attention
It used to be a LLM, but obviously now in the gen of trillions parameters model we can't say millions parameters model a LLM, but earlier it was a LLM , after few years these llama 2 and gpt 3 can't be LLM according to future models standard, but that does not change the fact of current scenario. hence, BERT is a LLM which trained on less parameters.
great video, can you make a video on question answering? and can we make a chatbot just using bert or will we be needing a Decoder along with bert for that
Hey, I really liked your video, although I have few doubts on a few matters, I would love to have a chat with you if it were possible, so you can help me. Do you think we can have a chat over Discord or similar?
Great video! any clue why I got so low accuracy ? The Secret to 90%+ Accuracy in Text Classification 32/32 ━━━━━━━━━━━━━━━━━━━━ 313s 10s/step - accuracy: 0.3567 - loss: 1.5853 [1.5781687498092651, 0.3684999942779541]
from the last 5-6 hrs i am trying to find a step-by-step proper material on how to fine to BERT with your dataset , finally found it , thank for making this video.
![BERT explained: Training, Inference, BERT vs GPT/LLamA, Fine tuning, [CLS] token](/img/1.gif)
Please Subscribe.
done👍
from the last 3-4 hrs i am trying to find a step-by-step proper material on how to fine to BERT with your dataset , finally found it , thank for making this video.
The video quality and the way you explained everything is Top-Notch. Thank you for this video
Really really amazing Pritish. This video is not like those boring lecture videos. The animations are amazing. your explanation is clear with goof pronunciation. Amazing. Keep it up. I hope you continue posting these type of videos. ❤❤❤❤
Amazing work Pritish. You definitely deserve more views. Hopefully you will get it soon❤.
Thanks a lot for this video. It's more than a simple tutorial, you really explain the most important concepts in a way that's clear
You have so much potential, amazing!
best explanation of fine tuning of bert , got good understanding from video thanks
best video i could find , easy, simple and to the point.
This is the best way of explaining the Models! Keep it up!!! I expect some plots/graphs on accuracy and predictions details!
Explanation done by you is the best compared to any others....awesome work Pritish ....keep it up
Best explanation in a simple and easy way.
i have some text files in which there are elements and their values but the pattern in which the text is displayed in the file are different from file to file. Is it possible to train Bert on these files so that when I ask it to extract only the element names and their corresponding values it will do that regardless of the text pattern?
Can you please explain how I can retrain the same model (after exporting) with new data. Basically, I want to train the same model in stages. Please help.
Very Nice Explanation and nice Animation 🔥🔥🔥🔥
Keep it up 👍🏻
i want to save this model and then convert it to tflite and then predict using tflite model what to do
so we don't need to freeze any layer of the pretrained model? i have a problem this is with VIT my image shape is 24x24 but the pretrained model input shape is 224x224 it is possible to fix that? and the learning parameter are 8900000 and i want to fine tune it on my dataset
Hi what if i want to train it on unsupervised learning like kmeans clustering?
Awesome video bro keep it up. ❤❤
recreated this in pycharm, when i want to use the model (i saved it first) i get this error: TypeError: No common supertype of TensorSpec(shape=(None, None), dtype=tf.int32, name=None) and TensorSpec(shape=(None, 78), dtype=tf.int64, name='input_ids_attention_mask'). is there a way to fix this without retraining the model?
all i want is to calculate precision, recall and f1 score of the model btw.
Very nice video; I wonder if it is possible to save the classifier for future use.
You can save the classifier as follows:
classifier.save("filename.h5")
How to fine tune csv dataset on BERT model
for the bert text summarization can we do in this way????
I'd like to ask, a paper I am trying to use for another dataset said they had optimal performance at epochs=50, however at epochs=3, it's already getting decent performance. May I ask why this is? Also, do you run bert in inference mode?
Can you do a video on how to do Natural language inference with Bert? Thanks!
Awesome explanation 👌
Excellent video. I got one error while running the code.
inputs = tokenizer(['Hello world', 'Hi how are you'], padding=True, truncation=True,
return_tensors='tf')
inputs
For this line I got the following error:
TypeError Traceback (most recent call last)
Cell In[50], line 1
----> 1 inputs = tokenizer(['Hello world', 'Hi how are you'], padding=True, truncation=True,
2 return_tensors='tf')
3 inputs
TypeError: 'BertTokenizer' object is not callable
Can you please help?
I’m not clear on what pooling in the video is.
Well done, Mishra
which model would be suitable for classifying if text is written by human or generated by LLM?
Yes.
it show me train_dataset is not defined????
Can we use bert for context aware similarity?
Can you make a llm by own data like chatgpt
thank bro!
Good explanation, it was easy to understand
How to import our dataset and train and test them
pritish will you create a video of simulating a robotic arm which is controlled by a GPT-language model , and can cook food in simulation ?
Absolutely brilliant!
Thanks brother
I am very impressed with the way you teach.
Very nicely explained
Hey, nice video! Just one question: how can you serialize a custom Keras model, such as yours -- class BERTForClassification(tf.keras.Model)?
You can serialize a custom Keras model like our BERTForClassification using the model.save('filename') method. This will save the entire model, including the architecture, weights, and optimizer, to a file. You can then load the saved model using the tf.keras.models.load_model('filename') method.
If the model.save() method doesn't work for you, you can use model.save_weights() instead. This method saves only the weights of the model to a file, so you will need to define the model architecture exactly as it was at the time of saving the weights in order to load the saved weights correctly.
model = ...
model.load_weights('/path/to/file')
@@PritishMishra This doesn't work. This is my code:
classifier.save(save_path)
classifier_2 = tf.keras.models.load_model(save_path)
I get this error:
TypeError: No common supertype of TensorSpec(shape=(None, None), dtype=tf.int32, name=None) and TensorSpec(shape=(None, 27), dtype=tf.int64, name='input_ids/attention_mask').
@@WoWmastersonTuralyon Could you please try with model.save_weights? Please let me know if this causes any errors.
@@PritishMishra Yes, upon further research I came upon this solution too. The code that works looks like this:
classifier.save_weights(save_path)
classifier_2 = BERTForClassification(bert_model=transformers_model, num_classes=no_classes)
classifier_2.load_weights(save_path)
example_input = "just some dummy input string"
encoded_example = tokenizer.encode(example_input, padding=True, truncation=True, return_tensors="tf")
_ = classifier_2(encoded_example)
The input is needed, because the bert input layer has a dynamic size, and gets built after running an input through it. Otherwise, other functions regarding the model (such as classifier_2.summary()) would return an error.
Thanks for your help!
@@WoWmastersonTuralyon Thank you very much for the answer, but I don't understand why the answer that the model gives me when I do classifier_2(encoded_input) is:
and not the category my input should be in
Really Good video.
Nice explanation.
Thanks
Thanks a lot for this video. Could you write the code how to do inference through pooler_outpt?
how to actually decode the output back to the classes is something this video did not explain : \
it was good video
hi bro i facing error
I'm using hugging face dataset if sentiment analysis.
where in the dataset contain sentiment column and data type of sentiment column is string how to convert into integer and label number 1,2,3
in your case automatically convert into int64
please guide me I'm stock from last 7 days
thanks for your attention
bert is not an LLM
It used to be a LLM, but obviously now in the gen of trillions parameters model we can't say millions parameters model a LLM, but earlier it was a LLM , after few years these llama 2 and gpt 3 can't be LLM according to future models standard, but that does not change the fact of current scenario. hence, BERT is a LLM which trained on less parameters.
great video, can you make a video on question answering? and can we make a chatbot just using bert or will we be needing a Decoder along with bert for that
Hey, I really liked your video, although I have few doubts on a few matters, I would love to have a chat with you if it were possible, so you can help me. Do you think we can have a chat over Discord or similar?
Great video! any clue why I got so low accuracy ?
The Secret to 90%+ Accuracy in Text Classification
32/32 ━━━━━━━━━━━━━━━━━━━━ 313s 10s/step - accuracy: 0.3567 - loss: 1.5853
[1.5781687498092651, 0.3684999942779541]
same here
accuracy: 0.3741 - loss: 1.5632
Hey, there's some problems with the code. Let me try to fix this.
I got 94% F1 with bert-base-uncased
from the last 5-6 hrs i am trying to find a step-by-step proper material on how to fine to BERT with your dataset , finally found it , thank for making this video.