i have been trying to do this exact same thing for the last month. I took the exact same approach without that 13th dimension and never got good results, its impressive how such a little thing can have such a huge impact. Thanks a lot, great video
You can also add a 14th dimension for even better results. The 13th channel shows where you can move pieces, but it doesn't show from which squares you can move them. I've already tried it; the results will be better, though blunders are inevitable, of course. Here is how you can modify the board_to_matrix function to include the 14th dimension (auxiliary_func.py, from line 22): for move in legal_moves: to_square = move.to_square from_square = move.from_square row_to, col_to = divmod(to_square, 8) row_from, col_from = divmod(from_square, 8) matrix[12, row_to, col_to] = 1 matrix[13, row_from, col_from] = 1 And thanks a lot for your comment! I'm glad you liked the video!
@@_NickTech I implemented your idea and something wasnt working so i decided to use your exact same architecture and it worked perfectly fine. Could you please explain how you decided to take that exact architecture? thanks! This was my architecture before: class Net(nn.Module): def __init__(self): super(Net, self).__init__() self.a1 = nn.Conv2d(5, 16, kernel_size=3, padding=1) self.a2 = nn.Conv2d(16, 16, kernel_size=3, padding=1) self.a3 = nn.Conv2d(16, 32, kernel_size=3, stride=2) self.b1 = nn.Conv2d(32, 32, kernel_size=3, padding=1) self.b2 = nn.Conv2d(32, 32, kernel_size=3, padding=1) self.b3 = nn.Conv2d(32, 64, kernel_size=3, stride=2) self.c1 = nn.Conv2d(64, 64, kernel_size=2, padding=1) self.c2 = nn.Conv2d(64, 64, kernel_size=2, padding=1) self.c3 = nn.Conv2d(64, 128, kernel_size=2, stride=2) self.d1 = nn.Conv2d(128, 128, kernel_size=1) self.d2 = nn.Conv2d(128, 128, kernel_size=1) self.d3 = nn.Conv2d(128, 128, kernel_size=1) self.last = nn.Linear(128, 1) def forward(self, x): x = F.relu(self.a1(x)) x = F.relu(self.a2(x)) x = F.relu(self.a3(x)) # 4x4 x = F.relu(self.b1(x)) x = F.relu(self.b2(x)) x = F.relu(self.b3(x)) # 2x2 x = F.relu(self.c1(x)) x = F.relu(self.c2(x)) x = F.relu(self.c3(x)) # 1x128 x = F.relu(self.d1(x)) x = F.relu(self.d2(x)) x = F.relu(self.d3(x)) x = x.view(-1, 128) x = self.last(x) # value output return F.tanh(x)
@nmxnunezz8214 hey, I see two major issues: 1. Why nn.Linear(128, 1) at the end? Are you solving a regression task? And why scale the output to the range [-1, 1] with F.tanh(x)? This should be a classification task, not regression. The last layer should be nn.Linear(128, number_of_unique_moves). Then apply softmax to get probabilities. 2. Do you only have 5 channels? Why so few? I inferred this from nn.Conv2d(5, ... ).
Interesting! I am also making my own chess engine at the moment and am shocked to see what is similar and different in our approaches. I believe I have some stuff that might make your model better. First off, the 13th channel in your input data might really confuse the model considering it only contains data about where pieces can move TO, not WHICH piece can do that and from where. Instead, you should have the 13th channel completely filled with 1s or 0s based on what side it is playing (since right now it has to also predict if it is playing black or white). Not directly telling the model which side it is playing IS THE REASON why it was previously prone to making many illegal moves since it could not train well (I did the same mistake of having 12 channels also). Sure, it might still want to make illegal moves sometimes, but instead of doing a legal check for each input board position, you can check if the predicted move is legal, and if it is not, choose the next best-predicted move. I saw you made a comment about a 14th dimension, and even though I can buy your reasoning, I do not believe this is the most effective strategy considering the model will have to make many extra predictions along the way when many FROM squares are filled and many TO squares are filled. Also during training, opening moves are irrelevant considering many of them are interchangeable. Your model will be punished by choosing one opening over another, which is not fair. I was shocked to see that even though my model never trained on opening moves, it could still predict near-perfect theory when getting better at the game as a whole. It also reached better precision after training without opening moves because it was not unfairly punished. Another big tip is using batch normalization after each convolutional layer. It might make the training process much slower, but each part of the training gets much more efficient, resulting in a much higher precision after the same amount of epochs. For example, my model took about 7000 games of training to reach 2% accuracy without batch normalization and after implementation, it was reached already after 500 games.
Hey! I love your tips, especially the one about openings-it's truly shocking! You're right-legal check is indeed algorithmically inefficient in my implementation. And telling the model which side it's playing sounds so logical! However, I still think it should have an extra channel indicating which pieces it can move (this should be tested in an experiment). Though I agree, telling WHERE it can move them is really ambiguous. By the way, accuracy doesn't seem like a reasonable metric here (I understand it's easier to compute). I would suggest Top-3 or Top-5 Accuracy, since sometimes there can be up to even 10 decent moves
@@_NickTech Yes, you could absolutely be right about the extra channel, but maybe one could create some system to embed more information inside it instead of using only 1s and 0s? For example some kind of embedding to make it possible for the model to understand exactly which piece can move to some specific square. It would be really interesting to see how these kinds of inputs change the outcome of the model, especially if it is a solution you come up with by yourself! And yes, you are completely right about the accuracy, it is a super good tip and something I will implement now! Looking forward to the reinforcement 😉
The problem is the sample size of your dataset, its wayyy too small. It learns the openings and part of the middlegame almost by heart but then generalizes very poorly once its out of book. A general rule of thumb is 1m positions per hidden layer node, and thats just for nnue and not a policy net as you are trying to train. The community around Leela Chess Zero should be able to help you with those kind of chess bots :)
What you can do to improve it is follow DeepMind's approach. You trained your supervised model and its pretty decent, It could probably beat any 900 elo player. What you wanna do now is take the model and chuck it into reinforcement learning to roughen up the endgame edges. I believe early game is strong because early game is consistent in training data, but no 2 end games are the same so it's lacking in training material there. I feel reinforcement learning here would greatly improve it.
That is an awesome video! I have two main questions: 1. Why did you shuffle the dataset, since the games itself might be considered as a series of moves through time (though one game has nothing to do with the other). 2. Why did you choose CNN, I understand that you could frame the input as an image, but it would be fun to see the difference to a regular MLP. Also I've played against your bot and it seems to be very solid (congrats) it also made the same blunder as it did when playing against you (giving away the rook with a check) - since the bot is actually learning from moves (and there are 10^120) and most games have early and midgames but not so many late games, it may seem that the use of a evaluation calculation might be needed (hence Reinforcement Learning)
Hey, thanks for your comment! 😊 The dataset includes sequential moves where each move depends on previous states. Shuffling helps prevent the model from learning sequential dependencies that do not generalize well to new games. Otherwise, it might only learn specific openings, for example. Shuffling ensures that the model sees a variety of positions in each batch, which is beneficial for learning. Using an MLP for this task is interesting, but I doubt it would catch any patterns as effectively as a CNN. CNNs seem logical to me for this kind of data. And yes, you are right: evaluation calculation is not only needed but essential! For the CNN model, there is no difference between a blunder and a brilliant move. Only evaluation can solve that problem. Regarding endgames, there is one obvious issue: there are so many possible positions. Openings are relatively standard: develop pieces, control the center, and castle your king. Middlegames are trickier, but endgames are extraordinarily varied! There are no clear patterns in them
i didnt get the action space part as ı understand your action space is the number of unique moves in the dataset, and ı think it leads to the model cant choose any other move that is not in the data set. The move index in the action space depends on the data as well. you can use alpha zeros action representation 56 queen moves (8 direction 7 max length) 8 knight moves and 9 underpromotion(3 pieces 3 direction) for 8*8 board in the and u have 73*8*8 matrix after flattening 4672 length action space.
Hey, you got it correctly! However, when the dataset is quite large, the probability that there won't be a needed move is really low (unless you start playing 'exotic' chess). You can plot a graph of the number of unique moves versus the number of parsed games. The convergence to the asymptote is good. But sure, you can create an action space that includes all possible moves
@@_NickTech why pytorch one is considerably better even with same architecture can you explain? How much ram ,vram does it used,also have you use multiprocessing
You mean in comparison with the TF model from my other video? If so, the architecture is different: I changed the input (there were 12 channels; I added one more). If you apply that change to the TF model, then you will get a similar result. Though PyTorch has a more efficient memory management system (for example, it uses a dynamic computation graph rather than a static one like in TF), which allowed me to train a model using more data. I didn't use multiprocessing, though you can easily add it to the data loading by specifying *num_workers*: data_loader = DataLoader(dataset, batch_size, num_workers=4) About RAM: 16-30 GB were in use, depending on how many samples you keep in your dataset.
@@ponmuthu..4796 My friend has already created an interface for it, which I'll cover in the next video: setday.github.io/chess-engine-online/ his github repo: github.com/setday/chess-engine-online But you can create it as well for the sake of practicing :)
Есть какое-то логическое обоснование тому, почему при одинаковых слоях и параметрах модели так по-разному отрабатывают? Или вся разница как раз в 13м канале? Спасибо за видео
Yes! You're absolutely right! The 13th channel did all the work here. (I don't remember the precise numbers, but when I trained the TF model with the 13th channel, the convergence and performance were about the same). It's actually logical because this 13th channel indicates some rules of chess, and the correlation with the target is obviously high. I should've mentioned this in the video 😬 Thank you for your comment!
Play against this engine here:
setday.github.io/chess-engine-online/
i have been trying to do this exact same thing for the last month. I took the exact same approach without that 13th dimension and never got good results, its impressive how such a little thing can have such a huge impact. Thanks a lot, great video
You can also add a 14th dimension for even better results. The 13th channel shows where you can move pieces, but it doesn't show from which squares you can move them. I've already tried it; the results will be better, though blunders are inevitable, of course.
Here is how you can modify the board_to_matrix function to include the 14th dimension (auxiliary_func.py, from line 22):
for move in legal_moves:
to_square = move.to_square
from_square = move.from_square
row_to, col_to = divmod(to_square, 8)
row_from, col_from = divmod(from_square, 8)
matrix[12, row_to, col_to] = 1
matrix[13, row_from, col_from] = 1
And thanks a lot for your comment! I'm glad you liked the video!
@@_NickTech I implemented your idea and something wasnt working so i decided to use your exact same architecture and it worked perfectly fine.
Could you please explain how you decided to take that exact architecture? thanks!
This was my architecture before:
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.a1 = nn.Conv2d(5, 16, kernel_size=3, padding=1)
self.a2 = nn.Conv2d(16, 16, kernel_size=3, padding=1)
self.a3 = nn.Conv2d(16, 32, kernel_size=3, stride=2)
self.b1 = nn.Conv2d(32, 32, kernel_size=3, padding=1)
self.b2 = nn.Conv2d(32, 32, kernel_size=3, padding=1)
self.b3 = nn.Conv2d(32, 64, kernel_size=3, stride=2)
self.c1 = nn.Conv2d(64, 64, kernel_size=2, padding=1)
self.c2 = nn.Conv2d(64, 64, kernel_size=2, padding=1)
self.c3 = nn.Conv2d(64, 128, kernel_size=2, stride=2)
self.d1 = nn.Conv2d(128, 128, kernel_size=1)
self.d2 = nn.Conv2d(128, 128, kernel_size=1)
self.d3 = nn.Conv2d(128, 128, kernel_size=1)
self.last = nn.Linear(128, 1)
def forward(self, x):
x = F.relu(self.a1(x))
x = F.relu(self.a2(x))
x = F.relu(self.a3(x))
# 4x4
x = F.relu(self.b1(x))
x = F.relu(self.b2(x))
x = F.relu(self.b3(x))
# 2x2
x = F.relu(self.c1(x))
x = F.relu(self.c2(x))
x = F.relu(self.c3(x))
# 1x128
x = F.relu(self.d1(x))
x = F.relu(self.d2(x))
x = F.relu(self.d3(x))
x = x.view(-1, 128)
x = self.last(x)
# value output
return F.tanh(x)
@nmxnunezz8214 hey, I see two major issues:
1. Why nn.Linear(128, 1) at the end? Are you solving a regression task? And why scale the output to the range [-1, 1] with F.tanh(x)?
This should be a classification task, not regression. The last layer should be nn.Linear(128, number_of_unique_moves). Then apply softmax to get probabilities.
2. Do you only have 5 channels? Why so few? I inferred this from nn.Conv2d(5, ... ).
Interesting! I am also making my own chess engine at the moment and am shocked to see what is similar and different in our approaches. I believe I have some stuff that might make your model better.
First off, the 13th channel in your input data might really confuse the model considering it only contains data about where pieces can move TO, not WHICH piece can do that and from where. Instead, you should have the 13th channel completely filled with 1s or 0s based on what side it is playing (since right now it has to also predict if it is playing black or white). Not directly telling the model which side it is playing IS THE REASON why it was previously prone to making many illegal moves since it could not train well (I did the same mistake of having 12 channels also). Sure, it might still want to make illegal moves sometimes, but instead of doing a legal check for each input board position, you can check if the predicted move is legal, and if it is not, choose the next best-predicted move. I saw you made a comment about a 14th dimension, and even though I can buy your reasoning, I do not believe this is the most effective strategy considering the model will have to make many extra predictions along the way when many FROM squares are filled and many TO squares are filled.
Also during training, opening moves are irrelevant considering many of them are interchangeable. Your model will be punished by choosing one opening over another, which is not fair. I was shocked to see that even though my model never trained on opening moves, it could still predict near-perfect theory when getting better at the game as a whole. It also reached better precision after training without opening moves because it was not unfairly punished.
Another big tip is using batch normalization after each convolutional layer. It might make the training process much slower, but each part of the training gets much more efficient, resulting in a much higher precision after the same amount of epochs. For example, my model took about 7000 games of training to reach 2% accuracy without batch normalization and after implementation, it was reached already after 500 games.
Hey! I love your tips, especially the one about openings-it's truly shocking!
You're right-legal check is indeed algorithmically inefficient in my implementation.
And telling the model which side it's playing sounds so logical! However, I still think it should have an extra channel indicating which pieces it can move (this should be tested in an experiment). Though I agree, telling WHERE it can move them is really ambiguous.
By the way, accuracy doesn't seem like a reasonable metric here (I understand it's easier to compute). I would suggest Top-3 or Top-5 Accuracy, since sometimes there can be up to even 10 decent moves
@@_NickTech Yes, you could absolutely be right about the extra channel, but maybe one could create some system to embed more information inside it instead of using only 1s and 0s? For example some kind of embedding to make it possible for the model to understand exactly which piece can move to some specific square. It would be really interesting to see how these kinds of inputs change the outcome of the model, especially if it is a solution you come up with by yourself!
And yes, you are completely right about the accuracy, it is a super good tip and something I will implement now!
Looking forward to the reinforcement 😉
The problem is the sample size of your dataset, its wayyy too small. It learns the openings and part of the middlegame almost by heart but then generalizes very poorly once its out of book. A general rule of thumb is 1m positions per hidden layer node, and thats just for nnue and not a policy net as you are trying to train. The community around Leela Chess Zero should be able to help you with those kind of chess bots :)
What you can do to improve it is follow DeepMind's approach. You trained your supervised model and its pretty decent, It could probably beat any 900 elo player. What you wanna do now is take the model and chuck it into reinforcement learning to roughen up the endgame edges. I believe early game is strong because early game is consistent in training data, but no 2 end games are the same so it's lacking in training material there. I feel reinforcement learning here would greatly improve it.
That is an awesome video! I have two main questions: 1. Why did you shuffle the dataset, since the games itself might be considered as a series of moves through time (though one game has nothing to do with the other). 2. Why did you choose CNN, I understand that you could frame the input as an image, but it would be fun to see the difference to a regular MLP.
Also I've played against your bot and it seems to be very solid (congrats) it also made the same blunder as it did when playing against you (giving away the rook with a check) - since the bot is actually learning from moves (and there are 10^120) and most games have early and midgames but not so many late games, it may seem that the use of a evaluation calculation might be needed (hence Reinforcement Learning)
Hey, thanks for your comment! 😊
The dataset includes sequential moves where each move depends on previous states. Shuffling helps prevent the model from learning sequential dependencies that do not generalize well to new games. Otherwise, it might only learn specific openings, for example. Shuffling ensures that the model sees a variety of positions in each batch, which is beneficial for learning.
Using an MLP for this task is interesting, but I doubt it would catch any patterns as effectively as a CNN. CNNs seem logical to me for this kind of data.
And yes, you are right: evaluation calculation is not only needed but essential! For the CNN model, there is no difference between a blunder and a brilliant move. Only evaluation can solve that problem.
Regarding endgames, there is one obvious issue: there are so many possible positions. Openings are relatively standard: develop pieces, control the center, and castle your king. Middlegames are trickier, but endgames are extraordinarily varied! There are no clear patterns in them
In future save the tensor data to a file and read it in chunks for training so you can train on a larger dataset without memory issues
Sure! Thank you for advising
really decent bot, it is better than my pure nural network bot,my bot gives evaluation value to a position as output than negamax
Thanks for watching!
i didnt get the action space part as ı understand your action space is the number of unique moves in the dataset, and ı think it leads to the model cant choose any other move that is not in the data set. The move index in the action space depends on the data as well. you can use alpha zeros action representation 56 queen moves (8 direction 7 max length) 8 knight moves and 9 underpromotion(3 pieces 3 direction) for 8*8 board in the and u have 73*8*8 matrix after flattening 4672 length action space.
Hey, you got it correctly! However, when the dataset is quite large, the probability that there won't be a needed move is really low (unless you start playing 'exotic' chess). You can plot a graph of the number of unique moves versus the number of parsed games. The convergence to the asymptote is good. But sure, you can create an action space that includes all possible moves
thats crazyyyy!
try using re-enforcement learning
it's on my TODO list, though I've been told that convergence will be poor. We'll see
@@_NickTech why pytorch one is considerably better even with same architecture can you explain? How much ram ,vram does it used,also have you use multiprocessing
You mean in comparison with the TF model from my other video? If so, the architecture is different: I changed the input (there were 12 channels; I added one more). If you apply that change to the TF model, then you will get a similar result. Though PyTorch has a more efficient memory management system (for example, it uses a dynamic computation graph rather than a static one like in TF), which allowed me to train a model using more data.
I didn't use multiprocessing, though you can easily add it to the data loading by specifying *num_workers*:
data_loader = DataLoader(dataset, batch_size, num_workers=4)
About RAM: 16-30 GB were in use, depending on how many samples you keep in your dataset.
@@_NickTech may I create a interface for that(I can use the frontend from by University project with improvements)
@@ponmuthu..4796 My friend has already created an interface for it, which I'll cover in the next video:
setday.github.io/chess-engine-online/
his github repo: github.com/setday/chess-engine-online
But you can create it as well for the sake of practicing :)
Есть какое-то логическое обоснование тому, почему при одинаковых слоях и параметрах модели так по-разному отрабатывают? Или вся разница как раз в 13м канале? Спасибо за видео
Yes! You're absolutely right! The 13th channel did all the work here. (I don't remember the precise numbers, but when I trained the TF model with the 13th channel, the convergence and performance were about the same).
It's actually logical because this 13th channel indicates some rules of chess, and the correlation with the target is obviously high. I should've mentioned this in the video 😬
Thank you for your comment!
@@_NickTech Thanks for the response. Good luck with your future videos!
weak model I train with 10k games but still cant play decent moves.