It really gives me joy when you upload new videos and to becfrank.. Your explaination has connected various dots in my mind in this Topic.. Really greatful to you.. You fit the definition of a Mentor Coach and a true teacher... Thank you
Hi Kapil, QQ: if we have in_channels=16, out_channels=4, groups=2, what will each filter bank see? I believe filter bank 1 and 2 should see input channel 1-8 and 9-16 respectively. How about filter bank 3 and 4?
First, your understanding of how many filter banks will be created is correct. Since we are asking for 4 out_channels we will have 4 filter banks. Now let's first go with a simple case of group=1 (i.e. default setup). This would mean that you have 16 kernels in every filter bank. Total kernels will be 16*4. With grouping, we can reduce the number of kernels. In your case, you want two 2 groups so your input channels are now in groups of 8, which means there will be 8 kernels per filter bank. But we still will have 4 filter banks since you specified 4 output channels. Here is what will happen in this scenario: Filter Bank 1 will use its 8 kernels to correlate with the first 8 input channels (i.e. group 1) Filter Bank 2 will use its 8 kernels to correlate with the first 8 input channels (i.e. group 1) as well Filter Bank 3 will use its 8 kernels to correlate with the second 8 input channels (i.e. group 2) Filter Bank 4 will use its 8 kernels to correlate with the second 8 input channels (i.e. group 2) as well It's very easy to get confused by all these arrangements. I prefer to manually verify it. I have created an additional notebook for you where you can verify the arithmetic and my answer for this configuration. See the link below. Hope this helps. colab.research.google.com/drive/1RR5fyyuvrE63q_3ELq8GFACSanmXWQyX?usp=sharing
@@KapilSachdeva Thank you so much for your explanation! It is clear to me now though I don't get why PyTorch adopted such arrangements (perhaps due to low-level computation reasons?). Your videos are so inspirational. I really appreciate your hard work sharing your knowledge!
Every input channel has corresponding kernel? Really? Actually, count(Kernel.channel) == count(input.channel). And the number of kernel == number of channels in output. I have not watched your other videos, but what you said in recap in this video sure sounds wrong/misleading.
Hi Mohammad, thanks for the expressing the concern. Would appreciate if u can watch the video where I clarify the difference between kernels and filters. ruclips.net/video/HALYTCQBcO4/видео.html Let me know if you still believe I am making a mistake and then we will sort it out.
Professor Kaiming He is the GOD of Deep Residual Networks .
It really gives me joy when you upload new videos and to becfrank.. Your explaination has connected various dots in my mind in this Topic.. Really greatful to you.. You fit the definition of a Mentor Coach and a true teacher... Thank you
🙏 thanks for your kind words. It makes me happy to know that these tutorials are helpful.
Hi Kapil, QQ: if we have in_channels=16, out_channels=4, groups=2, what will each filter bank see? I believe filter bank 1 and 2 should see input channel 1-8 and 9-16 respectively. How about filter bank 3 and 4?
First, your understanding of how many filter banks will be created is correct. Since we are asking for 4 out_channels we will have 4 filter banks. Now let's first go with a simple case of group=1 (i.e. default setup). This would mean that you have 16 kernels in every filter bank. Total kernels will be 16*4. With grouping, we can reduce the number of kernels. In your case, you want two 2 groups so your input channels are now in groups of 8, which means there will be 8 kernels per filter bank. But we still will have 4 filter banks since you specified 4 output channels.
Here is what will happen in this scenario:
Filter Bank 1 will use its 8 kernels to correlate with the first 8 input channels (i.e. group 1)
Filter Bank 2 will use its 8 kernels to correlate with the first 8 input channels (i.e. group 1) as well
Filter Bank 3 will use its 8 kernels to correlate with the second 8 input channels (i.e. group 2)
Filter Bank 4 will use its 8 kernels to correlate with the second 8 input channels (i.e. group 2) as well
It's very easy to get confused by all these arrangements. I prefer to manually verify it. I have created an additional notebook for you where you can verify the arithmetic and my answer for this configuration. See the link below. Hope this helps.
colab.research.google.com/drive/1RR5fyyuvrE63q_3ELq8GFACSanmXWQyX?usp=sharing
@@KapilSachdeva Thank you so much for your explanation! It is clear to me now though I don't get why PyTorch adopted such arrangements (perhaps due to low-level computation reasons?). Your videos are so inspirational. I really appreciate your hard work sharing your knowledge!
🙏
Every input channel has corresponding kernel? Really?
Actually, count(Kernel.channel) == count(input.channel). And the number of kernel == number of channels in output.
I have not watched your other videos, but what you said in recap in this video sure sounds wrong/misleading.
Hi Mohammad, thanks for the expressing the concern. Would appreciate if u can watch the video where I clarify the difference between kernels and filters. ruclips.net/video/HALYTCQBcO4/видео.html
Let me know if you still believe I am making a mistake and then we will sort it out.