One of the best intro to RL. Recommended to every student interested in this field to watch this amazing lecture. I have just completed it at 1:40 AM...Now waiting for Actor-Critic Type RL Agent to be released soon...Thanks and Good night.
Amazing intro to the subject. Since it is interrelated to control theory it is mandatory to have a good back ground on control theory such as state space models and optimal control
The total reward is from time 't' to a later time/time in far future (t^inf). Initial value of reward is r_t. The next one will be r_{t+1}, r_{t+2}, ..., till termination-assumed some time in the future but can be user chosen, e.g. time {t+n} as the termination time. Remember you can be rewarded now (t) or anytime in the far future (inf) so you sum over the entire duration.
You’re correct the lecturer screwed up here. What he says in spoken language does not match the equation he shows. His equation is the expected return (total future rewards) from time t given no uncertainty in future rewards as you follow the policy until the end of an episode, but in language he claims it is the sum of all rewards from time t=0 to time t, but that is clearly not what the equation states. I haven’t finished this but it’s likely the equation is right but his statement in language is wrong given he then shows the form where you discount future rewards. You would not discount past rewards which is why I think the equation is right but he just is not describing the equation properly in language.
can you advise for my startup, we applied for YC, we want to setup up indian team and RLHF as well as using SIMPO to agentify the hospital system and remove the inefficiences faced in the current hospital systems. im an aussie coming to america. we have hardware as well, been in guangzhou for the last 6 weeks finding the best containers and cameras triend to train for guaging container volume for measuring stock remaining.
Please repeat questions, question askers audio is blown out or intelligible. Some of the questions manage to be in the captions others but not all. The professors mic is perfect however with a great mix one of the few series where you don't have to be max volume all the time.
Can one conclude from the AlphaGo vs. AlphaZero showcase, that the bottleneck of "achieving" AGI/ASI, are we humans and the ethical/safety restrictions we have set?
Both alphago and zero rely on world models (and self play) which they can use to try out or plan different moves based on the simulated results. While it's super easy to do this simulation in board games, where the rules are deterministic, creating such a world model for something with drastically more complexity like the real world is far more challenging. Algorithims like MuZero, which use learned models, are getting their, but technically speaking, Deepminds got a lot more work to do before they can make Alpha-terminator ;)
Transformers can be used as a direct replacement for DRL since it can process sequences as well. There is an article in medium related to this alternative.
Hi, when i tried to run the modeling building part of lab 1, the line "tf.keras.layers.Embedding(vocab_size, embedding_dim, batch_input_shape=[batch_size, None])," does not work, and the error says batch_input_shape is an unrecognized keyword argument to Embeddings, has anyone else encountered this problem? I looked up the tf.keras.Embeddings documentation and couldnt' find anything to replace it...What did you guys to solve it? Thanks!
Hi, from my understanding when looking at TF documentation, Embeding doesn't take a batch_input_shape as parameter. I justg went ahead and executed the embedding as: tf.keras.layers.Embedding(input_dim=vocab_size, output_dim=embedding_dim) and it worked for me.
One of the best intro to RL. Recommended to every student interested in this field to watch this amazing lecture. I have just completed it at 1:40 AM...Now waiting for Actor-Critic Type RL Agent to be released soon...Thanks and Good night.
I'm curious to listen to this lecture. I need more concepts to apply in my Thesis. I'm looking forward to seeing this happen soon.
Babe wake up new 6.S191 lecture just dropped
Lol...
@@BheezHandle Feel the vibessssss
U got it
Amazing intro to the subject. Since it is interrelated to control theory it is mandatory to have a good back ground on control theory such as state space models and optimal control
Awesome job. Only curious if someone can explain how was the target part of the loss function computed at 26:40?
immediate reward + discounted best possible future reward
Lovely lecture.❤
Self driving car is a dynamic environment as compared to Gaming environment. It may be mentioned.
Following since 3 years
10:30 equation for total reward should be summation of rewards from t=0 to t=t, right? But in equation its from t to infinity...why?
The total reward is from time 't' to a later time/time in far future (t^inf). Initial value of reward is r_t. The next one will be r_{t+1}, r_{t+2}, ..., till termination-assumed some time in the future but can be user chosen, e.g. time {t+n} as the termination time. Remember you can be rewarded now (t) or anytime in the far future (inf) so you sum over the entire duration.
You’re correct the lecturer screwed up here. What he says in spoken language does not match the equation he shows. His equation is the expected return (total future rewards) from time t given no uncertainty in future rewards as you follow the policy until the end of an episode, but in language he claims it is the sum of all rewards from time t=0 to time t, but that is clearly not what the equation states. I haven’t finished this but it’s likely the equation is right but his statement in language is wrong given he then shows the form where you discount future rewards. You would not discount past rewards which is why I think the equation is right but he just is not describing the equation properly in language.
can you advise for my startup, we applied for YC, we want to setup up indian team and RLHF as well as using SIMPO to agentify the hospital system and remove the inefficiences faced in the current hospital systems. im an aussie coming to america. we have hardware as well, been in guangzhou for the last 6 weeks finding the best containers and cameras triend to train for guaging container volume for measuring stock remaining.
This is Awesome !!!!!
Please repeat questions, question askers audio is blown out or intelligible.
Some of the questions manage to be in the captions others but not all.
The professors mic is perfect however with a great mix one of the few series where you don't have to be max volume all the time.
Can one conclude from the AlphaGo vs. AlphaZero showcase, that the bottleneck of "achieving" AGI/ASI, are we humans and the ethical/safety restrictions we have set?
Both alphago and zero rely on world models (and self play) which they can use to try out or plan different moves based on the simulated results. While it's super easy to do this simulation in board games, where the rules are deterministic, creating such a world model for something with drastically more complexity like the real world is far more challenging. Algorithims like MuZero, which use learned models, are getting their, but technically speaking, Deepminds got a lot more work to do before they can make Alpha-terminator ;)
Transformers can be used as a direct replacement for DRL since it can process sequences as well. There is an article in medium related to this alternative.
Got a link?
Hi, when i tried to run the modeling building part of lab 1, the line "tf.keras.layers.Embedding(vocab_size, embedding_dim, batch_input_shape=[batch_size, None])," does not work, and the error says batch_input_shape is an unrecognized keyword argument to Embeddings, has anyone else encountered this problem? I looked up the tf.keras.Embeddings documentation and couldnt' find anything to replace it...What did you guys to solve it? Thanks!
Hi, from my understanding when looking at TF documentation, Embeding doesn't take a batch_input_shape as parameter. I justg went ahead and executed the embedding as: tf.keras.layers.Embedding(input_dim=vocab_size, output_dim=embedding_dim) and it worked for me.
Needed u
太牛了
👏
🎉
Is there an application of reinforcement learning for subsurface reservoir simulation?
Kenchin kokoro no.tabi.Study of the waste.