Видео

Is it correct that sarsa with greedy policy that choses action that have max value is equal to q learning?

does this mean it's not even using a neural network?

I don't feel like I understand the principle from your video- what is the purpose of partitioning the state into tiles? How and when are they assigned a Q value and when is it modified? Are the Q values just zero during the first epoch? Does this work for larger state spaces? Does the agent really learn anything substantial from a replay of a 40k steps Epoch?

The simplest explanation!

Hey Marcus. I hope this message finds you well! I recently discovered your RUclips channel while scrolling homepage and found your content to be amazing and incredibly helpful. I have a cooperation proposal for you and would love to discuss it further. Please let me know if you're interested! It will be great if you will give me an email address to send the details.

notebook ?

Good try.

mnist

Helpful, thank you!

I love this! I have been making some of my first bots for games like tictactoe and gomoku in a more traditional way with minimax and stuff like that but i am also looking forward to tackle these kind of learning models as well. you deserve more views.

All circles are proprtionate to each other, by having a fixed centerpoint for both you will have a fixed length for a single rotation because the length between the centerpoint and the surface being rolled upon is the same, youre essentially fixing a varible wich changes the path of movement, try taking to circles, one large one small that can fit in it, roll them both one rotation seperately with thier edges both touching the surface, you will absolutely have 2 seperate lengths

could you please do a video for monte carlo for chapter 5 from the reinforcement learning book (Sutton) you use please? Got an exam soon and would be of great help. Great videos!

I think the distance is not the same. Rather the displacement is the same. So the distance is different for same time period, hence different angular velocity.

Before watching the video, I don't know how to prove it mathematically but I think it's because the small circle is within the larger circle and you're talking the dot through a shorter path since it's not rolling flat. It's going to be slight higher when the the larger circle is on the table and when the dot on the larger circle reaches the peak, the smaller circle dot is lower. Therefore slightly shorter path

Why is he dividing by 2pi. It's just divide by 2 to get the radius, why are you involving pi

This is very cool progress do u have a code repo for your learning?

Yes, the cycloids. But, you forgot to see that by a rolling of 360° the smaller perimeter is covering the same linear distance as the bigger one. It means, both perimeters have the same length as the straight distance they cover with a 360° rolling. This is a big problem.

plagiarism means the use of another's work, words, or ideas without attribution. key word is another. i would consider another as someone with a conscious mind or thought and would argue that ai is not conscious yet. meaning no plagiarism hasn't occurred.

Nice job! Helps my understanding. Keep making such videos.

Nice.

Thanks

i love u man

this really boosted my understanding

Thank u for your explanation, keep posting us more videos please! ❤

what moves more? The centre moves an little. The outerlayer moved an lot more. So they don't go the same speed

This is the same phenomena as rolling a coin around another coin. The distance and number of rotations is related to the distance the center of the circle travels. Also the two circles, which are connected have the same angular velocity, but different linear velocities. The difference in linear distance can be accounted for by observing the path a single point on each circle takes, called a cycloid. Looking at the two cycloids, it is clear that the point on the inner circle travels a smaller distance.

Because of the gravitational acceleration do the same in space see the outcome 😁👍

Dummy-proof explanation for a helicopter pilot = Same RPM, Different velocity.

Think about the angular velocity and tangential velocity and compare them. The large and small circles have the same angular velocity, because they both do a full rotation over the same time. However, for that to be the case, they must have different tangential velocities, as their diameters are different. Since the large circle is travelling along a linear path, its tangential velocity will determine the distance, and the small circle will have a lower tangential velocity, making up for the “missing” distance the inner circle has to travel.

Because they're connected. If you took a 5mm wheel and a 10mm wheel and roll them along a table the 5mm has to do more totations. But now if you put the 5mm inside attached to the larger, like a hub for example it's elevated above the rolling surface by the same amount all the way around as they roll together.

woww, nice visualization and explanation, keep on making more vidoes, u deserve many more views, good luck!

Nice explanation

Thanks for this introductory video. It helped me a lot.

Wow I never thought about it like that! Cool video

Hello, where could i find code for that?

Very interesting!

Nice video! Thanks! : )

I've never understood the cycloid explanation, although I've heard it goes back centuries. Just to have a concrete set-up, let's divide the wheels into 360 sectors of 1 degree each. Let's further suppose that the larger wheel has a circumference of 360 cm and the smaller a circumference of 180 cm. So each 1-degree sector corresponds to 1 cm of circumference on the outer wheel and 0.5 cm of circumference on the inner wheel. The outer wheel rolls along a 360 cm long track and, at the same time, the inner wheel rolls along its own (real or imaginary) 360 cm long track. Let's mark off 1cm intervals along both tracks. We might even number the intervals from 1 to 360 (from left to right, with the wheel starting from the left end), and also number the sectors of the circumference (in the counterclockwise direction starting from the initial contact point) from 1 to 360. Here's how I see the paradox (and I think this is essentially how it is stated in the original Mechanica text): as the outer wheel rolls, first the 1 cm long sector #1 of the outer circumference contacts the 1 cm long interval #1 of the track, then the 1 cm long sector #2 of the outer circumference contacts the 1 cm long interval #2 of the track, then sector #3 contacts interval #3, and so on for all 360 sectors. At the same time this is happening, first the 0.5 cm long sector #1 of the inner circumference contacts (the whole of) the 1 cm long interval of its track, then the 0.5 cm long sector #2 of the inner circumference contacts (the whole of) the 1 cm long interval #2 of its track, then sector #3 contacts (the whole of) interval #3, and so on for all 360 sectors. I think a paradox arose in people's minds because they could perfectly well visualize the matching up of 1 cm outer circumference sectors with 1 cm track intervals as the wheel rolls, but they had a hard time understanding how the 0.5 cm inner circumference sectors could match up with the whole of the 1 cm track intervals in a smooth and continuous way. After having thought through the problem in detail I personally don't see any difficulty, but I think I do understand people's initial discomfort. Now if you agree with my framing of the paradox -- and I do believe it's true to the original -- how does drawing cycloids and curtate cycloids help resolve it? The paradox is concerned with what's going on in at the points of contact of the two wheels with their surfaces. If we put red dots in sector #1 on both circles, the cycloid and curtate cycloid show the movement of the red dots, but most of that movement takes place far from the contact point. (The red dots come back to the surface only once per revolution.) To understand the paradox don't you have to examine carefully how things are moving at the point of contact? This involves a succession of different sectors, not one single sector. Also I don't understand how you can go from "the red dot on the inner circle travels a more direct path through space than does the red dot on the outer circle" to "the inner circle travels a more direct path than the outer circle". I don't see the path of the circle and the path of a point on the circumference as the same thing. But the more important point is that the paradox is concerned with matching, not paths. It's concerned with the matching between circumference intervals and track intervals.

The bigger circle is taking the snaller one for a ride. Its being carried forward at a faster rate than it would if it was driving the rotation.

Aristotle musta been drinking the day he got stumped on this. The inner circle is attached to the outer circle that's why.

The inner circle douse not travel the same distance. Note the inner circles distance of travel is it’s own distance Plus both the outside distance both Sides to the out side circle.distance. Measure both the inside circle and out side circle you will See that I am right Simple Yes!

Brilliant!

stopped at 1:05 Because it doesn't travel the same distance on the same plane. The inner ring never touches the same surface as the outer ring. It is traveling along an imaginary plane above the table. Oh yeah, and the inner circle is traveling faster along the imaginary plane than the outer circle is along the table top. Edit: I see what you;re saying but I think my explanation is better lol