how would i specify my own NN architecture instead of using the basics in the config ? For example how would i define the network in pytorch and make the agent optimize that ?
Hey, firstly many thanks for your helpful tutorial. Unfortunately after playing to train I get this error: mlagents_envs.exception.UnityObservationException: Decompressed observation did not have the expected shape - decompressed had (84, 84, 3) but expected [3, 84, 84] I saw in some forum that I should update my ML-agent. But it is updated(3.0.0). Do you had this problem or maybe a solution for me?
I have always had a question about mlagents: they randomly select actions at the beginning of training. Can we incorporate human intervention into the training process of mlagents to make them train faster? Is there a corresponding method in mlagents? Looking forward to your answer.
Hey this is great, is it possible to adapt this to turn based experiments. Such as checkers for example where a single frame can be used rather than a full 30 fps. How would you adapt this to turn based games?
I think there may be some examples of how to do this in Unity’s ML-Agents repo actually. It probably shouldn’t be much different from the 30 fps version, just have it only make a decision every turn.
Hello, I am developing an agent that learns to move a box near a platform where a coin is placed. The agent must jump on the box and then on the platform to collect the coin. The problem is that in some executions, the agent learns to jump on the box and take the coin but walking backward. How could I penalize these actions so that he learns to pick up the coin walking forward? P.S. Your guide is very nice.
Very nice guide, thank you! My instinctive question as soon as I saw your camera, was that I'd try it first-person view, or a different camera where you choose the player (and any other unneeded objects and effects to be part of a layer that the camera is set to not draw. Also, the second question popping up was that maybe initial training would go faster if the platform and coin was closer to the subject, so it'd bump into them earlier in its random dance? Would it be viable to do random placement during testing, and make it move further away when the agent gets smarter?
If you add some slowdown to the jump, and add the speed it takes from spawning -> coin collected. Would it then maybe learn not to rotate this much, and also only jump when it's needed?
Is there anyway to setup two or more cameras for one agent. I'm curious if I could make a security bot that has multiple security cameras its looking through? I could always have a monitor wall that renders all of the security cameras but that seems like a round about way to get multiple camera inputs into the model. Thanks for the video, it was very helpful.
Very interesting. As a person who is new to all of this ... why would I use Camera sensors instead of Ray sensors? Because its more sensitive? Or perhaps because its more 3D?
More input data for decision making, plus a lot of people are interested in real world applications using cameras. If just designing for games, ray casts are usually easier and better performance.
Hey thank for the video. Can I have a camera not on the character but as a static entity like a roadside camera observing the environment Use the info from this camera to control the character
Great job! So far I didn't interested to train any Agent because of those PC games and its AI are not challenging for me. Now I got my mobile VR. I found all of these ML agent logic fits perfectly for "all VR interactive games and their AI". Mainly, its easy to feel how agents interacts with you while you are interacting in real time in VR.
Hi! Great tutorial :D I followed it to apply it to my game but for some reason I'm getting an error "NullReferenceException: Object reference not set to an instance of an object Unity.MLAgents.Sensors.CameraSensor.ObservationToTexture (UnityEngine.Camera obsCamera, UnityEngine.Texture2D texture2D, System.Int32 width, System.Int32 height)". I don't understand why/how it's happening and the error message is a bit useless because it doesn't say if the null reference is to the Camera object for Texture2D object... but I have linked a camera to the Camera Sensor and looking into the Unity script where ObservationToTexture is called, it creates a new Texture object in the constructor so I'm really not sure what's happening... Do you have any idea of what may be going on? Thanks in advance and keep up the great work!
@@ImmersiveLimit Hi! Thanks for the quick reply :) Problem is, the error is arising in ml-agents' code, not mine, and it seems one cannot edit their scripts (I tried adding debug statements but when I save there's a warning that the script was changed and so it reverts to a cached version instead, undoing my changes) :(
Another option: You can get the source code from the mlagents github repo, install the package manually using their instructions, and then you will be able to edit.
I was searching for this. Thank you very much for this content!! I also have a question on something. So most of the decisions have taken for a continuous action. Lets say we have to implement a something like a baseball shot, how do we get the timing right?
how would i specify my own NN architecture instead of using the basics in the config ?
For example how would i define the network in pytorch and make the agent optimize that ?
ooh exactly what I was hoping for
Thank you! I’ll try to keep including helpful info like that.
Thank you!
Nice to see the visual perception being used.
Maybe add a penalty for rotating from left to right to make the behavior more smooth.
That’s a tough one, hard not to accidentally penalize all rotations, some of which are necessary.
I think its just turning left and right because its body covers the coin. I think if the cactus was invisible it wouldn't turn so much.
Hey, firstly many thanks for your helpful tutorial. Unfortunately after playing to train I get this error:
mlagents_envs.exception.UnityObservationException: Decompressed observation did not have the expected shape - decompressed had (84, 84, 3) but expected [3, 84, 84]
I saw in some forum that I should update my ML-agent. But it is updated(3.0.0). Do you had this problem or maybe a solution for me?
I have always had a question about mlagents: they randomly select actions at the beginning of training. Can we incorporate human intervention into the training process of mlagents to make them train faster? Is there a corresponding method in mlagents? Looking forward to your answer.
Hey this is great, is it possible to adapt this to turn based experiments. Such as checkers for example where a single frame can be used rather than a full 30 fps.
How would you adapt this to turn based games?
I think there may be some examples of how to do this in Unity’s ML-Agents repo actually. It probably shouldn’t be much different from the 30 fps version, just have it only make a decision every turn.
@@ImmersiveLimit ok 🙏 thx for the reply ;)
Does camera output-> target display matter?
Hello, I am developing an agent that learns to move a box near a platform where a coin is placed. The agent must jump on the box and then on the platform to collect the coin. The problem is that in some executions, the agent learns to jump on the box and take the coin but walking backward. How could I penalize these actions so that he learns to pick up the coin walking forward? P.S. Your guide is very nice.
Nice tutorial, thanks for making it
Very nice guide, thank you! My instinctive question as soon as I saw your camera, was that I'd try it first-person view, or a different camera where you choose the player (and any other unneeded objects and effects to be part of a layer that the camera is set to not draw. Also, the second question popping up was that maybe initial training would go faster if the platform and coin was closer to the subject, so it'd bump into them earlier in its random dance? Would it be viable to do random placement during testing, and make it move further away when the agent gets smarter?
Thanks! Yes, you could try curriculum learning and gradually move the agent farther away. It probably would speed up training a bit.
Great video, it helped out a lot!
If you add some slowdown to the jump, and add the speed it takes from spawning -> coin collected. Would it then maybe learn not to rotate this much, and also only jump when it's needed?
Is there anyway to setup two or more cameras for one agent. I'm curious if I could make a security bot that has multiple security cameras its looking through? I could always have a monitor wall that renders all of the security cameras but that seems like a round about way to get multiple camera inputs into the model. Thanks for the video, it was very helpful.
Yeah, you could just put two camera sensor components on child cameras. Should work fine, but might slow down training with a lot of cameras at once.
Very interesting. As a person who is new to all of this ... why would I use Camera sensors instead of Ray sensors? Because its more sensitive? Or perhaps because its more 3D?
More input data for decision making, plus a lot of people are interested in real world applications using cameras. If just designing for games, ray casts are usually easier and better performance.
Hey thank for the video.
Can I have a camera not on the character but as a static entity like a roadside camera observing the environment
Use the info from this camera to control the character
Yes that would work
Great job! So far I didn't interested to train any Agent because of those PC games and its AI are not challenging for me.
Now I got my mobile VR. I found all of these ML agent logic fits perfectly for "all VR interactive games and their AI". Mainly, its easy to feel how agents interacts with you while you are interacting in real time in VR.
please when done post the github link here. i also wanted to get my hands on AR and VR or just give your github link so i can follow you
This is exactly what has brought me here as well!
Hi! Great tutorial :D I followed it to apply it to my game but for some reason I'm getting an error "NullReferenceException: Object reference not set to an instance of an object
Unity.MLAgents.Sensors.CameraSensor.ObservationToTexture (UnityEngine.Camera obsCamera, UnityEngine.Texture2D texture2D, System.Int32 width, System.Int32 height)". I don't understand why/how it's happening and the error message is a bit useless because it doesn't say if the null reference is to the Camera object for Texture2D object... but I have linked a camera to the Camera Sensor and looking into the Unity script where ObservationToTexture is called, it creates a new Texture object in the constructor so I'm really not sure what's happening... Do you have any idea of what may be going on? Thanks in advance and keep up the great work!
You’ll probably have to debug the code with Visual Studio and figure out what is null
@@ImmersiveLimit Hi! Thanks for the quick reply :) Problem is, the error is arising in ml-agents' code, not mine, and it seems one cannot edit their scripts (I tried adding debug statements but when I save there's a warning that the script was changed and so it reverts to a cached version instead, undoing my changes) :(
Another option: You can get the source code from the mlagents github repo, install the package manually using their instructions, and then you will be able to edit.
at 11:00 there is no errors and it's not moving because you're in heuristics mode :) (edit: seems like you figured it out at 15:00 )
exacly what i need for my next project cus rays wont cut it
I was searching for this. Thank you very much for this content!!
I also have a question on something. So most of the decisions have taken for a continuous action. Lets say we have to implement a something like a baseball shot, how do we get the timing right?
I wish I knew. I’ve never managed to get agents working on this kind of task.
Thank you for the guide video. It's really helpful (●'◡'●)
I have a question. Could I can use two cameras for ML-agent Learning???
Thanks! Yes, you should be able to hook up a second camera with a second camera sensor component
@@ImmersiveLimit Ohhhh it works!!! Thank you so much :)