Hi, thanks so much for the video. It helped me understand where I was going wrong with cloud-init. To help others, here's some additional info on understanding the proxmox "template" ideal workflow that I've found. Create the main image like it was described in this tutorial. Clone the main image - THEN convert the CLONE to a template. You can rename the clone/template to the date or use a versioning system e.g. DEB10-2021-v1.3. The main image still stays shutdown but you can update it when required. The best use case for this is when you need to be consistent or follow change management in your business. Because the template cannot be modified, so we can be confident it's the same as authorised. This works for linux but also especially well for Windows - after configuring the windows VM how you like, shutdown, clone it, THEN sysprep etc. - shutdown the cloned and sysprepped VM, then convert to proxmox template. This ensures the sysprepped machine cannot accidentally be started and have to be re-done. Also the same change management and versioning comments apply.
I was planning to do one, but unfortunately, Windows cloud init is very underdocumented, and some of the information doesn't even make sense. I'll keep an eye on when it gets better (or will write my own module).
So, if one did not want to run Qubes, it sounds like this would be a good alternative when you want to spin up a new temporary VM for Banking or other secure online tasks? Once you were done, you could delete the temporary VM. Thanks for the video.
Yes, that's an interesting use case, but in that instance you are better off using the console command via ssh to clone the VM, would be much faster that way. Look at the: qm clone ProxMox docs :)
@@GatewayITTutorials Thanks very much for the reply. I'm just now spinning up on XCP-ng and planned to get to ProxMox down the road, but I may have to rethink my strategy. ProxMox seems to have some attractive features.
Thank you so much!! its nice to have someone calmly explain the topic. I am a bit tired of those overly excited RUclipsrs that constantly tell you you NEEDDDD THIS RIGHT NOW!!!!..... Your videos are very informative and coherent. Keep up.
gracias maestro! even for me (not a native english speaker) it was really easy to understand you, muchas gracias! just a question: what about the machine-id? there is no problem if it is the same for all cloned VMs?
Thanks for the tutorial. I am trying to do the same with ubuntu 20.04. I don't get any errors, but when I start the VM or VM cloned from the image, it makes no difference. For instance, my original VM had a fixed IP but if I ask cloud-init to use dhcp, I still get the originally fixed IP, the hostname does not change and my new user is not created. Any ideas what I might be doing wrong, or does this method not work with Ubuntu?
@@GatewayITTutorials I used ubuntu-20.04.2-live-server-amd64.iso however I am not the only one having the issue, a lot of people have similar issues with ubuntu apparently. I think ubuntu where the "inventors" of cloud-init, the distro comes with it pre-installed and I think they have set it up to work specifically with OpenStack, lots of tweaks required to work outside that environment. I have tried with cent-os 7, it works the first time. I am going to give up with ubuntu, and for my servers, I would prefer centos anyway. I would install centos 8, but that does not seem to like being installed in a virtualized environment, on both xcp-ng and proxmox the installer fails to see the installation media. I tried local storage and NFS for the ISO storage neither work, but that is another story. Thanks for getting back to me.
I think it's better to use cloud images as outlined in the proxmox wiki as opposed to installing a vm from scratch to be used as a clone base. The cloud images support cloud-init out of the box and is perfect to be used as a clone base (be sure to install qemu-guest-agent to the base). I agree with your point about keeping them as a vm and I also usually keep mine as a live vm instead of converting it to a template, that way I can make changes to the vm or update the base with further things such as the swap config).
Fair point on the cloud images, but I learned so much from installing the VMs from scratch and converting them to CloudInit ready (like force CloudInit re-initialisation while VM is still up and running, etc), and I wouldn't even bother with it otherwise, because all the work was done for me beforehand. That's why I showed the installation/preparation part in my video.
I cannot leave here without liking and subbing. Your presentation was stunning in its attention to detail without "music" and droning on and on in areas like a lot of tutorials. I am looking forward to using you as a resource when I just want the facts, not the fluff. Excellently done! As your countrymen would say, "Just brilliant!".
Great vid. Only issue is have is after clone, the new VM has same machine id and my DHCP server(from pfsense) for some reason allocates static IP based on machine id (it seems) so it causes IP conflict. So I have to regenerate machine id manually every time I clone. Is there some better way of doing this?
I have a few queries. 1. Is it possible to create Windows [ Server or Desktop ] a cloud init process. If yes, please do a separate video on it. 2. Is it possible to have a nfs storage for cloud init storage. Can I select custom storage while cloning cloudinit vm ?
1. Cloud Init is supported by Windows. It's a good idea to make video about the subject, thanks for suggestion. 2. Yes, it is possible to clone to a different storage pool.
@@GatewayITTutorials With little bit searching on internet, I tried cloud-init for windows. Except IP , nothing can be injected [ user / password ] .... Please look into it. [ although I will try from my end as well and if successful I will have one more comment ]
Can I ask what the reasoning is behind the swap configuration changes you made? [Update for anyone reading: he also explains this during the "install debian" section of the video"
@@GatewayITTutorials thanks I really appreciate your reply. I actually went back and watched the debian installation, which I originally skipped because I know how to install debian, and found you explained it there. Thanks again
i have a question. when deploying a clone what is the default username/password? I understand regenerating the image but I would like to first know the creds it assigns. I am using ubuntu 20.04 cloudinit image. Thank you for this tutorial!
Is there anyway to stop creating default username with Cloud-init ? For example, I only want to to have root user on newly created VM, but with cloud-init we have to set default username on the system.
My go to solution is Ansible playbook: it will make sure that user Debian is absent on the system. If you don't have Ansible in your arsenal, just create a small bash script: If user Debian exist, delete it. Leave it in /root/bin or /usr/local/bin, and add a cronjob that will trigger this script on every system startup. That way it won't matter, if you are resetting root password or creating a new VM, user Debian will be removed.
@@GatewayITTutorials thank you for your reply. Unfortunately I'm very new with proxmox. So if there any documentation or tutorial you know please share. It will be very helpful for me to understand the whole scenario and how this thing works
CloudInit is a little different. ProxMox and CloudInit is better used with Ansible or SALT for deployments at scale. Vargant on the other hand is targeted at a single/test deployment.
@@GatewayITTutorials I am looking for something to deploy a Data Science VM to Azure on request in order to save costs for the storage plan, because you have steady storage costs in addition to the hourly fee per usage.
Options you have (at least in my opinion): 1. Write a bash deployment script 2. Use docker-compose 3. Use Ansible playbook 4. Don't use Azure and switch to Hetzner (you will save a lot of money by doing this, the only question is if you have the opportunity to do so).
Thanks for the walkthrough. In my opinion you miss the point of templates: as you have a default setup, you don't want anyone to mess with them. Cloud-init does a pretty good job in updating packages automatically on boot - so you're safe on that side on your cloned machine.
I agree, but the problem comes in when you have to update some of the configs, or install additional software, that should be available on all future VMs. ESX has a workaround - you can transform the VM into the Template or other way around. This could be a perfect solution to this problem on the Proxmox as well.
@@GatewayITTutorials I do get your urge of modifying templates. But the point is, there are tools like Ansible that are better suited for adding packages - so all your template needs is SSH - alternatively you can make a virtual package that pulls in your packages as dependencies and bump that package it if things change...
@@Toei-Rei I understand :) I use Ansible myself, but it's counter productive to use Ansible in a very small environment (~5-7 static VMs). On the other hand use of templates/golden images might also be counter productive in such cases. Anyway, I guess everyone looks at this issue from a different angle :)
Thank you for the tutorial. Just wanted to mention that root account is not disabled. Could be, that I first used root and then changed cloud init to user. Root can still be used to log in, to ssh in etc. Cloned template, root is still usable, although in cloud init root is not used. Also, you forgot to reset machine-id. For those who come later, check this video also ruclips.net/video/8qwnXd1yRK4/видео.html :)
@@GatewayITTutorials Just checked. If machine-id is not removed from base machine, it will propagate via cloning to clones. Cloud init is not resetting this. I have yet to change my base machine. So my base machine and clone had both the same machine-id. If I booted up clone, and original was not running, clone received the same ip via dhcp as the original from the router. This is not the problem with static ip addresses I guess.
@@GatewayITTutorials I did not. But when I'm checking master, first clone and second clone, they all have different MAC addresses. Cloning seems to be taking care of that. AFAIK, it is common issue with debian derivatives serverfault.com/questions/941891/dhcp-conflicts-on-two-ubuntu-18-04-servers
![Yailin La Mas Viral, @kreizyk.official - Dale 2 (Remix) [Video Oficial]](http://i.ytimg.com/vi/JaYHBzgqz94/mqdefault.jpg)
Just wanted to thank you for adding time stamps after I asked a little while ago!😁
No, thank you) I now understand why it's important)
Thanks for the advice)
Hi, thanks so much for the video. It helped me understand where I was going wrong with cloud-init.
To help others, here's some additional info on understanding the proxmox "template" ideal workflow that I've found. Create the main image like it was described in this tutorial. Clone the main image - THEN convert the CLONE to a template. You can rename the clone/template to the date or use a versioning system e.g. DEB10-2021-v1.3. The main image still stays shutdown but you can update it when required. The best use case for this is when you need to be consistent or follow change management in your business. Because the template cannot be modified, so we can be confident it's the same as authorised. This works for linux but also especially well for Windows - after configuring the windows VM how you like, shutdown, clone it, THEN sysprep etc. - shutdown the cloned and sysprepped VM, then convert to proxmox template. This ensures the sysprepped machine cannot accidentally be started and have to be re-done. Also the same change management and versioning comments apply.
Hi is it possible to add a tutorial for Windows cloud init image THX!! By the way Great Job , I like your tutorials !!
I was planning to do one, but unfortunately, Windows cloud init is very underdocumented, and some of the information doesn't even make sense.
I'll keep an eye on when it gets better (or will write my own module).
Great work! Thanks for this video.
Good stuff, Sir.
So, if one did not want to run Qubes, it sounds like this would be a good alternative when you want to spin up a new temporary VM for Banking or other secure online tasks? Once you were done, you could delete the temporary VM. Thanks for the video.
Yes, that's an interesting use case, but in that instance you are better off using the console command via ssh to clone the VM, would be much faster that way.
Look at the:
qm clone
ProxMox docs :)
@@GatewayITTutorials Thanks very much for the reply. I'm just now spinning up on XCP-ng and planned to get to ProxMox down the road, but I may have to rethink my strategy. ProxMox seems to have some attractive features.
Not gonna lie. Figlet makes my top list as well
Thank you so much!! its nice to have someone calmly explain the topic. I am a bit tired of those overly excited RUclipsrs that constantly tell you you NEEDDDD THIS RIGHT NOW!!!!.....
Your videos are very informative and coherent.
Keep up.
gracias maestro!
even for me (not a native english speaker) it was really easy to understand you,
muchas gracias!
just a question: what about the machine-id? there is no problem if it is the same for all cloned VMs?
Thanks, appreciate you liked the video.
Machine ID isn't an issue if you are using static IPs.
Hello, How to download the cloud-init ready image?
Info on the subject:
pve.proxmox.com/wiki/Cloud-Init_Support
@@GatewayITTutorials only Ubuntu is available
Thanks for the tutorial. I am trying to do the same with ubuntu 20.04. I don't get any errors, but when I start the VM or VM cloned from the image, it makes no difference. For instance, my original VM had a fixed IP but if I ask cloud-init to use dhcp, I still get the originally fixed IP, the hostname does not change and my new user is not created. Any ideas what I might be doing wrong, or does this method not work with Ubuntu?
Did you try using original IMG cloud image from Ubuntu website? Maybe cloudinit wasn't installed properly on your template VM.
@@GatewayITTutorials I used ubuntu-20.04.2-live-server-amd64.iso however I am not the only one having the issue, a lot of people have similar issues with ubuntu apparently. I think ubuntu where the "inventors" of cloud-init, the distro comes with it pre-installed and I think they have set it up to work specifically with OpenStack, lots of tweaks required to work outside that environment. I have tried with cent-os 7, it works the first time. I am going to give up with ubuntu, and for my servers, I would prefer centos anyway. I would install centos 8, but that does not seem to like being installed in a virtualized environment, on both xcp-ng and proxmox the installer fails to see the installation media. I tried local storage and NFS for the ISO storage neither work, but that is another story. Thanks for getting back to me.
I think it's better to use cloud images as outlined in the proxmox wiki as opposed to installing a vm from scratch to be used as a clone base. The cloud images support cloud-init out of the box and is perfect to be used as a clone base (be sure to install qemu-guest-agent to the base). I agree with your point about keeping them as a vm and I also usually keep mine as a live vm instead of converting it to a template, that way I can make changes to the vm or update the base with further things such as the swap config).
Fair point on the cloud images, but I learned so much from installing the VMs from scratch and converting them to CloudInit ready (like force CloudInit re-initialisation while VM is still up and running, etc), and I wouldn't even bother with it otherwise, because all the work was done for me beforehand.
That's why I showed the installation/preparation part in my video.
@@GatewayITTutorials it also seems like the cloud images use less ram out of the box
I cannot leave here without liking and subbing. Your presentation was stunning in its attention to detail without "music" and droning on and on in areas like a lot of tutorials. I am looking forward to using you as a resource when I just want the facts, not the fluff. Excellently done! As your countrymen would say, "Just brilliant!".
Thank you for the video it helped me out a lot
Thanks for this tutorial. It helped me a lot.
Great vid.
Only issue is have is after clone, the new VM has same machine id and my DHCP server(from pfsense) for some reason allocates static IP based on machine id (it seems) so it causes IP conflict. So I have to regenerate machine id manually every time I clone. Is there some better way of doing this?
It's a known issue for Ubuntu and Debian based VMs, I don't know if there was a fix for it tho
Would like to watch video on SDN with Proxmox....
That's an interesting topic, I'll definitely look into it
I have a few queries.
1. Is it possible to create Windows [ Server or Desktop ] a cloud init process. If yes, please do a separate video on it.
2. Is it possible to have a nfs storage for cloud init storage. Can I select custom storage while cloning cloudinit vm ?
1. Cloud Init is supported by Windows. It's a good idea to make video about the subject, thanks for suggestion.
2. Yes, it is possible to clone to a different storage pool.
@@GatewayITTutorials With little bit searching on internet, I tried cloud-init for windows. Except IP , nothing can be injected [ user / password ] .... Please look into it. [ although I will try from my end as well and if successful I will have one more comment ]
The most perfect english I ever heard. Great pleasure to listen to your speech .Thanks
Very nice of you, thanks :)
Can I ask what the reasoning is behind the swap configuration changes you made? [Update for anyone reading: he also explains this during the "install debian" section of the video"
Yes, absolutely:
There is no SWAP partition provided in cloud images (or the system I installed), so we need a SWAP file to replace such partition.
@@GatewayITTutorials thanks I really appreciate your reply. I actually went back and watched the debian installation, which I originally skipped because I know how to install debian, and found you explained it there. Thanks again
i have a question. when deploying a clone what is the default username/password? I understand regenerating the image but I would like to first know the creds it assigns. I am using ubuntu 20.04 cloudinit image.
Thank you for this tutorial!
Is there anyway to stop creating default username with Cloud-init ?
For example, I only want to to have root user on newly created VM, but with cloud-init we have to set default username on the system.
My go to solution is Ansible playbook: it will make sure that user Debian is absent on the system.
If you don't have Ansible in your arsenal, just create a small bash script:
If user Debian exist, delete it.
Leave it in /root/bin or /usr/local/bin, and add a cronjob that will trigger this script on every system startup. That way it won't matter, if you are resetting root password or creating a new VM, user Debian will be removed.
@@GatewayITTutorials thank you for your reply.
Unfortunately I'm very new with proxmox.
So if there any documentation or tutorial you know please share.
It will be very helpful for me to understand the whole scenario and how this thing works
@@GatewayITTutorials Thanks anyway, I have subscribed.. :)
Thanks :)
You will have to work on the VM template level, ProxMox unfortunately doesn't have the tools necessary to make this happen.
@@GatewayITTutorials Got it.. Thanks :)
Nice and easy, thanks man
No problem 👍
thank you for this very informative tutorial
Just a short question. How does this relate to concepts like in Vagrant?
CloudInit is a little different.
ProxMox and CloudInit is better used with Ansible or SALT for deployments at scale.
Vargant on the other hand is targeted at a single/test deployment.
@@GatewayITTutorials I am looking for something to deploy a Data Science VM to Azure on request in order to save costs for the storage plan, because you have steady storage costs in addition to the hourly fee per usage.
Options you have (at least in my opinion):
1. Write a bash deployment script
2. Use docker-compose
3. Use Ansible playbook
4. Don't use Azure and switch to Hetzner (you will save a lot of money by doing this, the only question is if you have the opportunity to do so).
@@GatewayITTutorials thank you so much. Why did you recommend Hetzner in particular? Is it a service partner?
No, they are not, it's just a very affordable service provider, offering bare metal servers, VPS VMs, and so on.
how to set up the hostname with same as the vm name dynamically
Proxmox sends VM name to Cloud Init, and then CI sets that name as a hostname. Regenerate the Cloud Init image if you need to :)
Thanks for the walkthrough. In my opinion you miss the point of templates: as you have a default setup, you don't want anyone to mess with them. Cloud-init does a pretty good job in updating packages automatically on boot - so you're safe on that side on your cloned machine.
I agree, but the problem comes in when you have to update some of the configs, or install additional software, that should be available on all future VMs. ESX has a workaround - you can transform the VM into the Template or other way around. This could be a perfect solution to this problem on the Proxmox as well.
@@GatewayITTutorials I do get your urge of modifying templates. But the point is, there are tools like Ansible that are better suited for adding packages - so all your template needs is SSH - alternatively you can make a virtual package that pulls in your packages as dependencies and bump that package it if things change...
@@Toei-Rei I understand :)
I use Ansible myself, but it's counter productive to use Ansible in a very small environment (~5-7 static VMs).
On the other hand use of templates/golden images might also be counter productive in such cases.
Anyway, I guess everyone looks at this issue from a different angle :)
Thank you for the tutorial. Just wanted to mention that root account is not disabled. Could be, that I first used root and then changed cloud init to user. Root can still be used to log in, to ssh in etc. Cloned template, root is still usable, although in cloud init root is not used. Also, you forgot to reset machine-id. For those who come later, check this video also ruclips.net/video/8qwnXd1yRK4/видео.html :)
Thanks :)
BTW, if you don't specify machine ID, ProxMox will use a random string on the backend.
@@GatewayITTutorials Just checked. If machine-id is not removed from base machine, it will propagate via cloning to clones. Cloud init is not resetting this. I have yet to change my base machine. So my base machine and clone had both the same machine-id. If I booted up clone, and original was not running, clone received the same ip via dhcp as the original from the router. This is not the problem with static ip addresses I guess.
Did you generate new a MAC address?
@@GatewayITTutorials I did not. But when I'm checking master, first clone and second clone, they all have different MAC addresses. Cloning seems to be taking care of that. AFAIK, it is common issue with debian derivatives serverfault.com/questions/941891/dhcp-conflicts-on-two-ubuntu-18-04-servers
Ahh, okay, that's good to know :)
I am using static IPs, that's why I didn't notice it.