{"id":14168,"date":"2023-02-03T14:16:27","date_gmt":"2023-02-03T19:16:27","guid":{"rendered":"https:\/\/www.opticality.com\/blog\/?p=14168"},"modified":"2023-02-03T14:16:29","modified_gmt":"2023-02-03T19:16:29","slug":"proxmox-vs-virt-manager","status":"publish","type":"post","link":"https:\/\/opticality.com\/blog\/2023\/02\/03\/proxmox-vs-virt-manager\/","title":{"rendered":"Proxmox vs virt-manager"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">I mentioned in my first post this year that I would cover many (if not most) of the technology projects that I&#8217;m working on in 2023. I&#8217;m well into it, and I haven&#8217;t posted about any of them (yet). One of the reasons is that I&#8217;m so deeply absorbed (in the best sense) that I haven&#8217;t wanted to \u201cslow down\u201d to blog about them.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">But, of course, if I don&#8217;t post when they&#8217;re fresh, I&#8217;m not going to get the important details right, and that&#8217;s the point of blogging.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This post is out of order, since I had a number of successes (and frustrations, but no failures yet!) in my 2023 technology odyssey, and I&#8217;m choosing to start with this, which was a number of projects in (but related to many of them). Since this one is <em>over<\/em>, I&#8217;m posting it first.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">So, a number of the projects I&#8217;m working on (or have in mind) will be running in a VM (each might be in separate VMs). I wanted to run them in a separate physical host, but in my home (technically known as a \u201chome lab\u201d). I bought a machine to dedicate to that (another post coming on that at some point).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I decided that I would install <a href=\"https:\/\/proxmox.com\/en\/\">Proxmox<\/a> as the base OS. People call it a <em>bare metal<\/em> hypervisor, but as I&#8217;ve learned (I too didn&#8217;t know) that it really runs Debian beneath the surface of Proxmox. I read a ton about Proxmox, Xen, XCP-NG, KVM (virt-manager\/virsh and cockpit). On my laptop, I have run VirtualBox for over a dozen years, and even though I&#8217;m perfectly comfortable with it, I knew it wouldn&#8217;t be my \u201cremote\u201d virtualization server of choice.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I had no trouble installing Proxmox and getting it up and running. I was annoyed (mildly) by all of the nagging to sign up for a paid subscription, given that I&#8217;m running it in a home lab and it&#8217;s an open source project. An occasional <em>gentle reminder<\/em> that a support subscription is <em>available<\/em> would be fine, but not what they were doing. That&#8217;s not the reason I&#8217;m no longer running Proxmox (oh wait, did I need to say <strong>spoiler alert<\/strong> before writing that?).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Proxmox is reasonably attractive and reasonably easy to figure out (with tons of documentation and forum posts to help you if you can&#8217;t figure it out easily). That&#8217;s all good. Except when it isn&#8217;t\u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Under the covers, Proxmox runs KVM (just like Xen and XCP-NG). I had a running VM that I wanted to migrate over to Proxmox (supposedly a very easy task). That VM was a <a href=\"https:\/\/www.home-assistant.io\/\">Home Assistant OS<\/a> install. It was in qcow2 format (the native format of <a href=\"https:\/\/www.qemu.org\/\">QEMU<\/a> disks). I was under the impression (apparently incorrectly!) that qemu was the default file format and virtualization engine for all KVM-based systems. I figured I&#8217;d merely have to copy over the qcow2 image and \u201cload\u201d it into Proxmox.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Nope! I had to convert the qcow2 image into a <strong>raw<\/strong> image (easy enough) and then load it in. It loaded fine.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">But did it? Sure, it was correctly loaded into the new VM, but it wouldn&#8217;t boot. I spent the next two days (possibly three!), trying every possible combination of things that you could tweak in Proxmox. SeaBIOS (legacy bios boot) and UEFI boot. I booted (successfully) off of CD-ROM ISO images with various bootloaders on them (SuperGRUB, etc.) and also systemrescucd (and some of its kin). Those all booted successfully, so the issue wasn&#8217;t the base Proxmox install.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Once those CDs were live, I was able to mount the Home Assistant (HA) drive. I tried doing various low-level things to it (updating its grub config among other things). Each time I reverted to booting off of the HA disk, it failed to boot (though after updating grub, I got a little further, as in I was dropped to a grub shell!).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">All the while, I was Googling my brains out (almost literally). I followed every piece of advice that other people were getting when they had VMs that weren&#8217;t booting. Nothing even came close to working.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I had to take a breath and consider why I chose Proxmox to begin with. There were two reasons:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>It felt like I would have to reboot the underlying system less often than if I installed KVM\/QEMU on top of a <em>regular<\/em> Linux distribution (I&#8217;ll explain below).<\/li>\n\n\n\n<li>While it was less complicated to administer than XCP-NG (from my research), it still had high end capabilities like migrating a running VM.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">When I spent some time thinking about it, my need for migrating a running VM (and some of the other cool features) were just not things I needed, or was even likely to ever use. I&#8217;m not load balancing a big cluster here, just running a few local VMs for specific purposes (the first being Home Automation via HA).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Now the explanation from <a rel=\"tag\" class=\"hashtag u-tag u-category\" href=\"https:\/\/opticality.com\/blog\/tag\/1\/\">#1<\/a> above. On my cloud servers I run Ubuntu (because that&#8217;s the default install for most of those providers and I&#8217;m familiar enough with administering it to be 100% comfortable accepting that as a default.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">However, on my home machines (laptop and many servers, both physical and VM) I run Arch Linux (my long-time preferred OS, for 11 years now). Arch is a <strong>rolling release<\/strong> OS (meaning it is constantly updated, there is no <em>big bang<\/em> upgrade on some regular cycle.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">As such, there could be a kernel update every single day (for example). I tend to run my installs on a <strong>bleeding edge<\/strong> basis, accepting all the updates and rebooting whenever a new kernel or systemd update are applied. That&#8217;s more reboots than a normal Linux sysadmin would accept, but hey, these are all in my home, and I&#8217;m not running a business on them\u2026<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Back to our saga, I thought Proxmox would at least mean that the machine running multiple VMs wouldn&#8217;t have to reboot as often, leaving the VMs to be individually restarted only when <strong>they<\/strong> had their own necessary update installed.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">After those <strong>days<\/strong> of frustration, I decided to revert to <strong>what I know<\/strong> personally. I wiped Proxmox off the machine and installed Arch Linux (ah, a breath of <strong>very fresh air<\/strong>). I installed the full virtualizaton suite of KVM\/QEMU on top of that.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Then I fired up virt-manager and imported the qcow2 image of HA and voila, it booted on the first try.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Since then, I&#8217;ve been able to do a number of extremely cool things (all of which would have worked fine on Proxmox as well!), but that&#8217;s a topic for another day.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I want to be super clear that I think Proxmox is a very cool project and I would have enjoyed using it. It might be my one very specific case that failed (though I also tried to install HA from scratch from an ISO and I failed at that as well!). I know from reading the forums that many people are running HA under Proxmox, so it could simply be me being stupid (certainly a distinct possibility), but I&#8217;m happy I&#8217;m back in the land of the familiar.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I mentioned in my first post this year that I would cover many (if not most) of the technology projects that I&#8217;m working on in 2023. I&#8217;m well into it, and I haven&#8217;t posted about any of them (yet). One of the reasons is that I&#8217;m so deeply absorbed (in the best sense) that I [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":4,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":""},"categories":[4,3,294],"tags":[1384,1383,1373],"class_list":["post-14168","post","type-post","status-publish","format-standard","hentry","category-4","category-3","category-technology","tag-kvm","tag-proxmox","tag-technology"],"_links":{"self":[{"href":"https:\/\/opticality.com\/blog\/wp-json\/wp\/v2\/posts\/14168","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/opticality.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/opticality.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/opticality.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/opticality.com\/blog\/wp-json\/wp\/v2\/comments?post=14168"}],"version-history":[{"count":1,"href":"https:\/\/opticality.com\/blog\/wp-json\/wp\/v2\/posts\/14168\/revisions"}],"predecessor-version":[{"id":14169,"href":"https:\/\/opticality.com\/blog\/wp-json\/wp\/v2\/posts\/14168\/revisions\/14169"}],"wp:attachment":[{"href":"https:\/\/opticality.com\/blog\/wp-json\/wp\/v2\/media?parent=14168"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/opticality.com\/blog\/wp-json\/wp\/v2\/categories?post=14168"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/opticality.com\/blog\/wp-json\/wp\/v2\/tags?post=14168"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}