Proxmox Troubleshooting: Common Problems and How to Fix Them

Four failures come up again and again in Proxmox VE homelab threads, and each has a specific cause:

• “No valid subscription” popup: the node has no subscription key, and the enterprise APT repository enabled by default needs one. The popup is cosmetic; the broken apt update behind it is not.
• Root filesystem full: journald, backups landing on local, or old kernels. It is almost never a mystery.
• Cluster lost quorum: corosync sharing a saturated or flapping link. /etc/pve goes read-only, and nodes running active HA services fence themselves.
• Guests will not start after an upgrade: storage that is not online yet, passthrough broken by a new kernel, or a machine type that moved under the guest.

Proxmox is a Debian host with a cluster filesystem, QEMU, and LXC bolted on. Almost every “Proxmox is broken” problem is really one of those four layers misbehaving, and the fix is usually boring once you know which layer to look at.

The subscription nag, and what it is actually telling you

Every login on a node without a subscription key throws a “No valid subscription” dialog. Annoying, but harmless. The real problem is one layer down: the pve-enterprise repository requires a key, so apt update gets an authentication failure from it. The other sources still refresh, but the run exits with an error, which is enough to break unattended-upgrade automation and to train you into ignoring apt update output entirely. A node that quietly stops receiving updates is a genuine security problem.

Fix the repository, not the dialog. In the web UI, go to the node, then Updates > Repositories. Disable the enterprise repository (on Ceph nodes there is a separate enterprise Ceph repository, disable that too) and add the no-subscription repository from the same panel. On the CLI the definitions live under /etc/apt/sources.list.d/. Recent Proxmox releases ship deb822-style .sources files instead of the old one-line .list entries, so check for both formats before concluding a repository is gone.

Then upgrade properly:

apt update
apt dist-upgrade

Use dist-upgrade (or full-upgrade), never a plain apt upgrade, on a Proxmox host. apt upgrade will pull in a new package to satisfy a dependency, but it never removes an installed one, so any upgrade that needs a package replaced or dropped gets held back instead. Kernel and pve-* transitions are exactly that shape, and holding them back leaves you with a half-upgraded stack that fails in confusing ways later. (apt-get upgrade is stricter again: it installs nothing new at all.)

No-subscription is the right default for a homelab; it just skips the extra enterprise validation pass. The test repository is for people who want to find bugs, not run workloads. Enterprise is the production answer, and the cheapest tier both removes the nag and gets you the vetted packages.

You will find one-liners that patch the nag out of proxmoxlib.js in the widget toolkit. It works. It also modifies a packaged file, gets reverted by the next proxmox-widget-toolkit update, and can break the UI if the upstream code moves. If the popup genuinely bothers you, buy a subscription. If it does not, click OK.

Root filesystem full, usually from logs

Symptoms are diffuse and misleading: task logs fail, the UI errors on actions that used to work, guests refuse to start, and /etc/pve misbehaves because the cluster filesystem cannot commit its database. Check disk space before you chase anything else.

df -h
du -xh --max-depth=1 /var | sort -h
journalctl --disk-usage
lvs

The -x on du keeps it on one filesystem so you do not wander into mounted storages and misread the result.

journald. systemd caps the journal at a fraction of the filesystem size rather than a fixed small number, which is generous on a large root and still meaningful on a small one. Set the limit explicitly in /etc/systemd/journald.conf:

SystemMaxUse=512M

Restart systemd-journald, then reclaim immediately with journalctl --vacuum-size=512M or journalctl --vacuum-time=7d. Cluster nodes generate far more journal than standalone ones, because corosync, the HA services, and storage replication all log continuously.

Backups landing on local. On a default install, local is a directory storage at /var/lib/vz sitting on the root logical volume, while local-lvm is the thin pool your guest disks live in. Point vzdump at local and every backup file lands on root. ISOs and container templates go there too. Give backups their own storage and a retention policy that actually prunes; Proxmox Backup Server handles retention and deduplication far better than a pile of dump files on the boot disk.

Old kernels. Kernel packages are large and accumulate across upgrades. Run apt autoremove --purge after upgrade cycles. On installs that boot via the EFI system partition, that partition is small and a full one breaks the next boot rather than the current one, which is a nasty way to find out. proxmox-boot-tool kernel list shows what is synced; remove unwanted kernel packages with apt, then run proxmox-boot-tool refresh.

It may not be the root filesystem at all. Check lvs for the thin pool’s Data% and Meta%. A thin pool at 100% takes guests read-only or kills them outright, and exhausted metadata is considerably harder to recover from than exhausted data. Thin provisioning lets you allocate more than you have; that is the feature and the trap. On ZFS the equivalent failure is a pool with no free space, covered in the Proxmox ZFS storage guide.

Free space first, then restart pve-cluster if /etc/pve is still stuck read-only.

Quorum loss on a cluster

Corosync gives each node a vote and requires a majority. Drop below it and the cluster filesystem turns /etc/pve read-only. Already-running guests keep running, but you cannot change a config or start anything, because those operations need to write cluster state. It goes further on a node holding active HA services: losing quorum lets the watchdog expire, and the node resets itself. People routinely misdiagnose that self-fence as a hardware fault.

Diagnose in this order:

pvecm status
corosync-cfgtool -s
journalctl -u corosync -b

pvecm status tells you expected votes, total votes, and whether the node is quorate. corosync-cfgtool -s shows per-link health, which is where flapping shows up.

Work through the likely causes in this order:

• Corosync sharing a link with bulk traffic. Corosync needs low, consistent latency, not bandwidth. A backup run or a live migration saturating the only NIC will starve it and drop the ring. Give corosync its own physical link, or at minimum a second redundant link on separate hardware so one busy path cannot take the cluster down. Getting this right at build time is the highest-value decision in cluster setup.
• Two-node clusters. Losing one node leaves the survivor with one vote out of two, which is not a majority, so a two-node cluster cannot survive a single failure on its own. The proper fix is a QDevice: a small always-on machine outside the cluster holding a tie-breaking vote, added with pvecm qdevice setup <ip>. It does not need to be powerful.
• Network changes. MTU mismatch after a switch change, a bond that flaps under load, or VLAN changes that quietly moved the corosync path. Current Proxmox uses corosync 3 with unicast by default, so multicast and IGMP snooping advice from older guides no longer applies. If you are reworking the network, start with bridges and VLANs.
• Name resolution drift. Nodes must resolve each other consistently. /etc/hosts entries left stale after an IP change break the ring in ways that look intermittent.

When you must work on a node you know is genuinely alone, pvecm expected 1 lowers expected votes on the running corosync instance so /etc/pve becomes writable again. It changes runtime state only and does not survive a corosync restart, which is a safety feature rather than a limitation. Treat it as the split-brain risk it is: only when the other nodes are confirmed down, not merely unreachable, and never on two nodes at once. If pve-cluster will not start at all, stop it and run pmxcfs -l for a local-mode writable /etc/pve, repair the config, then restart normally.

When editing corosync.conf, edit the cluster-wide copy in /etc/pve/ and increase config_version in the same edit. Skip the version bump and nodes will disagree about which config is current. If you run HA on top of this, the fencing behaviour is worth understanding before you need it, and Proxmox high availability covers what actually happens when a node drops.

Guests that will not start after an upgrade

First rule: get the real error. The GUI task summary truncates the useful part. Run it from a shell on the node:

qm start 100
pct start 101 --debug

Then match the error to one of these:

Storage is not online yet. Run pvesm status. NFS, CIFS, and iSCSI mounts race against guest autostart at boot, and a ZFS pool that failed to import leaves its storage offline. The error usually names the missing storage. Fix the mount ordering and stagger guest startup so anything depending on network storage waits for it.

PCI passthrough broken by the new kernel. A kernel upgrade brings a new driver set, and the host driver may now grab the device before vfio-pci does. The VM then fails with a device error, or starts without the hardware. Recheck that your binding and blacklist still apply after the initramfs was rebuilt, and that IOMMU grouping did not shift. GPU passthrough walks the full binding chain.

Machine type or CPU flags moved. QEMU retires old machine types and changes what CPU models expose. Pin the machine version on guests you care about so an upgrade cannot silently move them, and expect trouble with guests imported from an older cluster referencing a machine type the current QEMU no longer offers. Windows guests are the least forgiving about their virtual hardware changing underneath them.

Containers, not VMs. LXC shares the host kernel, so it is far more exposed to a host upgrade than a VM is. Containers built from very old distro templates whose init predates the current cgroup layout will not start on a modern host. AppArmor profile changes can also break nesting, FUSE, or bind mounts that previously worked, presenting as a start failure rather than anything dramatic. pct start --debug usually names the offending piece. Rebuild an affected container on a current template rather than fighting it; the tradeoffs are in LXC vs VM.

Roll the kernel back while you investigate. proxmox-boot-tool kernel pin <version> makes the node boot a known-good kernel from the next reboot onward, and it stays pinned until you unpin it. Fastest way to hand a working passthrough VM back to whoever needs it while you debug.

Prevention that actually pays off

• Record pveversion -v and your working kernel before every upgrade, so “known good” is a fact rather than a memory.
• Verify a restore, not just a green backup job. An untested backup is a hypothesis. See backup and restore strategy.
• For major version upgrades, Proxmox ships a checker script named after the transition. Run it and clear every warning before you start, not after.
• In a cluster, upgrade one node at a time and migrate guests off first. Never in parallel.
• Alert on root filesystem and thin pool usage. Both fail hard and both are trivially predictable.

Next steps

• Building a new cluster: Proxmox cluster setup guide.
• Locking down the management interface: Proxmox firewall configuration.
• Getting the base right the first time: Proxmox hardware requirements.