Just learned how to update OVMF_VARS.fd
offline before running QEMU:
virt-fw-vars \
--input "$OUTPUT/OVMF_VARS.fd" \
--output "$OUTPUT/OVMF_VARS.fd" \
--append-boot-filepath /EFI/debian/grubx64.efi
E.g., in Debian the package containing virt-fw-vars
is python3-virt-firmware
but it is also available from PIP: https://pypi.org/project/virt-firmware/