KVM: Option to deploy a VM with existing volume/snapshot

[slavkap]

Description

This PR allows the users to deploy virtual machines with an existing volume or a snapshot.
For now it is limited only for KVM hypervisors and Zone-wide storage pools

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)
• build/CI
• test (unit or integration test code)

Feature/Enhancement Scale

• Major
• Minor

How Has This Been Tested?

Environment
OS - Rocky Linux 8.9
2 Zones with one agent host on each
hypervisor KVM

Tested manually and with smoke tests with StorPool and NFS as a primary storage

[codecov]

Codecov Report

Attention: Patch coverage is 24.21875% with 97 lines in your changes missing coverage. Please review.

Project coverage is 16.43%. Comparing base (2df1ac5) to head (eb5af6b).

Files with missing lines	Patch %	Lines
.../src/main/java/com/cloud/vm/UserVmManagerImpl.java	36.48%	39 Missing and 8 partials ⚠️
...stack/engine/orchestration/VolumeOrchestrator.java	0.00%	27 Missing ⚠️
...he/cloudstack/api/command/user/vm/DeployVMCmd.java	27.27%	8 Missing ⚠️
...n/java/com/cloud/vm/VirtualMachineManagerImpl.java	0.00%	5 Missing ⚠️
...dstack/engine/orchestration/CloudOrchestrator.java	0.00%	3 Missing ⚠️
...a/com/cloud/network/lb/LoadBalanceRuleHandler.java	0.00%	1 Missing ⚠️
...etwork/contrail/management/ServiceManagerImpl.java	0.00%	1 Missing ⚠️
...a/com/cloud/network/vm/NetScalerVMManagerImpl.java	0.00%	1 Missing ⚠️
...om/cloud/consoleproxy/ConsoleProxyManagerImpl.java	0.00%	1 Missing ⚠️
...va/com/cloud/network/router/NetworkHelperImpl.java	0.00%	1 Missing ⚠️
... and 2 more

Additional details and impacted files

@@            Coverage Diff            @@
##               main   #10503   +/-   ##
=========================================
  Coverage     16.42%   16.43%           
- Complexity    13549    13562   +13     
=========================================
  Files          5673     5673           
  Lines        500080   500169   +89     
  Branches      60506    60529   +23     
=========================================
+ Hits          82153    82186   +33     
- Misses       408794   408839   +45     
- Partials       9133     9144   +11     

Flag	Coverage Δ
uitests	4.00% <ø> (ø)
unittests	17.29% <24.21%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12681

[DaanHoogland]

@blueorangutan test

[blueorangutan]

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-12598)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 63251 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10503-t12598-kvm-ol8.zip
Smoke tests completed. 140 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup	Error	1520.36	test_network.py
test_01_deploy_vm_with_existing_volume	Error	7.34	test_vm_lifecycle_with_snapshot_or_volume.py
test_02_deploy_vm_with_existing_snapshot	Error	0.06	test_vm_lifecycle_with_snapshot_or_volume.py
test_03_deploy_vm_with_existing_volume_deleted_template	Failure	861.05	test_vm_lifecycle_with_snapshot_or_volume.py
test_04_deploy_vm_with_existing_snapshot_deleted_template	Failure	862.76	test_vm_lifecycle_with_snapshot_or_volume.py
test_05_deploy_vm_with_existing_snapshot_deleted_volume	Failure	847.20	test_vm_lifecycle_with_snapshot_or_volume.py

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12727

[DaanHoogland]

@blueorangutan test

[blueorangutan]

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-12645)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 61921 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10503-t12645-kvm-ol8.zip
Smoke tests completed. 140 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup	Error	1520.50	test_network.py
test_01_deploy_vm_with_existing_volume	Error	0.00	test_vm_lifecycle_with_snapshot_or_volume.py
test_02_deploy_vm_with_existing_snapshot	Error	0.00	test_vm_lifecycle_with_snapshot_or_volume.py
test_03_deploy_vm_with_existing_volume_deleted_template	Error	0.00	test_vm_lifecycle_with_snapshot_or_volume.py
test_04_deploy_vm_with_existing_snapshot_deleted_template	Error	0.00	test_vm_lifecycle_with_snapshot_or_volume.py
test_05_deploy_vm_with_existing_snapshot_deleted_volume	Error	0.00	test_vm_lifecycle_with_snapshot_or_volume.py

[DaanHoogland]

@slavkap the tests fail because the self doesn't have a snapshot that was supposed to be created in the class setup method. Is this a timing issue?

[slavkap]

@blueorangutan package

[blueorangutan]

@slavkap a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12756

[slavkap]

@blueorangutan test

[github-actions]

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

[DaanHoogland]

@blueorangutan package

[blueorangutan]

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13223

[DaanHoogland]

@blueorangutan test

[blueorangutan]

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[blueorangutan]

[SF] Trillian test result (tid-13186)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 54496 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10503-t13186-kvm-ol8.zip
Smoke tests completed. 141 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_02_restore_vm_strict_tags_failure	Failure	60.49	test_vm_strict_host_tags.py
test_02_scale_vm_strict_tags_failure	Failure	58.64	test_vm_strict_host_tags.py
test_06_deploy_vm_on_any_host_with_strict_tags_failure	Failure	5.67	test_vm_strict_host_tags.py