Terraform with Proxmox VE tutorial

From Thomas-Krenn-Wiki
Jump to navigation Jump to search

This article is about the first use of Terraform with Proxmox VE. You will receive all the necessary information beginning with the installation of Terraform to the first deployment of a Proxmox resource (VM).

Overview

Once setup is complete, the basic workflow in Terraform with Proxmox VE is as follows:

  1. Install Terraform
  2. Create Proxmox API-Token: for Terraform to access Proxmox VE
  3. Create Terraform Provider: Communicates Terraform that Proxmox VE is the deployment target
  4. Create Cloud-Init-Image: For a simple and completed deployment including IP addresses
  5. Create or modify Terraform resources
  6. Terraform Init: Terraform tests the connection to the provider
  7. Terraform Plan: Terraform displays the planned changes
  8. Terraform Apply: Terraform displays/modifies the resources

Terraform installation

Terraform installation

A Debian 12 container has been created and Terraform has been installed with the following commands:[1]

apt install curl unzip &&
curl -O https://releases.hashicorp.com/terraform/1.5.3/terraform_1.5.3_linux_amd64.zip &&
unzip terraform_1.5.3_linux_amd64.zip
mv terraform /usr/local/bin

Creation of API token in PVE

First, an API token must be created in Proxmox VE. This can be made on datacenter level at Permissions -> API-Tokens -> Add. It is important that Privilege Separation is set to No for this functional test, or that you do not check the box when creating the API token. Please make a copy of the API key after you create it, as you will not be able to view it again later. In our case, this results in the following credentials:

root@pam!terraform2
b057aea1-a092-49c3-b530-a53fd9e6fccc

Terraform configuration

Creating Terraform provider

So that Terraform knows, that it should use Proxmox as provider, a provider.tf-file must be created:

touch provider.tf 
nano provider.tf
terraform {

        required_providers {
                proxmox = {
                        source = "telmate/proxmox"
                        version = "2.9.14"
                }
        }
}

variable "proxmox_api_url" {
        type = string
}

variable "proxmox_api_token_id" {
        type = string
        sensitive = true
}

variable "proxmox_api_token_secret" {
        type =  string
        sensitive = true
}

provider "proxmox" {

        pm_api_url= var.proxmox_api_url
        pm_api_token_id = var.proxmox_api_token_id
        pm_api_token_secret = var.proxmox_api_token_secret
        pm_tls_insecure = true
}

Enter API credentials

In addition, Terraform must now authenticate with Proxmox VE using the API user and API key. Here, the configuration file credentials.auto.tfvars can be used and adapted to the environment:

touch credentials.auto.tfvars
nano credentials.auto.tfvars
proxmox_api_url = "https://10.2.1.130:8006/api2/json"
proxmox_api_token_id = "root@pam!terraform2"
proxmox_api_token_secret = "b057aea1-a092-49c3-b530-a53fd9e6fccc"

Cloudinit under Proxmox VE

Since using cloud images makes sense in Terraform, here is a quick look at how to create and modify one. For this, an Ubuntu image is downloaded on the Proxmox hypervisor and the package libguestfs-tools is installed, which is required to make adjustments in the cloud image.[2] Details for the creation of Cloud-Init-Templates can be found in the article Cloud Init Templates in Proxmox VE - Quickstart

Virt-Customize

The Virt-Customize tool is quite extensive and is also able to perform the following option:

--run SCRIPT
--touch FILE
--firstboot SCRIPT
--mkdir DIR
--move SOURCE:DEST
--install PKG,PKG
--firstboot-install PKG,PKG
--append-line FILE:LINE

Overview in Proxmox VE

The templates are prepared and you can now use these templates to deploy as many VMs as you like. In the following example, there are 4 VMs that are ready to run immediately, including IP configuration, hostname, etc. To achieve this, 3 additional resource files were created with the names srv_demo_2.tf, srv_demo_3.tf and srv_demo_4.tf and adapted to our needs.

without

Terraform Workflow

Terraform Init

After this, it can be tested if Terraform can access the Proxmox host via API. For this, terraform init is used:

root@js-terraform-01:~# terraform init

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of telmate/proxmox from the dependency lock file
- Using previously-installed telmate/proxmox v2.9.14

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
root@js-terraform-01:~# 

Create Terraform resource

The goal of Terraform is, simply put, to be able to provision individual resources on various hypervisors (in this case with Proxmox VE). For this, Terraform configuration files are created, which describe what the resource to be provisioned should look like. Here is an example of the srvdemo1.tf file. We will be using a clone, so we need the VM template ubuntu2204-ci, which will later be created using a cloud image.

touch srvdemo1.tf
nano srvdemo1.tf
resource "proxmox_vm_qemu" "srv_demo_1" {
        name = "srv-demo-1"
        desc = "Ubuntu-Server"
        target_node = "PMX4"
        sshkeys = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj8ipwPHVSI/yvERzILBD52zL1jj6Ja3ptWlVhR0WK6ElBekwKL314Sps79xAitJb>
        agent = 1
        clone = "ubuntu2204-ci"
        qemu_os = "l26"
        # this l26 is a small l like linux
        cores = 2
        sockets = 1
        cpu = "host"
        memory = 8096
        scsihw = "virtio-scsi-pci"

        vga {
                type = "std"
        }

        disk {
                storage = "vm_nvme"
                type = "scsi"
                size = "83212M"
                discard = "on"
                ssd = "1"
        }

        network {
                bridge = "vmbr0"
                model = "virtio"
        }

        ## must match the template

        os_type = "cloud-init"
        ipconfig0 = "ip=dhcp"
        nameserver = "192.168.110.61"
        ciuser = "tk"
}

A VM is created, including IP configuration. In addition, a cloud image is used, which you can also customize to suit your own needs.

Terraform plan

With terraform plan , you can use Terraform to provision the VM. This means that no resources are created yet. The configuration file and the connection to the provider are tested and validated.

root@js-terraform-01:~# terraform plan
  + create

Terraform will perform the following actions:

  # proxmox_vm_qemu.srv_demo_1 will be created
  + resource "proxmox_vm_qemu" "srv_demo_1" {
      + additional_wait           = 5
      + agent                     = 1
      + automatic_reboot          = true
      + balloon                   = 0
      + bios                      = "seabios"
      + boot                      = (known after apply)
      + bootdisk                  = (known after apply)
      + ciuser                    = "tk"
      + clone                     = "ubuntu2204-ci"
      + clone_wait                = 10
      + cores                     = 2
      + cpu                       = "host"
      + default_ipv4_address      = (known after apply)
      + define_connection_info    = true
      + desc                      = "Ubuntu-Server"
      + force_create              = false
      + full_clone                = true
      + guest_agent_ready_timeout = 100
      + hotplug                   = "network,disk,usb"
      + id                        = (known after apply)
      + ipconfig0                 = "ip=dhcp"
      + kvm                       = true
      + memory                    = 8096
      + name                      = "srv-demo-1"
      + nameserver                = "192.168.110.61"
      + onboot                    = false
      + oncreate                  = true
      + os_type                   = "cloud-init"
      + preprovision              = true
      + reboot_required           = (known after apply)
      + scsihw                    = "virtio-scsi-pci"
      + searchdomain              = (known after apply)
      + sockets                   = 1
      + ssh_host                  = (known after apply)
      + ssh_port                  = (known after apply)
      + sshkeys                   = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj8ipwPHVSI/yvERzILBD52zL1jj6JaSKoN1t1ftXEKbp+cuwZPUWykvSaR13ptWlVhR0WK6ElBekwKL314Sps79xAitJbbD35yDuMVtLUK/Bo+j2ehuWWmFZMwi1UKsPgCzF/YarsX12aCIS2Gyyf2NnscgOGlQKIdcgtfO23Xz18yxQxmFuRFVbFscd7gghRoQWsoKKwldbCKS8JEbXE8Mrb5mA7XD1C4dQLMnFvoJUvx3UqZinQHc20lWjqlZIOZ0uxRz6ssVFoCn+bKKNdf43JEnwPhcxQC2vGmKWunojNsXCifdx16fvd/wegXrdL8uw2oWUAvNsjCIUYCFn+VOn5JXEAxlXCAUNN9Z9H6/64QhwTjAXibwzB8Yj2+mGqd2ODy9ZIs+nzqxsmITA8+ayaEigZngol54f6vafmQzRHrM6Zn768UEYqEGb2LgYtI/0eTClO+E0HAPVfwpRwy6T1MhmIY8otSizE3PN3pg+fcyxv9oyMnxVsYCpspzE= root@js-terraform-01>"
      + tablet                    = true
      + target_node               = "PMX4"
      + unused_disk               = (known after apply)
      + vcpus                     = 0
      + vlan                      = -1
      + vmid                      = (known after apply)

      + disk {
          + backup             = true
          + cache              = "none"
          + discard            = "on"
          + file               = (known after apply)
          + format             = (known after apply)
          + iops               = 0
          + iops_max           = 0
          + iops_max_length    = 0
          + iops_rd            = 0
          + iops_rd_max        = 0
          + iops_rd_max_length = 0
          + iops_wr            = 0
          + iops_wr_max        = 0
          + iops_wr_max_length = 0
          + iothread           = 0
          + mbps               = 0
          + mbps_rd            = 0
          + mbps_rd_max        = 0
          + mbps_wr            = 0
          + mbps_wr_max        = 0
          + media              = (known after apply)
          + replicate          = 0
          + size               = "83212M"
          + slot               = (known after apply)
          + ssd                = 1
          + storage            = "vm_nvme"
          + storage_type       = (known after apply)
          + type               = "scsi"
          + volume             = (known after apply)
        }

      + network {
          + bridge    = "vmbr0"
          + firewall  = false
          + link_down = false
          + macaddr   = (known after apply)
          + model     = "virtio"
          + queues    = (known after apply)
          + rate      = (known after apply)
          + tag       = -1
        }

      + vga {
          + type = "std"
        }
    }

Terraform Apply

You can then create the resources using terraform apply . Confirm with yes, and then (in this case, 4 VMs) will be deployed. The output has been shortened for clarity, so that only the complete output of srv_demo_1 is displayed:

root@js-terraform-01:~# terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # proxmox_vm_qemu.srv_demo_1 will be created
  + resource "proxmox_vm_qemu" "srv_demo_1" {
      + additional_wait           = 5
      + agent                     = 1
      + automatic_reboot          = true
      + balloon                   = 0
      + bios                      = "seabios"
      + boot                      = (known after apply)
      + bootdisk                  = (known after apply)
      + ciuser                    = "tk"
      + clone                     = "ubuntu2204-ci"
      + clone_wait                = 10
      + cores                     = 2
      + cpu                       = "host"
      + default_ipv4_address      = (known after apply)
      + define_connection_info    = true
      + desc                      = "Ubuntu-Server"
      + force_create              = false
      + full_clone                = true
      + guest_agent_ready_timeout = 100
      + hotplug                   = "network,disk,usb"
      + id                        = (known after apply)
      + ipconfig0                 = "ip=dhcp"
      + kvm                       = true
      + memory                    = 8096
      + name                      = "srv-demo-1"
      + nameserver                = "192.168.110.61"
      + onboot                    = false
      + oncreate                  = true
      + os_type                   = "cloud-init"
      + preprovision              = true
      + reboot_required           = (known after apply)
      + scsihw                    = "virtio-scsi-pci"
      + searchdomain              = (known after apply)
      + sockets                   = 1
      + ssh_host                  = (known after apply)
      + ssh_port                  = (known after apply)
      + sshkeys                   = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj8ipwPHVSI/yvERzILBD52zL1jj6JaSKoN1t1ftXEKbp+cuwZPUWykvSaR13ptWlVhR0WK6ElBekwKL314Sps79xAitJbbD35yDuMVtLUK/Bo+j2ehuWWmFZMwi1UKsPgCzF/YarsX12aCIS2Gyyf2NnscgOGlQKIdcgtfO23Xz18yxQxmFuRFVbFscd7gghRoQWsoKKwldbCKS8JEbXE8Mrb5mA7XD1C4dQLMnFvoJUv= root@js-terraform-01>"
      + tablet                    = true
      + target_node               = "PMX4"
      + unused_disk               = (known after apply)
      + vcpus                     = 0
      + vlan                      = -1
      + vmid                      = (known after apply)

      + disk {
          + backup             = true
          + cache              = "none"
          + discard            = "on"
          + file               = (known after apply)
          + format             = (known after apply)
          + iops               = 0
          + iops_max           = 0
          + iops_max_length    = 0
          + iops_rd            = 0
          + iops_rd_max        = 0
          + iops_rd_max_length = 0
          + iops_wr            = 0
          + iops_wr_max        = 0
          + iops_wr_max_length = 0
          + iothread           = 0
          + mbps               = 0
          + mbps_rd            = 0
          + mbps_rd_max        = 0
          + mbps_wr            = 0
          + mbps_wr_max        = 0
          + media              = (known after apply)
          + replicate          = 0
          + size               = "83212M"
          + slot               = (known after apply)
          + ssd                = 1
          + storage            = "vm_nvme"
          + storage_type       = (known after apply)
          + type               = "scsi"
          + volume             = (known after apply)
        }

      + network {
          + bridge    = "vmbr0"
          + firewall  = false
          + link_down = false
          + macaddr   = (known after apply)
          + model     = "virtio"
          + queues    = (known after apply)
          + rate      = (known after apply)
          + tag       = -1
        }

      + vga {
          + type = "std"
        }
    }


  # proxmox_vm_qemu.srv_demo_2 will be created
  # proxmox_vm_qemu.srv_demo_3 will be created
  # proxmox_vm_qemu.srv_demo_4 will be created
 
Plan: 4 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

proxmox_vm_qemu.srv_demo_4: Creating...
proxmox_vm_qemu.srv_demo_2: Creating...
proxmox_vm_qemu.srv_demo_3: Creating...
proxmox_vm_qemu.srv_demo_1: Creating...
...
proxmox_vm_qemu.srv_demo_4: Creation complete after 39s [id=PMX4/qemu/103]
proxmox_vm_qemu.srv_demo_3: Creation complete after 47s [id=PMX4/qemu/101]
proxmox_vm_qemu.srv_demo_1: Creation complete after 49s [id=PMX4/qemu/102]
proxmox_vm_qemu.srv_demo_2: Creation complete after 53s [id=PMX4/qemu/104]

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

Terraform allocates Proxmox VMs

More information

References

  1. Terraform Installation (developer.hashicorp.com)
  2. virt-customize - Customize a virtual machine (libguestfs.org)


Author: Jonas Sterr

Jonas Sterr has been working for Thomas-Krenn for several years. Originally employed as a trainee in technical support and then in hosting (formerly Filoo), Mr. Sterr now mainly deals with the topics of storage (SDS / Huawei / Netapp), virtualization (VMware, Proxmox, HyperV) and network (switches, firewalls) in product management at Thomas-Krenn.AG in Freyung.


Translator: Alina Ranzinger

Alina has been working at Thomas-Krenn.AG since 2024. After her training as multilingual business assistant, she got her job as assistant of the Product Management and is responsible for the translation of texts and for the organisation of the department.


Related articles

Cloud Init Templates in Proxmox VE - Quickstart
Installation of N8n
Proxmox Remote Migration