PowerShell to the Proxmox API part 1

Proxmox Virtual Environment (PVE) is a virtualization platform that supports virtual machines (QEMU), containers, templates, snapshots, per VM firewall, web-based management interface, etc. Proxmox runs on Debian Linux, and supports local command line tools for automation. So what do I do if I want to automate Proxmox from Windows PowerShell? Use the Proxmox API of course.

I wrote this script as a proof of concept to demonstrate how to use an API in PowerShell, and specifically how to consume the Proxmox API from PowerShell.

What does this script do?

This script will return the name, status, and uptime for a specific VM in a nicely formatted table. This is an example of the output:

Name               Status  UpDays UpHours UpMinutes UpSeconds
---- ------ ------ ------- --------- ---------
highcastle--debian running 9 17 1 46

How does this script work?

This script follows this common pattern: Set up, execute, output.

Set up

If you don’t already have an API token in Proxmox, see these instructions.

For the “prepare variables” section, you’ll need to customize these values to match how your Proxmox environment is set up. $tokenUser is in the format “username@pam!tokenname” and $tokenSecret is the value assigned when initially creating the token. $vmid is the VM’s ID number. $node is the PVE node name as listed under Datacenter. $pveserver can be the node’s IP address, or hostname. $pveport is the API port, keep the default of 8006, unless you have changed this in PVE.

$url is the API URL is the path to get the virtual machine status. The API Viewer documentation helped me to get the exact path needed.

$pveheader is the Authorization header that gets sent with the API call.

Execute

This is just the one line, using the command Invoke-RestMethod. It uses all the values we prepared in the Set up region. This is the command that actually connects to the Proxmox API. The results of this command are put in the variable $result.

Output

Here is an example of the data returned by the API call:

PS C:\code> $result.data


diskread : 190837326
ha : @{managed=0}
maxdisk : 21474836480
netout : 1919738
proxmox-support : @{pbs-library-version=1.4.1 (UNKNOWN); pbs-masterkey=True; backup-max-workers=True; query-bitmap-info=True; pbs-dirty-bitmap-savevm=True; pbs-dirty-bitmap=True;
backup-fleecing=True; pbs-dirty-bitmap-migration=True}
spice : 1
maxmem : 2147483648
clipboard :
pid : 2210159
blockstat : @{scsi0=; ide2=}
diskwrite : 755953664
name : highcastle--debian
mem : 355270656
cpus : 1
freemem : 1707507712
uptime : 872718
netin : 354281367
nics : @{tap100i0=}
disk : 0
ballooninfo : @{max_mem=2147483648; free_mem=1707507712; mem_swapped_in=0; mem_swapped_out=0; total_mem=2062778368; last_update=1719800413; actual=2147483648;
minor_page_faults=807811; major_page_faults=492}
running-machine : pc-i440fx-8.1+pve0
status : running
vmid : 100
cpu : 0.00591751100250928
running-qemu : 8.1.5
qmpstatus : running
balloon : 2147483648
agent : 1

That’s a lot to sift through, isn’t it? Remember that this script will return just the name, status, and uptime. To make this script useful, and so it doesn’t overwhelm us with this wall of text, I will narrow down the output into a PS Custom Object and format the results. The VM name and status are simple to pull out of the data properties of the same names. The uptime value returned is in seconds, so to make that useful, I convert it to a TimeSpan. In the PS Custom Object, I output that TimeSpan as days, hours, minutes and seconds to make it more easily understood.

To format the PS Custom Object, it is piped into Format-Table, so it displays as a horizontal table.

The script

#region SET UP
# prepare variables
$tokenUser   = "username@pam!tokenname"               # example: "username@pam!tokenname"
$tokenSecret = "11111111-2222-3333-4444-555555555555" # example: "11111111-2222-3333-4444-555555555555"
$vmid        = "100"                                  # ID of VM
$node        = "pve"                                  # PVE node name, default is "pve"
$pveserver   = "192.168.1.123"                        # PVE node IP Address or if using SSL certificate, then use name as in certificate
$pveport     = "8006"                                 # default port is "8006"

# form URL
$url = "https://${pveserver}:$pveport/api2/json/nodes/$node/qemu/$vmid/status/current"

# set up authorization header to use token values
$pveheader = @{Authorization = "PVEAPIToken=$tokenUser=$tokenSecret"}
#endregion

#region EXCUTE
# send the API request to PVE
$result = Invoke-RestMethod -Headers $pveheader -Method Get -Uri $url
#endregion

#region OUTPUT
# format the result
$uptimeSeconds = New-TimeSpan -Seconds $result.data.uptime
[pscustomobject]@{
    Name      = $result.data.name
    Status    = $result.data.status
    UpDays    = $uptimeSeconds.Days
    UpHours   = $uptimeSeconds.Hours
    UpMinutes = $uptimeSeconds.Minutes
    UpSeconds = $uptimeSeconds.Seconds
} | Format-Table
#endregion

Conclusion

This is a demo of using PowerShell to connect to the Proxmox API. In part 2, I will refine the script and turn it into a flexible function that you can use day to day.

How to Create an API Token for Proxmox VE

“Proxmox Virtual Environment is a complete, open-source server management platform for enterprise virtualization. It tightly integrates the KVM hypervisor and Linux Containers (LXC), software-defined storage and networking functionality, on a single platform. With the integrated web-based user interface you can manage VMs and containers, high availability for clusters, or the integrated disaster recovery tools with ease.” from Proxmox.com

You can authenticate to the Proxmox API using a user/password or using an API token. An API token is useful because it let’s you fine tune permissions to the task at hand, and if you need to revoke a token, it won’t impact the user that it’s associated with. Also, I liked the simplicity of using the same authentication header with each API call. With user/password authentication, I didn’t want to manage a ticket cookie over multiple API calls.

This blog post documents how to create an API token.

  1. Go to Datacenter > Permissions > Groups
    • Click the Create button to create a new group, call it for example “provisioning”
  2. Go to Datacenter > Permissions > Users
    • Click the Create button to create a new user
    • User name: for example “provisioner”
    • Realm: “Proxmox VE authentication server” (or you can select “Linux” but you will need to set the password on the Linux command line)
    • Group: select the group you created earlier
    • Click Add
  3. Go to Datacenter > Permissions
    • This will set permissions for the group, click Add
    • Path: To allow the permissions to be applied anywhere in Proxmox, select “/”, or to refine where they get applied, select that specific path.
    • Select Role, for example “Administrator”, or a more restrictive role depending on what you intend to do.
  4. Go to Datacenter > Permissions > API Tokens, to create the actual token
    • Click Add
    • Select the User you created above
    • Token ID: Type a name for the token, for example “prov”
    • When you click Add, the Token Secret will be displayed. It will only be displayed once, at creation, so copy it to a secure location. It is a string in the form of 36 random-ish characters.

Find invalid characters using PowerShell

Have you ever copied code from a web site or an email and have it not work because some of the characters auto-corrected to weird quotes or dashes or the like? I have, and it can be a real hassle finding those characters because they look so much like the normal characters.

I wrote this module to find those characters.

From the module description:

Finds out of range encoding (ASCII, ANSI, Unicode) of characters in a file or object. The default “in range” is 32 to 126, most of the “printable” ASCII.
Each line is printed to the console, preceded by a line number. In range characters are displayed in black and white. Out of range characters are displayed in yellow with a red background. At the end of the output is a log of each out of range character; listing line number, character number, the character as it is displayed, and the encoding value.
A custom range can be set by passing an array of in-range INTs. The range does not need to be consecutive, it just needs to be an array of integers.
This module will be especially useful in checking code pasted from email or the web with might have unacceptable characters.
Note that out of range characters are not always wrong, but in the wrong spot can cause problems.

<#
.Synopsis
   Finds out of range encoding of characters in a file or object 
.DESCRIPTION
   Finds out of range encoding (ASCII, ANSI, Unicode) of characters in a file or object. The default "in range" is 32 to 126, most of the "printable" ASCII.
   Each line is printed to the console, preceded by a line number. In range characters are displayed in black and white. Out of range characters are displayed in yellow with a red background. At the end of the output is a log of each out of range character; listing line number, character number, the character as it is displayed, and the encoding value.
   A custom range can be set by passing an array of in-range INTs. The range does not need to be consecutive, it just needs to be an array of integers.
   This module will be especially useful in checking code pasted from email or the web with might have unacceptable characters. 
   Note that out of range characters are not always wrong, but in the wrong spot can cause problems.
.EXAMPLE
   Find-OutOfRangeCharacters -Filename .\fate.ps1
.EXAMPLE
   Find-OutOfRangeCharacters -Object $(Get-Content -Path .\fate.ps1) -Range $(32..96)
.EXAMPLE
   Find-OutOfRangeCharacters -Object "hello wörld"
#>
function Find-OutOfRangeCharacters
{
    [CmdletBinding()]
    Param
    (
        # File of data to be searched
        [Parameter(ParameterSetName='Filename')]
        [ValidateScript({Test-Path $_})]
        [string]
        $Filename,

        # Data to be searched
        [Parameter(ParameterSetName='Object')]
        [string[]]
        $Object,

        # INT array of in-range encoding values
        [int[]]
        $Range = 32 .. 126
    )

#region retrieve data
    if ($Object) 
    {
        $c = $Object
    }
    else
    {
        $c = Get-Content $Filename #-ErrorAction Stop
    }
#endregion

    $message = ""

    # loop through each line
    for ($i = 0; $i -lt $c.Length; $i++)
    { 
        # need to display "$i + 1" because arrays count from 0 but humans count from 1
        $PrettyLineNumber = "[" + $($i + 1).ToString("000") + "]" 
        Write-Host -Object $PrettyLineNumber  -NoNewline -ForegroundColor White -BackgroundColor Blue
        $line = $c[$i]

        # loop through each character in the line
        for ($k = 0; $k -lt $line.Length; $k++)
        {
            # get encoding value for the character
            $CharValue = [int][char]$line[$k]

            if ($CharValue -notin $Range) 
            {
                # need to display "$i + 1" and "$k + 1" because arrays count from 0 but humans count from 1
                Write-Host -Object $line[$k] -NoNewline -ForegroundColor Yellow -BackgroundColor Red
                $message += "Line # $($i + 1), character # $($k + 1), displays as `"$($line[$k])`", encoding value $CharValue `n" 
            }
            else
            {
                Write-Host -Object $line[$k] -NoNewline -ForegroundColor White -BackgroundColor Black 
            }
        }
        # write end of line
        Write-Host

    }
    # write log of out of range characters found
    Write-Output $message
}

Get a random decimal number between 0 and 1 in PowerShell

Get-Random -Maximum ([Double]1)

How does this work? The Get-Random cmdlet returns a Double (that’s a decimal number) when the Maximum parameter is a Double. If we just wrote -Maximum 1, then PowerShell would interpret the 1 as an integer. So we cast the 1 as a Double by writing it as [Double]1. Since [Double]1 is not a simple value or variable, PowerShell will misinterpret this as a parameter. So we wrap if in parentheses to get: ([Double]1). This grouping tells PowerShell to evaluate it first, then use that result as the parameter.

Minimum defaults to 0, so we don’t need to specify it.

How this helps!

vi cheatsheet

Get the PDF here.

The bare minimum vi.

How to open a new or existing file in vi

vi filename

File navigation

Use [Page Up] , [Page Down]  to page through the file. Use the arrow keys to move the cursor.

Command mode and insert mode

When you first start vi, you are in command mode

To switch to insert mode

i

Once in insert mode, you can type in your text file.

To switch from insert mode to command mode

[ESC]

Command mode

Quit without saving changes

:q!

Quit

:q

Write (save)

:w

Write a copy of the file to another name

:w filename

Search for a string (replace “search” with the string you are searching for)

:/search

Global search and replace (replace “search” and “replace” with the strings you are searching and replacing)

:%s/search/replace/

Find Duplicate Files with PowerShell

This one-liner will find duplicate files in the current directory and all sub-directories. It uses hash values of the files, so it doesn’t matter if the file names have changed. If the content is the same, the hash will be the same and it will be considered a duplicate.

# find duplicate files
# Kenward Bradley 2016-12-29
Get-ChildItem -Recurse | Get-FileHash | Group-Object -Property Hash | Where-Object Count -GT 1 | foreach {$_.Group | select Path, Hash}

PowerShell: Return Debug and Verbose messages From Your Functions

Debug and Verbose let you return extra information from your scripts, for specific situations, and only when needed. Debug lets you return “programmer-level” information. Verbose lets you return more detailed information to the user. I’ll provide some examples.

Continue reading “PowerShell: Return Debug and Verbose messages From Your Functions”

Compare services of two computers in PowerShell

This function “Compare-Services” will allow you to compare the running services of two computers. The first one is called the Reference Computer, the second is the Difference Computer. The output defaults to Compare-Object output. If you need an object to do further work with, this is what you want. If you don’t want to puzzle out what the “=>” and “<=” indicators mean, and you want a simple report, you can use the -FriendlyText switch will give you English text output, which will make more sense to many humans, at least the English speaking humans reading this blog. Windows remoting needs to be enabled on the computers being checked for this to work.

Example output:

Compare-Object style (PS object) output:

InputObject                           SideIndicator
-----------                           -------------
Distributed Link Tracking Client      =>           
Active Directory Web Services         <=           
Application Information               <=           
DFS Namespace                         <=           
DFS Replication                       <=           
DHCP Server                           <=           
DNS Server                            <=           
Intersite Messaging                   <=           
Kerberos Key Distribution Center      <=           
Active Directory Domain Services      <=           
Smart Card Device Enumeration Service <=           
Shell Hardware Detection              <=           
Virtual Disk                          <=           

Friendly Text (English) output:
"Distributed Link Tracking Client" is on lab01 (not on dc1)
"Active Directory Web Services" is on dc1 (not on lab01)
"Application Information" is on dc1 (not on lab01)
"DFS Namespace" is on dc1 (not on lab01)
"DFS Replication" is on dc1 (not on lab01)
"DHCP Server" is on dc1 (not on lab01)
"DNS Server" is on dc1 (not on lab01)
"Intersite Messaging" is on dc1 (not on lab01)
"Kerberos Key Distribution Center" is on dc1 (not on lab01)
"Active Directory Domain Services" is on dc1 (not on lab01)
"Smart Card Device Enumeration Service" is on dc1 (not on lab01)
"Shell Hardware Detection" is on dc1 (not on lab01)
"Virtual Disk" is on dc1 (not on lab01)

And here is the function:

function Compare-Services ($ReferenceComputer, $DifferenceComputer, [switch]$FriendlyText) {

$ReferenceServices = (Get-Service -ComputerName $ReferenceComputer | where Status -EQ "Running").DisplayName
$DifferenceServices = (Get-Service -ComputerName $DifferenceComputer | where Status -EQ "Running").DisplayName

$CompareResult = Compare-Object -ReferenceObject $ReferenceServices -DifferenceObject $DifferenceServices

if ($FriendlyText) {
    $CompareResult | foreach {
        if ($_.SideIndicator -eq "=>") {"`"$($_.InputObject)`" is on $DifferenceComputer (not on $ReferenceComputer)"}
        if ($_.SideIndicator -eq "<=") {"`"$($_.InputObject)`" is on $ReferenceComputer (not on $DifferenceComputer)"}
        }
    }
    else {$CompareResult}
}

 

PowerShell cmdlet to create lab users

So you have Active Directory in your lab and want to create a bunch of users for some reason? My PowerShell cmdlet “New-LabADUser” should be just what you need.

Usage examples:

PS C:\> New-LabADUser


DistinguishedName : CN=BStevens,CN=Users,DC=south,DC=lab
Enabled           : True
GivenName         : Bonnie
Name              : BStevens
ObjectClass       : user
ObjectGUID        : 33319b1b-bf93-4647-8e8d-d193be91043d
SamAccountName    : BStevens
SID               : S-1-5-21-4025744032-3205137867-863233111-1711
Surname           : Stevens
UserPrincipalName : 




PS C:\> New-LabADUser -NumberOfUsers 3 | select Name

Name   
----   
DAustin
JTaylor
WOrtiz 



PS C:\> $UsersCreated = New-LabADUser -NumberOfUsers 10 | select DistinguishedName

PS C:\> $UsersCreated

DistinguishedName                      
-----------------                      
CN=FDavis,CN=Users,DC=south,DC=lab     
CN=MDixon,CN=Users,DC=south,DC=lab     
CN=RNichols,CN=Users,DC=south,DC=lab   
CN=ABailey,CN=Users,DC=south,DC=lab    
CN=TRose,CN=Users,DC=south,DC=lab      
CN=ECarter,CN=Users,DC=south,DC=lab    
CN=TGonzales,CN=Users,DC=south,DC=lab  
CN=KWashington,CN=Users,DC=south,DC=lab
CN=JHunter,CN=Users,DC=south,DC=lab    
CN=DGray,CN=Users,DC=south,DC=lab      

Users are created with Organization = “LabUser” to make them easy to find:

PS C:\>  Get-ADUser -Filter "Organization -EQ 'LabUser'" | select Name

Name       
----       
BStevens   
DAustin    
JTaylor    
WOrtiz     
FDavis     
MDixon     
RNichols   
ABailey    
TRose      
ECarter    
TGonzales  
KWashington
JHunter    
DGray

And the cmdlet…

function New-LabADUser
 {
 [CmdletBinding()]
 Param
 (
 # Param2 help description
 [int]$NumberOfUsers = 1
 )

Begin
 {
# build the names lists
 $GivenName = ("Aaron", "Adam", "Alan", "Albert", "Alice", "Amanda", "Amy", "Andrea", "Andrew", "Angela", "Ann", "Anna", "Anne", "Annie", "Anthony", "Antonio", "Arthur", "Ashley", "Barbara", "Benjamin", "Betty", "Beverly", "Billy", "Bobby", "Bonnie", "Brandon", "Brenda", "Brian", "Bruce", "Carl", "Carlos", "Carol", "Carolyn", "Catherine", "Charles", "Cheryl", "Chris", "Christina", "Christine", "Christopher", "Clarence", "Craig", "Cynthia", "Daniel", "David", "Deborah", "Debra", "Denise", "Dennis", "Diana", "Diane", "Donald", "Donna", "Doris", "Dorothy", "Douglas", "Earl", "Edward", "Elizabeth", "Emily", "Eric", "Ernest", "Eugene", "Evelyn", "Frances", "Frank", "Fred", "Gary", "George", "Gerald", "Gloria", "Gregory", "Harold", "Harry", "Heather", "Helen", "Henry", "Howard", "Irene", "Jack", "Jacqueline", "James", "Jane", "Janet", "Janice", "Jason", "Jean", "Jeffrey", "Jennifer", "Jeremy", "Jerry", "Jesse", "Jessica", "Jimmy", "Joan", "Joe", "John", "Johnny", "Jonathan", "Jose", "Joseph", "Joshua", "Joyce", "Juan", "Judith", "Judy", "Julia", "Julie", "Justin", "Karen", "Katherine", "Kathleen", "Kathryn", "Kathy", "Keith", "Kelly", "Kenneth", "Kevin", "Kimberly", "Larry", "Laura", "Lawrence", "Lillian", "Linda", "Lisa", "Lois", "Lori", "Louis", "Louise", "Margaret", "Maria", "Marie", "Marilyn", "Mark", "Martha", "Martin", "Mary", "Matthew", "Melissa", "Michael", "Michelle", "Mildred", "Nancy", "Nicholas", "Nicole", "Norma", "Pamela", "Patricia", "Patrick", "Paul", "Paula", "Peter", "Philip", "Phillip", "Phyllis", "Rachel", "Ralph", "Randy", "Raymond", "Rebecca", "Richard", "Robert", "Robin", "Roger", "Ronald", "Rose", "Roy", "Ruby", "Russell", "Ruth", "Ryan", "Samuel", "Sandra", "Sara", "Sarah", "Scott", "Sean", "Sharon", "Shawn", "Shirley", "Stephanie", "Stephen", "Steve", "Steven", "Susan", "Tammy", "Teresa", "Terry", "Theresa", "Thomas", "Timothy", "Tina", "Todd", "Victor", "Virginia", "Walter", "Wanda", "Wayne", "William", "Willie")
 $Surame = ("Smith", "Johnson", "Williams", "Jones", "Brown", "Davis", "Miller", "Wilson", "Moore", "Taylor", "Anderson", "Thomas", "Jackson", "White", "Harris", "Martin", "Thompson", "Garcia", "Martinez", "Robinson", "Clark", "Rodriguez", "Lewis", "Lee", "Walker", "Hall", "Allen", "Young", "Hernandez", "King", "Wright", "Lopez", "Hill", "Scott", "Green", "Adams", "Baker", "Gonzalez", "Nelson", "Carter", "Mitchell", "Perez", "Roberts", "Turner", "Phillips", "Campbell", "Parker", "Evans", "Edwards", "Collins", "Stewart", "Sanchez", "Morris", "Rogers", "Reed", "Cook", "Morgan", "Bell", "Murphy", "Bailey", "Rivera", "Cooper", "Richardson", "Cox", "Howard", "Ward", "Torres", "Peterson", "Gray", "Ramirez", "James", "Watson", "Brooks", "Kelly", "Sanders", "Price", "Bennett", "Wood", "Barnes", "Ross", "Henderson", "Coleman", "Jenkins", "Perry", "Powell", "Long", "Patterson", "Hughes", "Flores", "Washington", "Butler", "Simmons", "Foster", "Gonzales", "Bryant", "Alexander", "Russell", "Griffin", "Diaz", "Hayes", "Myers", "Ford", "Hamilton", "Graham", "Sullivan", "Wallace", "Woods", "Cole", "West", "Jordan", "Owens", "Reynolds", "Fisher", "Ellis", "Harrison", "Gibson", "Mcdonald", "Cruz", "Marshall", "Ortiz", "Gomez", "Murray", "Freeman", "Wells", "Webb", "Simpson", "Stevens", "Tucker", "Porter", "Hunter", "Hicks", "Crawford", "Henry", "Boyd", "Mason", "Morales", "Kennedy", "Warren", "Dixon", "Ramos", "Reyes", "Burns", "Gordon", "Shaw", "Holmes", "Rice", "Robertson", "Hunt", "Black", "Daniels", "Palmer", "Mills", "Nichols", "Grant", "Knight", "Ferguson", "Rose", "Stone", "Hawkins", "Dunn", "Perkins", "Hudson", "Spencer", "Gardner", "Stephens", "Payne", "Pierce", "Berry", "Matthews", "Arnold", "Wagner", "Willis", "Ray", "Watkins", "Olson", "Carroll", "Duncan", "Snyder", "Hart", "Cunningham", "Bradley", "Lane", "Andrews", "Ruiz", "Harper", "Fox", "Riley", "Armstrong", "Carpenter", "Weaver", "Greene", "Lawrence", "Elliott", "Chavez", "Sims", "Austin", "Peters", "Kelley", "Franklin", "Lawson")
 }
 Process
 {
 for ($i = 1; $i -le $NumberOfUsers; $i++)
 {
 $gn = $GivenName | Get-Random
 $sn = $Surame | Get-Random
 $AccountName = $gn.Substring(0,1) + $sn

# create a password to fit most requirements
$pw = ""
 do {$pw += (33..126) | foreach {[char]$_} | Get-Random}
 until ($pw.Length -ge 16)
 $AccountPassword = ConvertTo-SecureString $pw -AsPlainText -Force

New-ADUser -AccountPassword $AccountPassword -GivenName $gn -Surname $sn -Name $AccountName -Organization "LabUser" -Enabled $true -PassThru
 }
 }

 }