Find invalid characters using PowerShell

Have you ever copied code from a web site or an email and have it not work because some of the characters auto-corrected to weird quotes or dashes or the like? I have, and it can be a real hassle finding those characters because they look so much like the normal characters.

I wrote this module to find those characters.

From the module description:

Finds out of range encoding (ASCII, ANSI, Unicode) of characters in a file or object. The default “in range” is 32 to 126, most of the “printable” ASCII.
Each line is printed to the console, preceded by a line number. In range characters are displayed in black and white. Out of range characters are displayed in yellow with a red background. At the end of the output is a log of each out of range character; listing line number, character number, the character as it is displayed, and the encoding value.
A custom range can be set by passing an array of in-range INTs. The range does not need to be consecutive, it just needs to be an array of integers.
This module will be especially useful in checking code pasted from email or the web with might have unacceptable characters.
Note that out of range characters are not always wrong, but in the wrong spot can cause problems.

<#
.Synopsis
   Finds out of range encoding of characters in a file or object 
.DESCRIPTION
   Finds out of range encoding (ASCII, ANSI, Unicode) of characters in a file or object. The default "in range" is 32 to 126, most of the "printable" ASCII.
   Each line is printed to the console, preceded by a line number. In range characters are displayed in black and white. Out of range characters are displayed in yellow with a red background. At the end of the output is a log of each out of range character; listing line number, character number, the character as it is displayed, and the encoding value.
   A custom range can be set by passing an array of in-range INTs. The range does not need to be consecutive, it just needs to be an array of integers.
   This module will be especially useful in checking code pasted from email or the web with might have unacceptable characters. 
   Note that out of range characters are not always wrong, but in the wrong spot can cause problems.
.EXAMPLE
   Find-OutOfRangeCharacters -Filename .\fate.ps1
.EXAMPLE
   Find-OutOfRangeCharacters -Object $(Get-Content -Path .\fate.ps1) -Range $(32..96)
.EXAMPLE
   Find-OutOfRangeCharacters -Object "hello wörld"
#>
function Find-OutOfRangeCharacters
{
    [CmdletBinding()]
    Param
    (
        # File of data to be searched
        [Parameter(ParameterSetName='Filename')]
        [ValidateScript({Test-Path $_})]
        [string]
        $Filename,

        # Data to be searched
        [Parameter(ParameterSetName='Object')]
        [string[]]
        $Object,

        # INT array of in-range encoding values
        [int[]]
        $Range = 32 .. 126
    )

#region retrieve data
    if ($Object) 
    {
        $c = $Object
    }
    else
    {
        $c = Get-Content $Filename #-ErrorAction Stop
    }
#endregion

    $message = ""

    # loop through each line
    for ($i = 0; $i -lt $c.Length; $i++)
    { 
        # need to display "$i + 1" because arrays count from 0 but humans count from 1
        $PrettyLineNumber = "[" + $($i + 1).ToString("000") + "]" 
        Write-Host -Object $PrettyLineNumber  -NoNewline -ForegroundColor White -BackgroundColor Blue
        $line = $c[$i]

        # loop through each character in the line
        for ($k = 0; $k -lt $line.Length; $k++)
        {
            # get encoding value for the character
            $CharValue = [int][char]$line[$k]

            if ($CharValue -notin $Range) 
            {
                # need to display "$i + 1" and "$k + 1" because arrays count from 0 but humans count from 1
                Write-Host -Object $line[$k] -NoNewline -ForegroundColor Yellow -BackgroundColor Red
                $message += "Line # $($i + 1), character # $($k + 1), displays as `"$($line[$k])`", encoding value $CharValue `n" 
            }
            else
            {
                Write-Host -Object $line[$k] -NoNewline -ForegroundColor White -BackgroundColor Black 
            }
        }
        # write end of line
        Write-Host

    }
    # write log of out of range characters found
    Write-Output $message
}

Get a random decimal number between 0 and 1 in PowerShell

Get-Random -Maximum ([Double]1)

How does this work? The Get-Random cmdlet returns a Double (that’s a decimal number) when the Maximum parameter is a Double. If we just wrote -Maximum 1, then PowerShell would interpret the 1 as an integer. So we cast the 1 as a Double by writing it as [Double]1. Since [Double]1 is not a simple value or variable, PowerShell will misinterpret this as a parameter. So we wrap if in parentheses to get: ([Double]1). This grouping tells PowerShell to evaluate it first, then use that result as the parameter.

Minimum defaults to 0, so we don’t need to specify it.

How this helps!

vi cheatsheet

Get the PDF here.

The bare minimum vi.

How to open a new or existing file in vi

vi filename

File navigation

Use [Page Up] , [Page Down]  to page through the file. Use the arrow keys to move the cursor.

Command mode and insert mode

When you first start vi, you are in command mode

To switch to insert mode

i

Once in insert mode, you can type in your text file.

To switch from insert mode to command mode

[ESC]

Command mode

Quit without saving changes

:q!

Quit

:q

Write (save)

:w

Write a copy of the file to another name

:w filename

Search for a string (replace “search” with the string you are searching for)

:/search

Global search and replace (replace “search” and “replace” with the strings you are searching and replacing)

:%s/search/replace/

Find Duplicate Files with PowerShell

This one-liner will find duplicate files in the current directory and all sub-directories. It uses hash values of the files, so it doesn’t matter if the file names have changed. If the content is the same, the hash will be the same and it will be considered a duplicate.

# find duplicate files
# Kenward Bradley 2016-12-29
Get-ChildItem -Recurse | Get-FileHash | Group-Object -Property Hash | Where-Object Count -GT 1 | foreach {$_.Group | select Path, Hash}

PowerShell: Return Debug and Verbose messages From Your Functions

Debug and Verbose let you return extra information from your scripts, for specific situations, and only when needed. Debug lets you return “programmer-level” information. Verbose lets you return more detailed information to the user. I’ll provide some examples.

Continue reading “PowerShell: Return Debug and Verbose messages From Your Functions”

Compare services of two computers in PowerShell

This function “Compare-Services” will allow you to compare the running services of two computers. The first one is called the Reference Computer, the second is the Difference Computer. The output defaults to Compare-Object output. If you need an object to do further work with, this is what you want. If you don’t want to puzzle out what the “=>” and “<=” indicators mean, and you want a simple report, you can use the -FriendlyText switch will give you English text output, which will make more sense to many humans, at least the English speaking humans reading this blog. Windows remoting needs to be enabled on the computers being checked for this to work.

Example output:

Compare-Object style (PS object) output:

InputObject                           SideIndicator
-----------                           -------------
Distributed Link Tracking Client      =>           
Active Directory Web Services         <=           
Application Information               <=           
DFS Namespace                         <=           
DFS Replication                       <=           
DHCP Server                           <=           
DNS Server                            <=           
Intersite Messaging                   <=           
Kerberos Key Distribution Center      <=           
Active Directory Domain Services      <=           
Smart Card Device Enumeration Service <=           
Shell Hardware Detection              <=           
Virtual Disk                          <=           

Friendly Text (English) output:
"Distributed Link Tracking Client" is on lab01 (not on dc1)
"Active Directory Web Services" is on dc1 (not on lab01)
"Application Information" is on dc1 (not on lab01)
"DFS Namespace" is on dc1 (not on lab01)
"DFS Replication" is on dc1 (not on lab01)
"DHCP Server" is on dc1 (not on lab01)
"DNS Server" is on dc1 (not on lab01)
"Intersite Messaging" is on dc1 (not on lab01)
"Kerberos Key Distribution Center" is on dc1 (not on lab01)
"Active Directory Domain Services" is on dc1 (not on lab01)
"Smart Card Device Enumeration Service" is on dc1 (not on lab01)
"Shell Hardware Detection" is on dc1 (not on lab01)
"Virtual Disk" is on dc1 (not on lab01)

And here is the function:

function Compare-Services ($ReferenceComputer, $DifferenceComputer, [switch]$FriendlyText) {

$ReferenceServices = (Get-Service -ComputerName $ReferenceComputer | where Status -EQ "Running").DisplayName
$DifferenceServices = (Get-Service -ComputerName $DifferenceComputer | where Status -EQ "Running").DisplayName

$CompareResult = Compare-Object -ReferenceObject $ReferenceServices -DifferenceObject $DifferenceServices

if ($FriendlyText) {
    $CompareResult | foreach {
        if ($_.SideIndicator -eq "=>") {"`"$($_.InputObject)`" is on $DifferenceComputer (not on $ReferenceComputer)"}
        if ($_.SideIndicator -eq "<=") {"`"$($_.InputObject)`" is on $ReferenceComputer (not on $DifferenceComputer)"}
        }
    }
    else {$CompareResult}
}

 

PowerShell cmdlet to create lab users

So you have Active Directory in your lab and want to create a bunch of users for some reason? My PowerShell cmdlet “New-LabADUser” should be just what you need.

Usage examples:

PS C:\> New-LabADUser


DistinguishedName : CN=BStevens,CN=Users,DC=south,DC=lab
Enabled           : True
GivenName         : Bonnie
Name              : BStevens
ObjectClass       : user
ObjectGUID        : 33319b1b-bf93-4647-8e8d-d193be91043d
SamAccountName    : BStevens
SID               : S-1-5-21-4025744032-3205137867-863233111-1711
Surname           : Stevens
UserPrincipalName : 




PS C:\> New-LabADUser -NumberOfUsers 3 | select Name

Name   
----   
DAustin
JTaylor
WOrtiz 



PS C:\> $UsersCreated = New-LabADUser -NumberOfUsers 10 | select DistinguishedName

PS C:\> $UsersCreated

DistinguishedName                      
-----------------                      
CN=FDavis,CN=Users,DC=south,DC=lab     
CN=MDixon,CN=Users,DC=south,DC=lab     
CN=RNichols,CN=Users,DC=south,DC=lab   
CN=ABailey,CN=Users,DC=south,DC=lab    
CN=TRose,CN=Users,DC=south,DC=lab      
CN=ECarter,CN=Users,DC=south,DC=lab    
CN=TGonzales,CN=Users,DC=south,DC=lab  
CN=KWashington,CN=Users,DC=south,DC=lab
CN=JHunter,CN=Users,DC=south,DC=lab    
CN=DGray,CN=Users,DC=south,DC=lab      

Users are created with Organization = “LabUser” to make them easy to find:

PS C:\>  Get-ADUser -Filter "Organization -EQ 'LabUser'" | select Name

Name       
----       
BStevens   
DAustin    
JTaylor    
WOrtiz     
FDavis     
MDixon     
RNichols   
ABailey    
TRose      
ECarter    
TGonzales  
KWashington
JHunter    
DGray

And the cmdlet…

function New-LabADUser
 {
 [CmdletBinding()]
 Param
 (
 # Param2 help description
 [int]$NumberOfUsers = 1
 )

Begin
 {
# build the names lists
 $GivenName = ("Aaron", "Adam", "Alan", "Albert", "Alice", "Amanda", "Amy", "Andrea", "Andrew", "Angela", "Ann", "Anna", "Anne", "Annie", "Anthony", "Antonio", "Arthur", "Ashley", "Barbara", "Benjamin", "Betty", "Beverly", "Billy", "Bobby", "Bonnie", "Brandon", "Brenda", "Brian", "Bruce", "Carl", "Carlos", "Carol", "Carolyn", "Catherine", "Charles", "Cheryl", "Chris", "Christina", "Christine", "Christopher", "Clarence", "Craig", "Cynthia", "Daniel", "David", "Deborah", "Debra", "Denise", "Dennis", "Diana", "Diane", "Donald", "Donna", "Doris", "Dorothy", "Douglas", "Earl", "Edward", "Elizabeth", "Emily", "Eric", "Ernest", "Eugene", "Evelyn", "Frances", "Frank", "Fred", "Gary", "George", "Gerald", "Gloria", "Gregory", "Harold", "Harry", "Heather", "Helen", "Henry", "Howard", "Irene", "Jack", "Jacqueline", "James", "Jane", "Janet", "Janice", "Jason", "Jean", "Jeffrey", "Jennifer", "Jeremy", "Jerry", "Jesse", "Jessica", "Jimmy", "Joan", "Joe", "John", "Johnny", "Jonathan", "Jose", "Joseph", "Joshua", "Joyce", "Juan", "Judith", "Judy", "Julia", "Julie", "Justin", "Karen", "Katherine", "Kathleen", "Kathryn", "Kathy", "Keith", "Kelly", "Kenneth", "Kevin", "Kimberly", "Larry", "Laura", "Lawrence", "Lillian", "Linda", "Lisa", "Lois", "Lori", "Louis", "Louise", "Margaret", "Maria", "Marie", "Marilyn", "Mark", "Martha", "Martin", "Mary", "Matthew", "Melissa", "Michael", "Michelle", "Mildred", "Nancy", "Nicholas", "Nicole", "Norma", "Pamela", "Patricia", "Patrick", "Paul", "Paula", "Peter", "Philip", "Phillip", "Phyllis", "Rachel", "Ralph", "Randy", "Raymond", "Rebecca", "Richard", "Robert", "Robin", "Roger", "Ronald", "Rose", "Roy", "Ruby", "Russell", "Ruth", "Ryan", "Samuel", "Sandra", "Sara", "Sarah", "Scott", "Sean", "Sharon", "Shawn", "Shirley", "Stephanie", "Stephen", "Steve", "Steven", "Susan", "Tammy", "Teresa", "Terry", "Theresa", "Thomas", "Timothy", "Tina", "Todd", "Victor", "Virginia", "Walter", "Wanda", "Wayne", "William", "Willie")
 $Surame = ("Smith", "Johnson", "Williams", "Jones", "Brown", "Davis", "Miller", "Wilson", "Moore", "Taylor", "Anderson", "Thomas", "Jackson", "White", "Harris", "Martin", "Thompson", "Garcia", "Martinez", "Robinson", "Clark", "Rodriguez", "Lewis", "Lee", "Walker", "Hall", "Allen", "Young", "Hernandez", "King", "Wright", "Lopez", "Hill", "Scott", "Green", "Adams", "Baker", "Gonzalez", "Nelson", "Carter", "Mitchell", "Perez", "Roberts", "Turner", "Phillips", "Campbell", "Parker", "Evans", "Edwards", "Collins", "Stewart", "Sanchez", "Morris", "Rogers", "Reed", "Cook", "Morgan", "Bell", "Murphy", "Bailey", "Rivera", "Cooper", "Richardson", "Cox", "Howard", "Ward", "Torres", "Peterson", "Gray", "Ramirez", "James", "Watson", "Brooks", "Kelly", "Sanders", "Price", "Bennett", "Wood", "Barnes", "Ross", "Henderson", "Coleman", "Jenkins", "Perry", "Powell", "Long", "Patterson", "Hughes", "Flores", "Washington", "Butler", "Simmons", "Foster", "Gonzales", "Bryant", "Alexander", "Russell", "Griffin", "Diaz", "Hayes", "Myers", "Ford", "Hamilton", "Graham", "Sullivan", "Wallace", "Woods", "Cole", "West", "Jordan", "Owens", "Reynolds", "Fisher", "Ellis", "Harrison", "Gibson", "Mcdonald", "Cruz", "Marshall", "Ortiz", "Gomez", "Murray", "Freeman", "Wells", "Webb", "Simpson", "Stevens", "Tucker", "Porter", "Hunter", "Hicks", "Crawford", "Henry", "Boyd", "Mason", "Morales", "Kennedy", "Warren", "Dixon", "Ramos", "Reyes", "Burns", "Gordon", "Shaw", "Holmes", "Rice", "Robertson", "Hunt", "Black", "Daniels", "Palmer", "Mills", "Nichols", "Grant", "Knight", "Ferguson", "Rose", "Stone", "Hawkins", "Dunn", "Perkins", "Hudson", "Spencer", "Gardner", "Stephens", "Payne", "Pierce", "Berry", "Matthews", "Arnold", "Wagner", "Willis", "Ray", "Watkins", "Olson", "Carroll", "Duncan", "Snyder", "Hart", "Cunningham", "Bradley", "Lane", "Andrews", "Ruiz", "Harper", "Fox", "Riley", "Armstrong", "Carpenter", "Weaver", "Greene", "Lawrence", "Elliott", "Chavez", "Sims", "Austin", "Peters", "Kelley", "Franklin", "Lawson")
 }
 Process
 {
 for ($i = 1; $i -le $NumberOfUsers; $i++)
 {
 $gn = $GivenName | Get-Random
 $sn = $Surame | Get-Random
 $AccountName = $gn.Substring(0,1) + $sn

# create a password to fit most requirements
$pw = ""
 do {$pw += (33..126) | foreach {[char]$_} | Get-Random}
 until ($pw.Length -ge 16)
 $AccountPassword = ConvertTo-SecureString $pw -AsPlainText -Force

New-ADUser -AccountPassword $AccountPassword -GivenName $gn -Surname $sn -Name $AccountName -Organization "LabUser" -Enabled $true -PassThru
 }
 }

 }

 

The Script Development Process

Script Development Process
Script Development Process

This script development process is an iterative process designed to help you rapidly write quality scripts. The principle is to understand and define the problem. Break up your solution into small steps. Code one small step at a time. Test each small step and get it right before moving on to the next piece. Repeat these steps until you have your solution. All while managing your time.

When is automation the solution?

The benefits of automation are primarily to save time and to decrease errors associated with manual actions. You will need to make the calculation if a script will benefit your situation. How many hours will it take for you to write these script? How many hours of manual work will the script save?

Continue reading “The Script Development Process”

Output the PowerShell assignment command from an array

This function will take an array and output the PowerShell assignment command with the elements from the array. It would be useful if you want to hardcode a string array and don’t want to manually format it.

The function is below, along with an example usage and example output.

function ArrayAssignment ($MyArray) {
    $ArrayContent = "`$Array = @("
    # Output elements except the last
    $MyArray[0..$($MyArray.Count-2)] | foreach {$ArrayContent += "`"$_`","}
    # Output the last element
    $ArrayContent += "`"$($MyArray[$MyArray.Count-1])`")"
    $ArrayContent
    }

# Create an array of the first 20 services names for an example
$Example = (Get-Service).DisplayName[0..19]

# Use the function
ArrayAssignment $Example

# Example Output:
PS C:\>
$Array = @("Adobe Flash Player Update Service","AllJoyn Router Service","Application Layer Gateway Service","AMD External Events Utility","AMD FUEL Service","Application Identity","Application Information","Apple Mobile Device Service","Application Management","App Readiness","AppX Deployment Service (AppXSVC)","ASP.NET State Service","Windows Audio Endpoint Builder","Windows Audio","ActiveX Installer (AxInstSV)","BitLocker Drive Encryption Service","Base Filtering Engine","Background Intelligent Transfer Service","Bonjour Service","Background Tasks Infrastructure Service")