Lazy Diary @ Hatena Blog

PowerShell / Java / miscellaneous things about software development, Tips & Gochas. CC BY-SA 4.0/Apache License 2.0

PowerShell

Difference of behavior of String#split() in Java and -split operator in PowerShell

Both of String#split() in Java and -split operator in PowerShell take regex as argument, and split string into a list or an array, but there is some difference in behavior when you pass an empty string as argument. In Java: System.out.prin…

How to extract non-MS932 (Shift_JIS) compliant characters from string

function Get-NonMS932CompliantCharacter { Param( [Parameter(ValueFromPipeline=$true,Mandatory=$true)] [string] $TargetString ) process { $TargetStringBytes = [Text.Encoding]::UTF32.GetBytes($TargetString); for ($i=0; $i -lt $TargetStringBy…

tr equivalent in PowerShell (Unicode surrogate pair-aware)

There is no straightforward tr equivalent in Windows, so I made an cmdlet that you can use like tr command. This tr cmdlet is aware of Unicode characters including surrogate pairs. function tr { Param( [Parameter(ValueFromPipeline=$true,Ma…

How to capture network packet without additional software in Windows

(A) Use "netsh trace" command pros: You can use it in isolated network. cons: You should convert the captured file with Microsoft Message Analyzer if you want to see packets with Wireshark. (You can also view the packets with Microsoft Mes…

How to change locale of error messages of PowerShell interpreter

Problem You can set locale of the current thread in PowerShell like below: [System.Threading.Thread]::CurrentThread.CurrentCulture = [System.Globalization.CultureInfo]("en-US") But error messages of PowerShell interpreter shows in current …

How to use additional character encodings in PowerShell Core (Linux) and Desktop (Windows)

Problem You have to call [System.Text.Encoding]::RegisterProvider([System.Text.CodePagesEncodingProvider]::Instance) when you want to use additional character encodings in PowerShell Core (Linux). On the other hand, calling [System.Text.En…

Memory Consumption to Read XMLs in PowerShell

Background PowerShell has an useful function to read and parse XML files: [XML]. But reading XML files and (especially) building DOMs are memory consuming function. Question How much memory do you need to read XML files with [XML]? Methods…

How to escape special characters in PowerShell

Ways to escape special characters in PowerShell are different between its context, and so complecated... # Context Escaped char " ' \ ` `` 1 String with " " ✓ ✘ ✘ ✓ ✘ 2 String with " $ ✘ ✘ ✘ ✓ ✘ 3 String with " ' - - - - - 4 String wi…

Invoke-RestMethod to GitLab API causes mojibake

Context In PowerShell, You can call REST API with Invoke-RestMethod like: Invoke-RestMethod -Headers $headers -Method Get -Timeout 10 -Uri "https://api.github.com/users/octocat/orgs" Problem The result of Invoke-RestMethod causes mojibake …

Format operator with string contains "{" and "}"

Context In PowerShell, you can format a string with -F operator like: Get-Content foo.csv | ConvertFrom-Csv -Header Name,Code,Address | ` ForEach-Object { "{0} `n {1} {2}" -F $_.Name,$_.Code,$_.Address } Problem -F operator returns error l…

Get CodePoint-Property Pair from Scripts.txt on Unicode.org

Context You want to make a list of pair of unicode codepoint and its character property, like below: 00009,Cc 00020,Zs 00021,Po 00024,Sc ... Solution with PowerShell You can make the list from ftp://ftp.unicode.org/Public/UNIDATA/PropList.…

Convert deeply nested hash or array to JSON with ConvertTo-Json

Context You can read a JSON file like below with ConvertFrom-Json, and write with ConvertTo-Json properly. PS > Get-Content ./foo.json { "outerHash": { "innerHash": { "key": "value" } } } PS > Get-Content ./foo.json | ConvertFrom-Json | Co…

Convert an array to CSV with PowerShell

Problem You cannot convert an array with just pipeline the array to ConvertTo-Csv. PS > $array = ("a", "b", "c", "a", "d") PS > $array | ConvertTo-Csv "Length" "1" "1" "1" "1" "1" ... or just passing the array to ConvertTo-Csv. PS > Conver…

XmlNode.SelectNodes() always returns List in PowerShell 2.0

Problem: In PowerShell, by using XML DOM API in .NET, you can access to a child element in XML as a ordinary property. PS > $xml = New-Object System.Xml.XmlDocument PS > $xml.LoadXml('<a><b id="1">foo</b></a>') PS > $xml.SelectNodes('//a').b id #text -- ----- …

How to get a #text in XML even if the tag doesn't have attributes

Background: In PowerShell (even in C# or VB.NET?), you can get a body of the tag (text content) with '#text#' property. > $xml = New-Object System.Xml.XmlDocument > $xml.LoadXml('<a><b id="1">foo</b></a>') > $xml.SelectNodes('//a').b.'#text' foo Problem: If a …

Difference of GitHub API and GitLab API

Format of Personal Access Tokens In GitHub, personal access tokens are hex string, like e72e16c7e42f292c6912e7710c838347ae178b4a. In GitLab, personal access tokens are like Base62 string, like 9koXpg98eAheJpvBs5tK. Personal Access Tokens a…

Run a command with specific user priviledge in Windows (like sudo in Unix)

Problem: You can use runas command to run a command with specific user privilege. (Note: You have to type password into command prompt): C:\>runas /user:domain\username cmd Enter the password for domain\username: Attempting to start cmd as…

PowerShell does banker's rounding in type casting to int

Problem: PowerShell rounds 0.5 to 0 (banker’s rounding) in type casting to int. PS > @(-3.5,-2.5,-1.5,-0.5,0.5,1.5,2.5,3.5) | ForEach-Object { "{0} -> {1}" -F $_, [int]$_ } -3.5 -> -4 -2.5 -> -2 -1.5 -> -2 -0.5 -> 0 0.5 -> 0 1.5 -> 2 2.5 -…

Supported character encodings in Get-Content and Import-Csv (in PowerShell 2.0/4.0)

Tested in Windows 7 (Japanese). Import-Csv does not have -Encoding option in PowerShell 2.0. There are no option for UTF-32BE in PowerShell 2.0. (note: PowerShell ISE can handle UTF-32BE Files) Import-Csv does not support Unknown and Strin…

You should not pass the result of Get-ChildItem into Get-Content (and the like) directly

Context: You can pass the result of Get-ChildItem into Get-Content directly: PS /home/satob/tmp> Get-ChildItem | Where-Object { $_.Name -like "*.csv" } | ForEach-Object { Get-Content $_ } "a","x" "b","2" ... Problem: You cannot pass the re…

You cannot use -Encoding option with Import-Csv in PowerShell 2.0

Context: You use PowerShell 2.0 (Windows 7 or Windows Server 2008 R2). You want to read CSV file that contain non-ASCII characters. Problem: In PowerShell 2.0, Import-Csv cmdlet doesn’t have -Encoding option. Solution: If you want to read …

Difference of acceptable parameters for -Encoding option

Acceptable parameters for -Encoding option are different for Get-Content, Set-Content, Export-Csv, Import-Csv, and Out-File. # cmdlet Default ASCII UTF-7 UTF-8 UTF-16LE UTF-16BE UTF-32LE UTF-32BE Byte Default OEM String Unknown 1 Get-Conte…

How to write result of ConvertTo-Csv to a file in UTF-8 without BOM

Context: You want to write the result of ConvertTo-Csv in UTF-8 encoding without BOM. e.g. You need a file that can be read by a Java program (Java File API cannot handle BOM in UTF-8 encoded files). UTF-8 in PowerShell, e.g. ConvertTo-Csv…

PowerShellで法務省 戸籍統一文字情報のページからあるコードポイントの文字の情報を取得する

(In English: How to get information about a Japanese character from 戸籍統一文字情報 site (managed by The Ministry of Justice (Japan)) with PowerShell) 問題: ある漢字に関する情報(読みや、子の名に使える文字か等)を調べたければ、法務省の戸…

ConvertFrom-String is not available on PowerShell Core Edition

Problem: ConvertFrom-String is not available on PowerShell Core Edition. Reason: It seems by design. According to the source of PowerShell on GitHub, ConvertFrom-String is exported for Desktop Edition, but not exported for Core Edition for…

System.Text.Encoding.GetEncodings() does not show all available encodings after call RegisterProvider()

Problem: If you want to use character encodings other than the default registered encodings, you have to call [System.Text.Encoding]::RegisterProvider([System.Text.CodePagesEncodingProvider]::Instance). But even if you call that method, th…

How to convert from a code point (U+xxxx) to a code point in another character encoding

function Convert-CodePoint { Param( [Parameter(ValueFromPipeline=$true,Mandatory=$true)] [string] $CodePoint, [Parameter(ValueFromPipeline=$false,Mandatory=$true)] $From, [Parameter(ValueFromPipeline=$false,Mandatory=$true)] $To ) begin { …

Trimmed characters by trim() are different between languages

String object in .NET (PowerShell), JavaScript, and Java all have trim() method. But trimmed characters by trim() are different between these languages. # codepoint .NET JavaScript Java 1 0000-0008 ✘ ✘ ✔ 2 0009-000D ✔ ✔ ✔ 3 000E-001F ✘ ✘ ✔…

How to convert from a code point (U+xxxx) to a character

function ConvertFrom-CodePoint { Param( [Parameter(ValueFromPipeline=$true,Mandatory=$true)] [string] $CodePoint, [Parameter(ValueFromPipeline=$false,Mandatory=$true)] $From ) begin { [System.Text.Encoding]::RegisterProvider([System.Text.C…

You can't use > in comparation (and it will be silently failed)

In PowerShell, you have to use -lt and -gt, instead of < and > in comparation, like POSIX sh. Especially if you wrongly use > like below, the comparison will be silently failed (evaluated as $false) and value of left-hand operand is wrote …