Lazy Diary @ Hatena Blog

PowerShell / Java / miscellaneous things about software development, Tips & Gochas. CC BY-SA 4.0/Apache License 2.0

How to join nested collections with Stream API

For example, in PushOperation.java in EGit, PushOperationSpecification contains LinkedHashMap<URIish, Collection<RemoteRefUpdate>>. So you can join all remote name with Stream API like: String specs = this.specification.getURIs().stream() .map(uri -> this.specification.getRe</uriish,>…

You cannot wrap checked exceptions with lambda in Java

Context You can use lambda as lazy evaluation syntax in Java. Problem You can wrap RuntimeException with lambda like (it shows "omit exception"): import java.util.function.Supplier; public class OmitRuntimeException { public static void ma…

プログラムに対する指摘の受け取りかたの分類

前提: プログラムに対する指摘への回答が全て「仕様です」で返ってくる、という笑い話は前世紀からある。 ここで、実際にはどういった回答が考えられるか。 検討観点: プログラムの設計時に検討対象に挙がっていた内容か? 実現可能性はあるか? 実際にプロ…

Run a command with specific user priviledge in Windows (like sudo in Unix)

Problem: You can use runas command to run a command with specific user privilege. (Note: You have to type password into command prompt): C:\>runas /user:domain\username cmd Enter the password for domain\username: Attempting to start cmd as…

You cannot have two forks from one repository in GitHub

Problem: “Fork” button in GitHub forks a repository into your account. You cannot rename the repository when you fork it. You cannot have two repositories with identical name. So, you cannot have two forks from one repository simultaneousl…

ISO/IEC 12207 (JIS X 0160:2012) ソフトウェア構成管理プロセスのうちバージョン管理システム、GitHub/GitLab/etc、git-flow/GitHub Flowでカバーできる内容

(In English: What can you do for ISO/IEC 12207 (JIS X 0160:2012) Software Configuration Management Process with VCSs, GitHub/GitLab/etc, and git-flow/GitHub Flow) ISO/IEC 12207 (JIS X 0160:2012)では、ソフトウェア開発におけるソフトウェア構…

Rounding strategy in Java/JavaScript/PowerShell

tl;dr Java: Floor in type casting. Round half up for Math.round(). You can choose rouding strategy for BigDecimal. JavaScript: Round half up for Math.round(). Round half away from zero for toFixed(). PowerShell (.NET CLR): Round half to ev…

PowerShell does banker's rounding in type casting to int

Problem: PowerShell rounds 0.5 to 0 (banker’s rounding) in type casting to int. PS > @(-3.5,-2.5,-1.5,-0.5,0.5,1.5,2.5,3.5) | ForEach-Object { "{0} -> {1}" -F $_, [int]$_ } -3.5 -> -4 -2.5 -> -2 -1.5 -> -2 -0.5 -> 0 0.5 -> 0 1.5 -> 2 2.5 -…

Supported character encodings in Get-Content and Import-Csv (in PowerShell 2.0/4.0)

Tested in Windows 7 (Japanese). Import-Csv does not have -Encoding option in PowerShell 2.0. There are no option for UTF-32BE in PowerShell 2.0. (note: PowerShell ISE can handle UTF-32BE Files) Import-Csv does not support Unknown and Strin…

Who does recommend changeing password periodically and who doesn't

Who recommends changeing password periodically Some say you should change password periodically. PCI DSS version 3.2 8.2.4 Change user passwords/passphrases at least once every 90 days. プライバシーマーク制度 (JIPDEC)(In Japanese) based on…

How to convert from ADIF AAC to another format (ffmpeg and libav can't handle it)

Context: You have some AAC ADIF files and you want to convert them into MP3 (and the like) format. Problem: ffmpeg and libav don’t (and will never) support ADIF AAC files because it is “bad format”. see: https://lists.libav.org/pipermail/f…

You should not pass the result of Get-ChildItem into Get-Content (and the like) directly

Context: You can pass the result of Get-ChildItem into Get-Content directly: PS /home/satob/tmp> Get-ChildItem | Where-Object { $_.Name -like "*.csv" } | ForEach-Object { Get-Content $_ } "a","x" "b","2" ... Problem: You cannot pass the re…

You cannot use -Encoding option with Import-Csv in PowerShell 2.0

Context: You use PowerShell 2.0 (Windows 7 or Windows Server 2008 R2). You want to read CSV file that contain non-ASCII characters. Problem: In PowerShell 2.0, Import-Csv cmdlet doesn’t have -Encoding option. Solution: If you want to read …

Difference of acceptable parameters for -Encoding option

Acceptable parameters for -Encoding option are different for Get-Content, Set-Content, Export-Csv, Import-Csv, and Out-File. # cmdlet Default ASCII UTF-7 UTF-8 UTF-16LE UTF-16BE UTF-32LE UTF-32BE Byte Default OEM String Unknown 1 Get-Conte…

Functional limitation of The Nu Html Checker (v.Nu)

Problem: The Nu Html Checker (v.Nu) is useful HTML validator. It can be used not only from a web browser, but also from command line. But, this validator has some functional limitations (for both web interface and CLI version). v.Nu cannot…

How to write result of ConvertTo-Csv to a file in UTF-8 without BOM

Context: You want to write the result of ConvertTo-Csv in UTF-8 encoding without BOM. e.g. You need a file that can be read by a Java program (Java File API cannot handle BOM in UTF-8 encoded files). UTF-8 in PowerShell, e.g. ConvertTo-Csv…

.NET cannot distinguish Shift_JIS from MS932(Windows-31J)

Context: Japanese character encoding Shift_JIS (シフトJIS) and Microsoft Codepage 932 (a.k.a. MS932, Windows-31J in IANA) are slightly different. For example, full-width cent sign (¢) is 0x8191 in both Shift_JIS and MS932, but it is mappe…

PowerShellで法務省 戸籍統一文字情報のページからあるコードポイントの文字の情報を取得する

(In English: How to get information about a Japanese character from 戸籍統一文字情報 site (managed by The Ministry of Justice (Japan)) with PowerShell) 問題: ある漢字に関する情報(読みや、子の名に使える文字か等)を調べたければ、法務省の戸…

ConvertFrom-String is not available on PowerShell Core Edition

Problem: ConvertFrom-String is not available on PowerShell Core Edition. Reason: It seems by design. According to the source of PowerShell on GitHub, ConvertFrom-String is exported for Desktop Edition, but not exported for Core Edition for…

CharsetEncoder#canEncode() equivalent for PowerShell

Context: You want to test whether a codepoint is valid in a specific character encoding. Problem In .NET, there are no equivalent functions to CharsetEncoder#canEncode() in Java. Solution If you want to test whether a character is valid in…

Meanings of :owner, :repo, ... in GitHub API document

In GitHub API document (ex. https://developer.github.com/v3/repos/ ), examples of how to access API showed like GET /repos/:owner/:repo/contributors. Meanings of indicator variable-like valiables such as :owner, :repo are listed below: # v…

How to convert string to number with AngularJS

There is more than one way to convert string to number in JavaScript. Context: Use AngularJS to bind text boxes to variables. Sum up input values in some text boxes. Omit ng-required, ng-pattern, ng-minlength, ng-maxlength, etc., wherever …

URL of GitHub Enterprise API is different from github.com's one

Problem: I have tried to access GitHub Enterprise API with a URL like showed below, based on examples from some websites, but the response was 404 error. https://x.x.x.x/repos/Project/Repository/git/refs/heads Reason: URL of GitHub Enterpr…

System.Text.Encoding.GetEncodings() does not show all available encodings after call RegisterProvider()

Problem: If you want to use character encodings other than the default registered encodings, you have to call [System.Text.Encoding]::RegisterProvider([System.Text.CodePagesEncodingProvider]::Instance). But even if you call that method, th…

How to convert from a code point (U+xxxx) to a code point in another character encoding

function Convert-CodePoint { Param( [Parameter(ValueFromPipeline=$true,Mandatory=$true)] [string] $CodePoint, [Parameter(ValueFromPipeline=$false,Mandatory=$true)] $From, [Parameter(ValueFromPipeline=$false,Mandatory=$true)] $To ) begin { …

What does "US-ASCII only" means in Java regexp?

Java Patten class (regexp) supports POSIX character classes like \p{XDigit}. They are very useful when you want to check hex strings. In Java API Document, POSIX character classes say (US-ASCII only). What does it mean? https://docs.oracle…

Trimmed characters by trim() are different between languages

String object in .NET (PowerShell), JavaScript, and Java all have trim() method. But trimmed characters by trim() are different between these languages. # codepoint .NET JavaScript Java 1 0000-0008 ✘ ✘ ✔ 2 0009-000D ✔ ✔ ✔ 3 000E-001F ✘ ✘ ✔…

How to convert from a code point (U+xxxx) to a character

function ConvertFrom-CodePoint { Param( [Parameter(ValueFromPipeline=$true,Mandatory=$true)] [string] $CodePoint, [Parameter(ValueFromPipeline=$false,Mandatory=$true)] $From ) begin { [System.Text.Encoding]::RegisterProvider([System.Text.C…

You can't use > in comparation (and it will be silently failed)

In PowerShell, you have to use -lt and -gt, instead of < and > in comparation, like POSIX sh. Especially if you wrongly use > like below, the comparison will be silently failed (evaluated as $false) and value of left-hand operand is wrote …

Escape "*" in -like and -match

When you use -like, escape character to escape “*” depends on quotation marks. # quotation escape command Result 1 “ N/A $_ -like “*” Not escaped / Match to any strings 2 “ ` $_ -like “`*” Not escaped / Match to any strings 3 “ `` $_ -like…

You cannot set a result of cmdlets to a property of object by Set-Variable

Context: You can set a result of cmdlets to a variable by Set-Variable. Problem: You cannot set a result of cmdlets to a property of object by Set-Variable. PS > $foo = New-Object PSObject | Select-Object Property PS > Get-ChildItem | Set-…

Mandatory parameter from pipeline, and empty lines

In a function that retrieve parameters from pipeline, if you make a parameter mandatory (Mandatory=$true), you will get error when an empty line is passed from the pipeline. If you receive parameters from the data which may contain empty l…

You cannot retrieve a required parameter from "either pipeline or argument"

## Context: A mandatory parameter of a function retrieved from pipeline is defined like this: param( [Parameter(ValueFromPipeline=$true, Mandatory=$true)] [string] $foo ) A mandatory parameter of a function retrieved from argument is defin…

ConvertTo-Csv or Export-Csv always add double quote to all columns

ConvertTo-Csv or Export-Csv always add double quote to all columns. Get-Help Export-Csv -Full shows samples of CSV with no double quotes, but it seems there are no options to suppress double quote. This behavior doesn’t depend on system lo…

Avoid error when adding duplicated keys to hash (case sensitivity in hash tables on PowerShell)

Problem: Keys of hash tables are case sensitive in most languages (Java/JavaScript/Perl/Ruby/…). But in PowerShell, key of hash table is case insensitive (if you use string as key), so you will bump into errors in code like: PS > $hash = @…

You can't sort strings in ASCII order

When you pass chars to Sort-Object, the chars sorted in ASCII order. PS > (0x7E..0x21 | ForEach-Object { [char]$_ } | Sort-Object) -join '' !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ But …

Difference of Sort-Object -Unique, Sort-Object | Get-Unique, and Select-Object -Unique

If you want to uniq a list in PowerShell, there is more than one way to do it and these are slightly different. # Cmdlets Sort-Object -Unique Sort-Object | Get-Unique Select-Object -Unique 1 Sort elements Yes Yes No 2 Case sensitive in ded…

Characters treated as a part of a variable name

Context: You can expand value of variables when you write variable name in double-quoted strings like: > $var = "123" > Write-Host "$var" 123 Problem: Some characters are treated as a part of a variable name, so you can’t use these charact…

You can't expand array[index] in a string

Context: You can expand variables in a string like: > $var = "123" > Write-Host "$var" 123 > Problem: You can’t expand array with index: > $arr = @("456", "789") > Write-Host "$arr[0]" 456 789[0] It is also ineffective to add braces: > Wri…

The creator of branch is not contained in return values of GitHub API

Context: On GitHub, you can see which branches have you created (see “Your branches” section of “Branches” page). Problem: In GitHub API, there is no way to get the creator of the branches. The return values of GitHub References API don’t …

powershell -Version is unsupported in Linux environment

Context: In Windows environment, you can use -Version option to get Powershell pretend to be an older version. Problem: In Linux environment, -Version option is unsupported. Reason: manpage of powershell(1) doesn’t have any content about -…

Size field is folded when the size > 10GB in Get-ChildItem

Context: You use Powershell version 4 or earlier. You have a file that the size is > 10GB. Problem: Size field is folded when you Get-ChildItem to the file that the size is > 10GB. Mode LastWriteTime Length Name ---- ------------- ------ -…

Ways to convert relative paths to absolute paths

Some methods (e.g. System.IO.File.WriteAllLines()) needs absolute paths for parameters, so you have to convert from relative paths to absolute paths. In PowerShell, there is more than one way to do it ($RelativePath contains relative path …

Return value of Invoke-RestMethod or ConvertFrom-Json looks like a list but it's not a list

Context: Return value of Invoke-RestMethod looks like a list of objects: > $headers = @{ Accept = "application/json"; Authorization = "Bearer 0123456789abcdef0123456789abcdef01234567"; } > Invoke-RestMethod -Headers $headers -Method Get -U…

Delombok removes super(); in default constructors

Context: You have a class that has one or more constructors other than default constructor, so you have to define the default constructor explicitly. You use Lombok to generate some methods/fields. You use static code analyzers that warn e…

You can't access HTTP responce headers with Invoke-RestMethod

Context: You can call API of web services (e.g. GitHub) with Invoke-RestMethod. GitHub pagination API contains the last page number in HTTP responce headers. Problem: You can’t access HTTP responce headers with Invoke-RestMethod or the ret…

If you pass an empty string to ConvertFrom-Json

According to the spec of JSON, an empty string is not a valid JSON. If you pass an empty string to ConvertFrom-Json, it returns $null (doesn’t raise an exception). If you pass an empty JSON (“[]”) to ConvertFrom-Json, it returns empty list…

Set-Variable doesn't overwrite a variable when an empty list passed through pipe

Context: You can set a value to variable with Set-Variable. > "foo" | Set-Variable var > $var foo You can also overwrite a value of variable like this: > "foo" | Set-Variable var > $null | Set-Variable var > $var Problem: The value of vari…

ConvertFrom-Json with empty JSON returns an empty list

Context: You want to receive an JSON and check whether the JSON is empty or not. Problem: You cannot check empty JSON with -eq operator, and -eq doesn’t return true/false. > $hoge = (Get-Content ./empty.json | ConvertFrom-Json) > $hoge -eq…

Difference between Remove-Item -Force and rm -f

Context: You want to remove a file with Remove-Item. You want to ignore errors even if the file doesn’t exist. Problem: You get an error when execute Remove-Item -Force in the file that does not exist. Reason: The behavior is by design. rm…