Lazy Diary @ Hatena Blog

PowerShell / Java / miscellaneous things about software development, Tips & Gochas. CC BY-SA 4.0/Apache License 2.0

Supported character encodings in Get-Content and Import-Csv (in PowerShell 2.0/4.0)

PowerShell charset

Tested in Windows 7 (Japanese).

  • Import-Csv does not have -Encoding option in PowerShell 2.0.
  • There are no option for UTF-32BE in PowerShell 2.0. (note: PowerShell ISE can handle UTF-32BE Files)
  • Import-Csv does not support Unknown and String Encoding.
  • String, Default, OEM, and without -Encoding option are the same.
  • File in UTF-16 with BOM can be read even if -Encoding option is wrong
  • UTF-32BE with BOM could be misdetermined to UTF-16BE (because starting bytes are FEFF)

f:id:satob:20170601231527p:plain

Who does recommend changeing password periodically and who doesn't

Who recommends changeing password periodically

Some say you should change password periodically.

I think IPA should have another report that denies the effect of periodic password changing, but I cannot found a such information.

また、破られにくいパスワードを使っていても、長期間変更せずにいると漏えいする危険性が高まります。パスワードは、定期的に（例えば月毎）変更するようにしましょう。

Who doesn't recommend changeing password periodically

Some say you don't have to (or should not) change password periodically.

Who neither recommends nor denies changeing password periodically

Conclusion:

You can choose either one of them according to what your customer said!

2017-07-07

Add 教育情報セキュリティポリシーに関するガイドライン案 (MEXT)

2017-08-12

Add コンピュータウイルス・不正アクセスの届出状況[2010年2月分]について(IPA) and JIS Q 27001:2014

2018-01-05

2019-01-29

  • Move プライバシーマーク制度 (JIPDEC) JIS Q 15001:2006をベースにした個人情報保護マネジメントシステム実施のためのガイドライン 第2版 from who recommends to who neither recommends nor denies

2019-02-22

  • Move 医療情報システムの安全管理に関するガイドライン from who recommends to who doesn't recommend

How to convert from ADIF AAC to another format (ffmpeg and libav can't handle it)

ffmpeg

Context:

You have some AAC ADIF files and you want to convert them into MP3 (and the like) format.

Problem:

ffmpeg and libav don’t (and will never) support ADIF AAC files because it is “bad format”.

Solution:

You can convert an ADIF AAC files into a ordinary (handled with ffmpeg) .m4a file with this adif2mp4.exe on this page: https://rtfreesoft.blogspot.jp/2009_10_01_archive.html

If you use Ubuntu Linux 16.04, you can run adif2.exe from Wine 1.6.2 (installed from Ubuntu 16.04 apt repository). For example, you can convert all *.AAC files in current directory into .m4a format:

$ for i in *.AAC; do wine ../bin/adif2mp4.exe $i `basename $i .AAC`.m4a; done

Once the files are converted to .m4a format, you can handle them with ffmpeg like this:

$ for i in *.m4a; do ffmpeg -i $i `basename $i .m4a`.mp3; done

You should not pass the result of Get-ChildItem into Get-Content (and the like) directly

PowerShell

Context:

You can pass the result of Get-ChildItem into Get-Content directly:

PS /home/satob/tmp> Get-ChildItem | Where-Object { $_.Name -like "*.csv" } | ForEach-Object { Get-Content $_ }          
"a","x"
"b","2"
...

Problem:

You cannot pass the result of Get-ChildItem into Get-Content directly when you Get-ChildItem from other than current directory:

PS /home/satob> Get-ChildItem /tmp | Where-Object { $_.Name -like "*.csv" } | ForEach-Object { Get-Content $_ }         
Get-Content : Cannot find path '/home/satob/foobar.csv' because it does not exist.
At line:1 char:80
+ ... -Object { $_.Name -like "*.csv" } | ForEach-Object { Get-Content $_ }
+                                                          ~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (/home/satob/foobar.csv:String) [Get-Content], ItemNotFoundException
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand

It is troublesome because the problem will not occur when you test with the files in current directory.

Reason:

The result of Get-ChildItem is handled as relative path when you pass them to Get-Content.

Note: The problem will not occur even without FullName when you use wildcard with Get-ChildItem.

PS /home/satob> Get-ChildItem tmp/*.csv | ForEach-Object { Get-Content $_ }                                             
"a","x"
"b","2"
...

Solution:

You should specify FullName property explicitly:

PS /home/satob> Get-ChildItem tmp/ | Where-Object { $_.Name -like "*.csv" } | ForEach-Object { Get-Content $_.FullName }
"a","x"
"b","2"
...

You can also use FullName property with wildcards:

PS /home/satob> Get-ChildItem tmp/*.csv | ForEach-Object { Get-Content $_.FullName }                                    
"a","x"
"b","2"
...

Note: You should not use Name property with or without wildcards. It contains relative path:

PS /home/satob> Get-ChildItem tmp/ | Where-Object { $_.Name -like "*.csv" } | ForEach-Object { Get-Content $_.Name }    
Get-Content : Cannot find path '/home/satob/a.csv' because it does not exist.
At line:1 char:80
+ ... ct { $_.Name -like "*.csv" } | ForEach-Object { Get-Content $_.Name }
+                                                     ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (/home/satob/a.csv:String) [Get-Content], ItemNotFoundException
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand

PS /home/satob> Get-ChildItem tmp/*.csv | ForEach-Object { Get-Content $_.Name }                                        
Get-Content : Cannot find path '/home/satob/a.csv' because it does not exist.
At line:1 char:44
+ Get-ChildItem tmp/*.csv | ForEach-Object { Get-Content $_.Name }
+                                            ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (/home/satob/a.csv:String) [Get-Content], ItemNotFoundException
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand

You cannot use -Encoding option with Import-Csv in PowerShell 2.0

PowerShell charset

Context:

Problem:

In PowerShell 2.0, Import-Csv cmdlet doesn’t have -Encoding option.

Solution:

If you want to read a CSV file without CRLF in cells, you can use Get-Content -Encoding and ConvertFrom-Csv like:

PS > Get-Content -Encoding UTF8 ./foobar.csv | ConvertFrom-Csv

If you want to read a CSV file with CRLF in cells, you should convert the CSV file to UTF-16 with BOM, and read the file with Import-Csv (Import-Csv can recognize UTF-16 encoding automatically even in PowerShell 2.0):

PS > Get-Content -Encoding UTF8 ./foobar.csv | Out-File -Encoding Unicode -Path "./foobar-utf16.csv"
PS > Import-Csv ./foobar-utf16.csv