Subj : Feature request
To   : Michiel van der Vlist
From : Andrew Leary
Date : Sat Feb 21 2015 11:32:15


 Mvd> * Originally in NETMAIL001
 Mvd> * Crossposted in MAKENL_NG

 Mvd> Hello Andrew,

 Mvd> I request that the following option be added to Makenl:

 Mvd> RemoveBOM 1

 Mvd> What should it do?

 Mvd> Remove any UTF-8 Byte Order Mark (BOM).


 Mvd> Why?

 Mvd> Now that a project has started with a daily nodelist in UTF-8 in Z2,
 Mvd> nodelist clercks need a simple way to edit UTF-8 text files. Some UTF-8
 Mvd> capable editors insist on adding a BOM to the edited file. Adding a BOM
 Mvd> to an UTF-8 file is an outdated concept, but some editors, notably,
 Mvd> Notepad, insist on adding one anyway. For most applications this does no
 Mvd> harm, so people are not inclined to dump their favorite editor in favour
 Mvd> of one that does not insist on adding a BOM. For nodelist processing this
 Mvd> is a show stopper. A BOM interferes with nodelist processing. There
 Mvd> should be no BOM in the nodelist. Forcing nodelist clercks to change
 Mvd> editors or jump through some other hoops just to get rid of a BOM raises
 Mvd> a threshold that may stop them from participating in the project. If
 Mvd> MakeNl had the ability to remove BOMs, they could continue to use their
 Mvd> favorite editor without having to worry about BOMs.


 Mvd> Notes:

 Mvd> Normally a BOM would only be found at the start of a file, but some
 Mvd> nodelist clercks use scripts that concatenate files, so a BOM might also
 Mvd> appear in the middle of a file. It may even be found in the middle of a
 Mvd> line.

 Mvd> The UTF-8 BOM consists of the three bytes 0xEF 0xBB 0xBF. Whenever this
 Mvd> sequence appears it should be removed.


 Mvd> Cheers, Michiel

I will see what I can do on this.  It will probably require some changes in how 
the input lines are parsed, so I expect it will not be done until 3.5.0 at the 
earliest.

Regards,

Andrew

--- AfterShock/Android 1.6
 * Origin: Bits & Bytes Droid (1:320/119.2)

.