Subj : Feature request
To   : Andrew Leary
From : Michiel van der Vlist
Date : Fri Jul 27 2018 01:25:02

* Originally in NETMAIL001
* Crossposted in MAKENL_NG

Hello Andrew,

I request that the following option be added to Makenl:

RemoveBOM 1

What should it do?

Remove any UTF-8 Byte Order Mark (BOM).


Why?

Now that a project has started with a daily nodelist in UTF-8 in Z2, nodelist
clercks need a simple way to edit UTF-8 text files. Some UTF-8 capable editors
insist on adding a BOM to the edited file. Adding a BOM to an UTF-8 file is an
outdated concept, but some editors, notably, Notepad, insist on adding one
anyway. For most applications this does no harm, so people are not inclined to
dump their favorite editor in favour of one that does not insist on adding a
BOM. For nodelist processing this is a show stopper. A BOM interferes with
nodelist processing. There should be no BOM in the nodelist. Forcing nodelist
clercks to change editors or jump through some other hoops just to get rid of a
BOM raises a threshold that may stop them from participating in the project. If
MakeNl had the ability to remove BOMs, they could continue to use their
favorite editor without having to worry about BOMs.


Notes:

Normally a BOM would only be found at the start of a file, but some nodelist
clercks use scripts that concatenate files, so a BOM might also appear in the
middle of a file. It may even be found in the middle of a line.

The UTF-8 BOM consists of the three bytes 0xEF 0xBB 0xBF. Whenever this
sequence appears it should be removed.


Cheers, Michiel




--- GoldED+/W32-MSVC 1.1.5-b20130111
 * Origin: http://www.vlist.eu (2:280/5555)

.