Synchronet NewsLink Installation/Configuration
==============================================

$Id: newslink.txt,v 1.6 2012/07/18 14:52:35 deuce Exp $

Background
----------
NewsLink is an external module for Synchronet BBS Software v3.1+.
It is used to link one or more Synchronet sub-boards (aka message areas)
with one or more USENET (or other NNTP network) newsgroups on one or
more News Servers.

News Servers use the Network News Transfer Protocol (NNTP) to transfer
messages to and from a News Client (aka newsreader). Popular News Clients
include Microsoft Outlook Express, Netscape Communicator, Mozilla, Pine,
Xnews, and many others. NewsLink emulates a News Client based on the NNTP
standard. News Servers normally listen for incoming connections on TCP
port 119 and usually require an authenticated user login.

NewsLink is written in 100% JavaScript and the source code (if you're
interested) is contained in the files newslink.js and newsutil.js in the
Synchronet EXEC directory.


Step-by-Step Installation/Configuration
---------------------------------------

If you have installed Synchronet v3.10 or v3.11 from scratch (not an upgrade
from an earlier version), then some of the following steps may have already
been completed for you.


1. Go into SCFG->External Programs->Timed Events and created a timed event
   for NewsLink (if one doesn't already exist). Example:

   [NEWSLINK Timed Event]
    1: Internal Code                   NEWSLINK
    2: Start-up Directory
    3: Command Line                    ?newslink
    4: Enabled                         Yes
    5: Execution Node                  1
    6: Execution Days of Month         Any
    7: Execution Days of Week          All
    8: Execution Frequency             48 times a day
    9: Requires Exclusive Execution    No
   10: Force Users Off-line For Event  No
   11: Native (32-bit) Executable      <doesn't matter>
   12: Use Shell to Execute            <doesn't matter>
   13: Background Execution            <doesn't matter>
   14: Always Run After Init/Re-init   No

   You may change the Execution Frequency to suit your needs.
   
2. Go into SCFG->Message Areas and create a new Message Group for your USENET
   messsage areas (assuming you're linking to a USENET News Server). Example:

   [USENET Group]
   1: Long Name                  USENET Newsgroups
   2: Short Name                 USENET
   3: Internal Code Prefix       USENET_
   4: Access Requirements

   Note: At this point you may want to login to your New Server with a regular
         News Client (aka newsreader), retrieve a list of available newsgroups
         and make a list of the newsgroups you would like linked into your
         BBS's message areas.

3. Now go into SCFG->Message Areas->USENET->Sub-boards and create some
   sub-boards to link to newsgroups on the News Server. Example:

   [alt.bbs.allsysop Sub-board]
    1: Long Name                  alt.bbs.allsysop
    2: Short Name                 alt.bbs.allsysop
    3: QWK Name                   allsysop
    4: Internal Code              USENET_ALLSYSOP
    5: Newsgroup Name             alt.bbs.allsysop
    6: Access Requirements
    7: Reading Requirements
    8: Posting Requirements       REST NOT G
    9: Operator Requirements
   10: Moderated Posting User
   11: Maximum Messages           5000
   12: Purge by Age               Disabled
   13: Duplicate Checking         Enabled (10000 message CRCs)

4. For each NewsLinked sub-board, set Network Options->Internet (UUCP/NNTP)
   to "Yes".

5. Edit your ctrl/newslink.cfg file:

   A. Change the "SERVER" line to specify the host name or IP address of the
      News Server you are linking with. If the News Server is listening on a
      non-standard TCP port (e.g. other than 119), you will need to add a 
      "PORT" line specifying the TCP port number of the server.

   B. If the News Server requires an authenticated user login, change the 
      "USER" and "PASS" lines to specify the username and password you use 
      to login to the above News Server. If the "USER" line is missing or
      commented-out (with a semicolon), then no user authentication will
      be attempted (sometimes called "anonymous login").

   C. For each sub-board you configured in step 3, add an "AREA" line,
      specifying each sub-board's "Internal Code" (from SCFG->Message Areas).
      Example: AREA USENET_ALLSYSOP

   D. By default, NewsLink will only import articles (aka messages) posted
      *after* the first scan of the newsgroups (i.e. it will not import
      *any* articles during the first run). If you want NewsLink to import 
      *all* existing articles for *all* linked newsgroups (potentially many
      *thousands* of messages, depending on the News Server), add the line 
      "FLAGS I" (case insensitive).
      WARNING: Using this feature may cause the initial NewsLink execution
      to take a long time to download all the existing articles (potentially
      many thousands) that may be stored on the News Server.

   E. Remove or comment-out (with a semicolon) the "DISABLED" line.

   F. If you wish to use TLS/SSL, incomment (remove the semicolon) the "TLS"
      line.

6. Now you may force the NEWSLINK timed event to run. In SBBSCTRL (Win32 GUI),
   you can do this by clicking BBS->Force Timed Event->NEWSLINK, or for the
   console or NT services versions of Synchronet, you can touch or create the
   file data/newslink.now to force the event to run. 

7. Watch your events log for the NewsLink execution progress and details.
   Remember, if the 'I' flag was not specified (globally or per-newsgroup)
   in your ctrl/newslink.cfg, then no articles (aka messages) will be imported
   during the first run and only newly-posted articles will be imported on
   subsequent runs.

Area Flags
----------
Each "AREA" line in the NewsLink configuration file can specify one or more
"flag characters" to enable special behaviors. These same flag characters may
be specified globally (for all linked areas) using the "FLAGS" line.

The supported flags characters are:

x   do not add tearlines & taglines to exported messages
n   do not add "From Newsgroup" text to imported messages
t   do not add tearline to imported messages
a   convert extended-ASCII chars to ASCII on imported messages
r   remove "Newsgroups:" header field from imported messages
b   decode single-part (uuencoded or yenc-encoded) binary attachments
i   import all articles on first run (not just new articles)
s   no subject filtering (do not check against text/subject.can)
m   moderate imported messages (a subop will need to validate every message)

To specify one or more flag characters for a specific area, you must use the
following "AREA" line syntax:

    AREA <sub-code> <newsgroup> <flags> [attachment_dir]

where:

    <sub-code> is the sub-board's complete internal code (including the
    group's code prefix, if there is one).

    <newsgroup> is the name of the newsgroup as it is represented on the
    News Server. If this field is missing or is less than 2 characters in
    length, the newsgroup name from SCFG is used.

    <flags> is the optional list of flag characters.

    [attachment_dir] is the optional destination directory for decoded
    binary file attachments (only used with the 'b' flag)

Examples:

    AREA USENET_ALLSYSOP alt.bbs.allsysop nib /sysop_attachments
    AREA USENET_ALLSYSOP - asr


Multiple Servers/Config Files
-----------------------------
ctrl/newslink.cfg is just the default NewsLink configuration file name. If you
specify the path to a different file on the "?newslink" command-line, NewsLink
will use that configuration file instead. This allows you to have multiple
NewsLink timed events configured with different configurations for different
News Servers.


Command-line Options
--------------------
Other supported NewsLink command-line options:

-d      enable debug output
-ri     reset import pointers (import all)
-ui     update import pointers (import none)
-re     reset export pointers (export all)
-ue     update export pointers (export none)
-ne     no e-mail addresses
-nm     no mangling of e-mail addresses (disable anti-SPAM measures)
-um     un-mangle e-mail addresses when importing
-ix [n] import a fixed number (n) of articles/messages (default is 500)


Controlling SPAM
----------------
Most SPAM messages on public NNTP networks (e.g. USENET) are directed at
multiple newsgroups. For this reason, the newslink.cfg file supports a
"max_newsgroups_per_article" value which defaults to 5.

If you find NewsLink is filtering valid messages with an error in the log like:
!Too many newsgroups (7)
try increasing this value.

If you find NewsLink is allowing SPAM through that is addressed to multiple
newsgroups, try reducing this value. Set max_newsgroups_per_article to 0 to
disable this feature.

NewsLink also uses the text/subject.can filter file to filter (ignore)
articles with specific text in the subject and the ctrl/twitlist.cfg file to
ignore messages from specific users or domains.


Yielding
--------
NewsLink will yield time-slices based on the "lines_per_yield" (default: 5)
and "yield_length" (default: 1) values. If you find NewsLink is consuming
a large portion of your CPU cycles, try decreasing the "lines_per_yield"
value or increasing the "yield_length" value.


Stopping NewsLink
-----------------
NewsLink can be stopped by "touching" (creating or updating) the file:
data/newslink.stop


JSexec
------
With Synchronet v3.11, it is also possible to run NewsLink externally from
Synchronet (or as a background timed event) by using JSexec.

/* End of file */
