|
|
3k Associates
NetMail/3000 Engine
Product enhancement report
Updated: 03/22/2007
PROGRAM FIXES/ENHANCEMENTS
Fixed Date
in Code Description
---- -------- ----------------------------------------------------------------
**** ******** ARPA PROGRAM
B.09 05/04/13 Added JCW NETMAILSMTPPORTIN which allows sites to change the
tcp port# NetMail/3000 & DeskLink listen on for incoming
mail connections, and NETMAILSMTPPORTOUT which sets the TCP
port SMTPOUT will use for OUTBOUND mail connections
(default -if not specified- is the SMTP standard port 25)
B.09 04/08/11 Fixed bug in quote-of-day service, so messages will wrap
after the last message in the catalog is reached
B.09 04/03/25 Added ARPANODAYTIME and ARPANOQOTD JCWs; when set to>0, they
disable the activation of the DAYTIME and Quote-of-the-Day
services (respectively)
B.09 03/12/29 Fixed bug when extended logging enabled, file gets re-opened
repeatedly (til process runs out of descriptors)
B.08 01/09/05 Added code to support extended logging
B.07 00/08/04 Program now quits (parm=24) if it encounters an error on
startup indicating that network transport wasn't started
B.07 00/05/09 New JCW ARPANOCHARGEN if set to '1' will disable the chargen
service normally started by ARPA (chargen isn't needed by
us, it was just provided as a useful test service, but has
become a target of denial-of-service attacks)
B.07 99/11/16 Modified quote-of-the-day to only return one unique quote per
day (max; i.e. if queried several times per day, you'll get the
same quote til the next day - or next day it's queried)
B.07 98/10/30 Fixed problem where ARPA wouldn't run for systems where only
the API was licensed.
B.07 98/10/15 Recoding tremendously increased responsiveness when multiple/
rapid-fire connection requests come in. Should solve problems
some POP sites experienced with heavy loads and users getting
timeouts
B.07 98/10/13 We now override the default maximum of 7 "queued" connection
requests for incoming network connections. Set to 255.
May help sites with heavy loads experiencing timeout or connect-
ion refused errors.
B.07 98/04/17 We now check for a file called "NONETWRK.PUB.THREEK" which,if
present, will ensure that no network (smtp/pop/finger/etc)
services get started up.
B.07 98/03/12 Changed DESKUSEGATEWAY jcw to allow a value of 3.
B.06 98/01/23 Added support for INT2SMTP process on startup (DeskLink
Intrinsic gateway)
B.06 98/01/23 Added "DESKUSEGATEWAY" jcw to determine which (or both)
HPDesk gateway process(es) to use
B.06 97/12/08 Quote-of-the-day now 'remembers' the last quote issued; it
no longer resets each time the job is stopped/started
B.06 97/12/04 Added CRLF to end of "quote of the day" messages sent out by
the server
B.06 97/11/21 Program now recovers from recurring IOWait CCL errors; these
used to disable further network connections. IOWait problem
with MPE recurred in MPE/iX 5.5 PP3.
B.06 97/05/09 Refused connections (from refuse/excludes files) are now
logged to the errorlog.netmail file
B.06 97/02/14 New SEPARATESTDLISTS jcw in job stream causes netmail/desklink
job to create separate stdlist spoolfiles for each son process
B.06 97/02/04 License file now supports dates>1999.
B.05 96/11/14 Shuts down much quicker; kills idle processes in parallel now
rather than in sequence (as before)
B.05 96/09/05 Obeys the number of pop2 and pop3 servers (from global config)
B.04 96/08/02 No longer tries to start up gopher servers if product not
licensed
B.04 96/06/12 Native Mode version
B.03 96/05/16 New jcw - if there is networking software available, but it's
not configured (or user doesn't want to use it for some reason)
then add !SETJCW ARPANONETWORK=1 to the netmail job before the
!RUN ARPA.SYS.THREEK line.
B.03 96/03/12 If mailman job is accidentally run in the THREEK account
(instead of HPOFFICE) and DeskLink is licensed then the program
prints an error message in $stdlist and to console then stops.
B.03 96/02/27 Fixed problem with IOWAIT errors when 3000 gets a tcp request
from a system it can't get back to (route problems in nmmgr)
arpa no longer aborts; and new requests can continue
B.02 95/11/02 Now looks for ARPAQUEUE jcw on startup - if =0 (or not set)
ARPA automatically puts itself in the CS queue. If =1, then
it puts itself in DS, and if =2 it puts itself in ES.
B.02 95/06/20 Added support for running FINGSERV finger server program
B.02 95/06/20 Added support for an "excludes.data.threek" file where the
admin can list ip addresses which will (+nnn.nnn.nnn.nnn)or
will NOT (-nnn.nnn.nnn.nnn) be allowed to make any incoming
connection (SPAM prevention and added security)
(Addresses NOT listed are allowed to connect by default)
B.02 95/06/20 Fixed code so that if gopher database was not initialized, then
we won't start any gophserv processes
B.02 95/06/20 Changed connecting socket# to print as an octal number instead
of a signed integer
B.01 95/06/01 Added date/time display to $STDLIST for socket connections
B.01 94/11/04 Fixed lockup problem on MPE/iX 5.0 (server suspends and never
awakens)
A.06 94/08/30 Workaround for bug in MPE/iX 4.5 that caused SMTPSERV processes
to be created but not re-used
A.06 94/01/24 Enhanced code so printing to slaved printers ($LOCAL) will
automatically eject the page for laser printers or other
printers that normally buffer output; also for reflection
users that used to have to manually hit the "close printer"
button
A.06 94/01/20 Added support for SMTPSERV and POP2SERV processes to be launched
and left inactive til awakened (greatly speeding up response
time to new connections). Number of server processes for each
is configurable on netmaint control screen
**** ******** MTA PROGRAM
B.09 07/03/22 Fixed to handle mailing lists with entries that are actually
aliases (in ALIASES file)
B.09 03/03/28 Now periodically scans for incomplete mail messages left in
MAIL or MAIL-TO-FORWARD that are older than 24 hours and
deletes them. (Previously ONLY did this check when the job
first started up).
B.08 01/09/05 Added code to support extended logging
B.07 01/05/29 Removed call to routine that scans all mailing-list addresses
cleaning up bad entries whenever a bad mailing list address is
found. DNS lookups on all the hostnames were taking VERY long
times
B.07 00/11/17 Now compare for already-running SMTPOUT hostnames are case-
INsensitive (no more multiple copies of messages when more
than one recipient on a given host had the hostname entered
in different cases)
B.07 99/06/02 Added new trusted-gateway routing option "R"=Y w/no source
routing, and "A"=F w/no source routing
B.07 99/05/17 Can now set limit to max# simultaneous outbound SMTPOUT
processes by :setjcw MAXSMTPOUTS=1..100
B.07 99/02/16 Fixed problem parsing invalid email addresses on mailing lists
which when encountered, caused MTA (and all outbound email) to
lockup
B.07 98/06/24 MTA initializes the poplog file with permanent allowed-relay
IP entries from the file "RELAY.NETMAIL". (If relaypop JCW is
enabled).
B.07 98/05/04 tracing and extlog settings can now be changed while the job is
running (see traceon.xeq/traceoff.xeq/extlog.xeq for examples).
B.07 98/03/18 MTA now deletes any incomplete messages in MAIL or MAIL-TO-FORWA
RD that are older than 2 days each time it starts up
B.07 98/03/04 Bug (below) also fixed for systems using the trusted gateway
flag set to "Y" or "F"
B.07 98/03/02 Local mailboxes forwarded to another mailbox (or list) with
a forward-to-node specified -where the node is the same as the
local-systemname - caused mail to be "lost". It would be
delivered to the mailbox name (even though it should have been
forwarded)
B.06 98/01/23 Now activates SMTP2INT (HPDesk Intrinsic gateway process) for
messages destined for HPDesk via the Intrinsic gateway
B.06 97/11/24 Fixed bug when a mailing list address was a local mailbox which
is forwarded to another mailbox (same name) on another system-
was losing the address
B.06 97/02/04 License file now supports dates>1999.
B.06 97/02/14 Fixed mailing list/mailbox forwarding so mailing lists can
contain other lists/forwarded mailboxes, and forwarded
mailboxes can be forwarded to other mailboxes or mail lists
B.04 96/08/14 Fixed bug where incoming mail to a mailing list, when delivered
to users with 'rules', caused the rules to not get executed.
B.04 96/06/12 Native Mode version
B.03 96/02/27 Fixed problem with hostnames in database in lower case causing
MTA to not launch SMTPOUT process correctly
B.01 95/02/14 Checks for network=DIALUP on outbound mail (host) records - if
found, won't run SMTPOUT to send the message (messages may be
sent outbound manually by running SMTPOUT in a job stream)
**** ******** NETMAINT PROGRAM
B.09 03/02/06 Was failing to display #messages and #bytes for some mailboxes
in mail-user screen
B.09 02/12/04 Now deletes RULES linked to mailbox when you delete the mailbox
B.07 01/08/24 Modified mail-users screen to accept new 'advertise' values of
'C' and 'H'
B.07 99/09/17 Modifed globals screen to accept new 'trusted gateway' codes
of 'R' and 'A'
B.07 98/05/04 Fixed bug in forms file that caused problems when adding
external programs (i.e. editors or spellcheckers)
B.06 98/01/23 Added HPDesk Intrinsic gateway screen for DeskLink users
B.06 97/02/13 "Return To Sender" key on "Individual Message Details"
screen of "Database Information" option.
B.06 97/02/13 New "Database Control" option to schedule purging of old
messages (related to "Aging Days" in Mailbox Maintenance).
B.06 97/02/12 New "FAX" field on Global Default Settings.
B.06 97/02/11 Added new Maintenance option for NetWork statistics.
B.06 97/02/10 Spaces and other special characters are no longer permitted
in mailbox and mailing list names.
B.06 97/02/07 Network-info screen added; keeps online statistics for mail
traffic among networks/the Internet.
B.06 97/02/04 License file now supports dates>1999.
B.05 96/10/16 "forward to user" on users screen can now be set to a local
mailing list.
B.03 96/06/04 Some values were toggled when mailbox renamed
B.03 96/06/04 All mail was lost when mailbox was renamed
B.03 96/04/22 REALNAME was not getting updated correctly when a mailbox's
department changed.
B.03 96/02/10 Fixed problem with DDX (CURRENT capacity was examined vs. MAXIMUM)
B.03 96/02/09 Allow operations on user/maillist names which contain spaces
of hours of retrying before a normal/urgent message is returned
B.03 96/02/05 Added fields in "global" settings screen to configure #
of hours of retrying before a normal/urgent message is returned
B.03 96/01/30 Fixed bug in settings of autolist field in user screen
B.02 95/08/15 Enhanced mailing-lists screen - in particular to handle
lists to be used with a listserver.
B.02 95/07/13 Type-secured field now initialized when list is created;
to '0N' if owner is $PUBLIC or $SYSTEM, else '0Y'.
B.02 95/07/12 1) Can't add/rename a user of same name as a mailing list.
2) Can't add a list of same name as a user.
B.02 95/07/11 Problem with "forward to user": when a user is renamed
any occurrences of user as "forward to user" in another user's
record is also renamed. Also, a user can't be deleted if he
appears as a "forward to user" on another user's record.
B.02 95/07/10 Added "# display lines" to user-default and user screens
B.02 95/06/22 Added new statistical items to known-host screen
B.01 95/03/01 Options added to HPDesk gateway config
B.01 95/03/01 Menu interface option added to user mailbox screens
B.01 95/03/01 If department is specified and it doesn't exist already,
it's added to mail-names but no mail-list entry was added.
A.06 94/07/25 Wasn't allowing program to be run from all users with type=M
(wasn't parsing session names in aliases)
A.06 94/04/19 Mail status detail screen has a 'delete' key to delete a
selected outbound message
A.06 94/01/28 Each Screen Print comes out as a separaate spoolfile now;
no need to exit program before screen would print
A.06 94/01/20 Major renovations; control options split into two screens to
accomodate many new fields
A.06 94/01/20 New options under database status for outbound mail status;
summary and detailed information in separate screens
**** ******** SMTPSERV PROGRAM
B.09 05/35/06 Added Civar MAXMSGSIZE, which if set, globally limits the
number of bytes an incoming message can be (value is the
actual upper limit on #bytes)
B.09 05/23/06 Added check to eliminate empty messages
B.09 05/04/06 Added Civar NETMAIL_INBOUNDHOST which can be set to an IP
address (format 00n.00n.00n.00n - i.e. 192.001.002.010)
and if set will restrict ALL inbound email (SMTP) connections
except those from that IP address
B.09 04/11/05 JCW NOWINDOWSCHARSET now also filters =?UTF-8?, =?KS_C?,
=?ISO-8859-1?, =?KOI8-R? subject lines
B.09 04/10/21 Added NETMAILNOVRFY jcw, which if set (>0) will disable the
SMTP "VRFY" command altogether
B.09 04/05/12 Modified log'msg routine (SMTPINLG/OULG,DESKINLG/OULG) routine
to retry opening log file before giving up (previously it tried
one time, and if the file open failed, data wasn't logged)
B.09 04/04/21 Forced SMTP-FROM="<>" within the Received: header for messages
with a null return path (purportedly bounce/DSN emails)
B.09 04/04/19 Added a check to verify the HELO hostname for forgeries and
bogus names
B.09 04/03/04 Modified errorlog to log DNS hostname of sending system if
available; only log SMTP HELO hostname if no DNS name
B.09 04/03/03 Added checks of sending machine's DNS hostname against the
REFUSE/REFUSEL files
B.09 03/12/23 New JCW NOWINDOWSCHARSET when set in the background job will
now filter all incoming messages with =?windows-####? in the
subject line (i.e. subjects using non-english character set)
Obviously, do not use this filter if you receive messages from
sites using these character sets, though NetMail/3000 does not
support the character set conversion in subject lines anyway;
such messages would only be readable by POP clients that
had the other character sets installed
B.09 03/11/21 Modification to reduce the number of netmail/3000 error message
emails generated caused by routine SMTP/network disconnects
B.09 03/03/27 Fixed a bug in comparing strings in the refuse/includes files
B.09 03/03/25 Added logging to errorlog file of matches in INCLUDES,INCLUDEL,
REFUSE, and REFUSEL (modified arpcat00 required also)
B.09 03/02/10 Wasn't checking SMTP-FROM or From/Sender fields in the message
against the various excludes (domain name checks)
B.09 03/02/05 Delete orphaned(incomplete) message if QUIT received before
entire message processed (some M$ Outlook clients quit at
ANY invalid recipient even after some valid recipients have
been recorded; but before the message contents have been
sent).
B.09 02/07/11 Added several new entries to the extlog file; we now log
info records (start with a "?") in the file, denoting
-when a blocked host connects, a info record is logged
showing that the host is blocked and by which list
-whenever a message id is allocated while processing
messages
-if other exclusion reasons are hit while processing the
message (invalid domain, no reverse dns, forged domains)
B.09 02/07/09 Expanded delays for slowing down blocked mail servers
(includes 1 second delay between each LINE of the smtp
welcome banner, and 10-15 second pauses before returning
any responses) **Only to servers blocked by whatever
real-time blacklists are defined on the site.
Fixed defect that sometimes deleted incoming mail after it
was received and logged
Fixed defect where RCPT TO: responses were affirmative for
local users even when incoming message was going to be
refused as spam(blocked site or domain). Now all users that
utilize blacklists on their mailboxes generate 5xx permanent
errors in reponse to RCPT TOs from spamming machines; hopefully
this will cause some addresses to be removed from future spam
runs.
B.09 02/07/08 Modified all references to EXTLOG so file is opened and closed
around each access, so the file (circular file) can be printed
or copied while the background job is running
B.09 02/06/28 Fixes to exclusion logic so if a host is specifically INCLUDED
(+IP address in excludes.data) any mail will be allowed from
that host, including items that would normally bounce like
no-reverse-dns or forged-domain flagging
Added better refuse messages for remote mailers when mail is
refused for their host not having reverse-dns, or the domain
of their message flagged as forged.
Added a new CIVar to the netmail job, REFUSE_TLD, which
contains a list of country codes. When a server connects
and has reverse-dns that identifies it in that country,
all mail is refused. [Recommended to block korea (.kr),
brazil (.br), argentina (.ar), russia (.ru) and others
where almost ALL outbound mail is spam]
B.09 02/06/05 You can now set VERIFYDOMAINS, which in addition to the
behaviour when =1, now ALSO will cause email from ANY server
that does NOT have any reverse-dns entry to be blocked
CIVAR WK_DOMAINS can be set in the job stream to contain a
list of email domain names. Mail received from any address
in this list of domains (i.e. aol.com, hp.com) MUST come from
a mail server which has a reverse-DNS name which matches this
domain (i.e. spam from joeblow@aol.com coming from anything but
an AOL.COM mail server will be blocked)
B.08 01/10/31 Added internal calls to DBXBEGIN/DBXEND to dynamically rollback
the database state in case the program is aborted
B.07 01/08/24 Blacklist filtering is configurable per mailbox. Globally
define which lists will be used in the job stream as before,
but the "advertise" attribute in the user-mailbox screen now
also defines whether blacklist filtering will be enabled for
that mailbox. Values now are:
Y = advertise (allow queries for this mailbox) AND DON'T use
blacklist filtering
N = don't advertise and DON'T use blacklist filtering
C = advertise AND use blacklist filtering defined in job
H = don't advertise and use blacklist filtering defined in
background job
B.07 01/08/23 All blacklist names are now configurable via CIVars in the
netmail/desklink job stream. Old (hard-coded) names are used
if no user-defined names provided via CIVars. Each name, if
provided MUST begin with a "." and provide the fully qualif-
ied domain suffix to use in DNS lookups. Examples:
!setvar NETMAIL_ORCA_LIST ".dul.orca.bc.ca"
!setvar NETMAIL_MAPS_LIST ".rbl.maps.vix.com"
!setvar NETMAIL_ORBS_LIST ".orbs.dorkslayers.com"
!setvar NETMAIL_IMRSS_LIST ".relays.mail-abuse.org"
!setvar NETMAIL_DSSL_LIST ".dssl.imrss.org"
(these are all the default values if not overridden)
Also, contact URLS can be provided for each of these
blacklists (which are included in mail rejection messages)
to point people to who to contact. It is recommended that
you use these if you completely redirect the services to
different services. Examples:
!setvar NETMAIL_ORCA_CONTACT "www.orbl.org"
!setvar NETMAIL_MAPS_CONTACT "www.ordb.org"
!setvar NETMAIL_ORBS_CONTACT "www.dorkslayers.com"
!setvar NETMAIL_IMRSS_CONTACT "www.mail-abuse.org"
!setvar NETMAIL_DSSL_CONTACT "www.imrss.org"
B.07 01/06/11 ORBS blacklist went offline; new replacement linked in
(orbs.gst-group.co.uk)
B.07 01/03/22 Fixed problem parsing headers in incoming messages; sometimes
parser didn't stop at first blank line and interpreted what
looked like headers inside mail message bodies
B.07 01/03/13 Due to naming changes at ORBS.ORG, updated internal server name
used in ORBS blacklist lookups
B.07 00/03/21 Modified to parse Importance: high headers that M$Exchange uses
for urgent messages
B.07 00/03/09 Modified internal locking strategy to increase performance and
concurrency. Adding records to MESSAGES set now locks on INDEX
vs entire dataset.
B.07 99/12/20 New JCW; throttlepop - if >0, then we force a delay on every
socket read after a message exceeds 1Mb coming in
(delay equal to 'throttlepop' tenths of seconds per read)
B.07 99/09/20 Now interprets MSMail/Exchange's X-MSMAIL-PRIORITY: HIGH header
an equivalent to PRIORITY: URGENT
B.07 99/08/31 Added workaround to trim trailing "]"s off inbound email address
as Exchange likes to do when forwarding messages. Was causing
domain lookups to fail if verifydomains was enabled
B.07 99/06/02 Added new trusted-gateway routing option "R"=Y w/no source
routing, and "A"=F w/no source routing
B.07 99/06/01 Added connection counter (default 300) which will cause each
process to terminate after 300 connections (another process
will automatically take it's place). Added due to some internal
socket resource problems in MPE causing processes to hang after
a number of socket errors (IPC Error 64s)
Can add :SETJCW MAXCONNECTIONS=nnnn to change the limit
B.07 98/11/19 Errorlog file records excluded hosts and the message now tells
you why it was excluded. Error messages returned to the mail
sender also note the source of their exclusion.
B.07 98/11/19 Added ability to select any/none of three different online
dynamic blacklists to filter spam
B.07 98/08/17 Corrected error message that sometimes reported "domain in
refuse file" for invalid (no DNS) domains
B.07 98/08/14 Corrected error message that reported a spam relay-attempt
for e-mail addressed to an invalid HPDesk address
B.07 98/06/25 Fixed problem with SMTP Server processes sometimes "eating"
sockets (I.e. allocating them and not properly closing them out)
B.07 98/06/25 If "relaypop" jcw is set >0, then the mail server will not relay
mail (i.e. accept an incoming message for delivery to ANY non-
local mailbox) UNLESS:
1) the recipient has a local forwarded mailbox
2) the recipient is identified in the ALIASES file w/a
redirected non-local address
3) the sending host's IP address is in the "RELAY.NETMAIL"
file
or
4) the sending host has logged into a pop mailbox (i.e.
successfully checked their mail) on this system within the
last 'relaypop' hours
Any other attempts to relay are met with error messages to the
sending host, entries logged in the errorlog file, and an urgent
mail message sent to the mail administrator
B.07 98/06/10 Added support for "relaypop" jcw; if set (to #hours>0 and <24)
then NO (SMTP) mail messages will be "relayed" unless they are
submitted by a POP mail user that has checked their mail within
the last 'relaypop' hours. This setting completely eliminates
the possibility of your system being used to relay spam by
outside person(s), but if you have POP users, they must get used
to 'checking their mail' before sending mail. This setting does
NOT affect incoming mail to local (or forwarded) recipients or
ANY outbound mail from NetMail users.
B.07 98/06/04 Fixed bug parsing addresses with |