|
3k Associates NetMail/3000 Engine Product enhancement report Updated: 03/22/2007 PROGRAM FIXES/ENHANCEMENTS Fixed Date in Code Description ---- -------- ---------------------------------------------------------------- **** ******** ARPA PROGRAM B.09 05/04/13 Added JCW NETMAILSMTPPORTIN which allows sites to change the tcp port# NetMail/3000 & DeskLink listen on for incoming mail connections, and NETMAILSMTPPORTOUT which sets the TCP port SMTPOUT will use for OUTBOUND mail connections (default -if not specified- is the SMTP standard port 25) B.09 04/08/11 Fixed bug in quote-of-day service, so messages will wrap after the last message in the catalog is reached B.09 04/03/25 Added ARPANODAYTIME and ARPANOQOTD JCWs; when set to>0, they disable the activation of the DAYTIME and Quote-of-the-Day services (respectively) B.09 03/12/29 Fixed bug when extended logging enabled, file gets re-opened repeatedly (til process runs out of descriptors) B.08 01/09/05 Added code to support extended logging B.07 00/08/04 Program now quits (parm=24) if it encounters an error on startup indicating that network transport wasn't started B.07 00/05/09 New JCW ARPANOCHARGEN if set to '1' will disable the chargen service normally started by ARPA (chargen isn't needed by us, it was just provided as a useful test service, but has become a target of denial-of-service attacks) B.07 99/11/16 Modified quote-of-the-day to only return one unique quote per day (max; i.e. if queried several times per day, you'll get the same quote til the next day - or next day it's queried) B.07 98/10/30 Fixed problem where ARPA wouldn't run for systems where only the API was licensed. B.07 98/10/15 Recoding tremendously increased responsiveness when multiple/ rapid-fire connection requests come in. Should solve problems some POP sites experienced with heavy loads and users getting timeouts B.07 98/10/13 We now override the default maximum of 7 "queued" connection requests for incoming network connections. Set to 255. May help sites with heavy loads experiencing timeout or connect- ion refused errors. B.07 98/04/17 We now check for a file called "NONETWRK.PUB.THREEK" which,if present, will ensure that no network (smtp/pop/finger/etc) services get started up. B.07 98/03/12 Changed DESKUSEGATEWAY jcw to allow a value of 3. B.06 98/01/23 Added support for INT2SMTP process on startup (DeskLink Intrinsic gateway) B.06 98/01/23 Added "DESKUSEGATEWAY" jcw to determine which (or both) HPDesk gateway process(es) to use B.06 97/12/08 Quote-of-the-day now 'remembers' the last quote issued; it no longer resets each time the job is stopped/started B.06 97/12/04 Added CRLF to end of "quote of the day" messages sent out by the server B.06 97/11/21 Program now recovers from recurring IOWait CCL errors; these used to disable further network connections. IOWait problem with MPE recurred in MPE/iX 5.5 PP3. B.06 97/05/09 Refused connections (from refuse/excludes files) are now logged to the errorlog.netmail file B.06 97/02/14 New SEPARATESTDLISTS jcw in job stream causes netmail/desklink job to create separate stdlist spoolfiles for each son process B.06 97/02/04 License file now supports dates>1999. B.05 96/11/14 Shuts down much quicker; kills idle processes in parallel now rather than in sequence (as before) B.05 96/09/05 Obeys the number of pop2 and pop3 servers (from global config) B.04 96/08/02 No longer tries to start up gopher servers if product not licensed B.04 96/06/12 Native Mode version B.03 96/05/16 New jcw - if there is networking software available, but it's not configured (or user doesn't want to use it for some reason) then add !SETJCW ARPANONETWORK=1 to the netmail job before the !RUN ARPA.SYS.THREEK line. B.03 96/03/12 If mailman job is accidentally run in the THREEK account (instead of HPOFFICE) and DeskLink is licensed then the program prints an error message in $stdlist and to console then stops. B.03 96/02/27 Fixed problem with IOWAIT errors when 3000 gets a tcp request from a system it can't get back to (route problems in nmmgr) arpa no longer aborts; and new requests can continue B.02 95/11/02 Now looks for ARPAQUEUE jcw on startup - if =0 (or not set) ARPA automatically puts itself in the CS queue. If =1, then it puts itself in DS, and if =2 it puts itself in ES. B.02 95/06/20 Added support for running FINGSERV finger server program B.02 95/06/20 Added support for an "excludes.data.threek" file where the admin can list ip addresses which will (+nnn.nnn.nnn.nnn)or will NOT (-nnn.nnn.nnn.nnn) be allowed to make any incoming connection (SPAM prevention and added security) (Addresses NOT listed are allowed to connect by default) B.02 95/06/20 Fixed code so that if gopher database was not initialized, then we won't start any gophserv processes B.02 95/06/20 Changed connecting socket# to print as an octal number instead of a signed integer B.01 95/06/01 Added date/time display to $STDLIST for socket connections B.01 94/11/04 Fixed lockup problem on MPE/iX 5.0 (server suspends and never awakens) A.06 94/08/30 Workaround for bug in MPE/iX 4.5 that caused SMTPSERV processes to be created but not re-used A.06 94/01/24 Enhanced code so printing to slaved printers ($LOCAL) will automatically eject the page for laser printers or other printers that normally buffer output; also for reflection users that used to have to manually hit the "close printer" button A.06 94/01/20 Added support for SMTPSERV and POP2SERV processes to be launched and left inactive til awakened (greatly speeding up response time to new connections). Number of server processes for each is configurable on netmaint control screen **** ******** MTA PROGRAM B.09 07/03/22 Fixed to handle mailing lists with entries that are actually aliases (in ALIASES file) B.09 03/03/28 Now periodically scans for incomplete mail messages left in MAIL or MAIL-TO-FORWARD that are older than 24 hours and deletes them. (Previously ONLY did this check when the job first started up). B.08 01/09/05 Added code to support extended logging B.07 01/05/29 Removed call to routine that scans all mailing-list addresses cleaning up bad entries whenever a bad mailing list address is found. DNS lookups on all the hostnames were taking VERY long times B.07 00/11/17 Now compare for already-running SMTPOUT hostnames are case- INsensitive (no more multiple copies of messages when more than one recipient on a given host had the hostname entered in different cases) B.07 99/06/02 Added new trusted-gateway routing option "R"=Y w/no source routing, and "A"=F w/no source routing B.07 99/05/17 Can now set limit to max# simultaneous outbound SMTPOUT processes by :setjcw MAXSMTPOUTS=1..100 B.07 99/02/16 Fixed problem parsing invalid email addresses on mailing lists which when encountered, caused MTA (and all outbound email) to lockup B.07 98/06/24 MTA initializes the poplog file with permanent allowed-relay IP entries from the file "RELAY.NETMAIL". (If relaypop JCW is enabled). B.07 98/05/04 tracing and extlog settings can now be changed while the job is running (see traceon.xeq/traceoff.xeq/extlog.xeq for examples). B.07 98/03/18 MTA now deletes any incomplete messages in MAIL or MAIL-TO-FORWA RD that are older than 2 days each time it starts up B.07 98/03/04 Bug (below) also fixed for systems using the trusted gateway flag set to "Y" or "F" B.07 98/03/02 Local mailboxes forwarded to another mailbox (or list) with a forward-to-node specified -where the node is the same as the local-systemname - caused mail to be "lost". It would be delivered to the mailbox name (even though it should have been forwarded) B.06 98/01/23 Now activates SMTP2INT (HPDesk Intrinsic gateway process) for messages destined for HPDesk via the Intrinsic gateway B.06 97/11/24 Fixed bug when a mailing list address was a local mailbox which is forwarded to another mailbox (same name) on another system- was losing the address B.06 97/02/04 License file now supports dates>1999. B.06 97/02/14 Fixed mailing list/mailbox forwarding so mailing lists can contain other lists/forwarded mailboxes, and forwarded mailboxes can be forwarded to other mailboxes or mail lists B.04 96/08/14 Fixed bug where incoming mail to a mailing list, when delivered to users with 'rules', caused the rules to not get executed. B.04 96/06/12 Native Mode version B.03 96/02/27 Fixed problem with hostnames in database in lower case causing MTA to not launch SMTPOUT process correctly B.01 95/02/14 Checks for network=DIALUP on outbound mail (host) records - if found, won't run SMTPOUT to send the message (messages may be sent outbound manually by running SMTPOUT in a job stream) **** ******** NETMAINT PROGRAM B.09 03/02/06 Was failing to display #messages and #bytes for some mailboxes in mail-user screen B.09 02/12/04 Now deletes RULES linked to mailbox when you delete the mailbox B.07 01/08/24 Modified mail-users screen to accept new 'advertise' values of 'C' and 'H' B.07 99/09/17 Modifed globals screen to accept new 'trusted gateway' codes of 'R' and 'A' B.07 98/05/04 Fixed bug in forms file that caused problems when adding external programs (i.e. editors or spellcheckers) B.06 98/01/23 Added HPDesk Intrinsic gateway screen for DeskLink users B.06 97/02/13 "Return To Sender" key on "Individual Message Details" screen of "Database Information" option. B.06 97/02/13 New "Database Control" option to schedule purging of old messages (related to "Aging Days" in Mailbox Maintenance). B.06 97/02/12 New "FAX" field on Global Default Settings. B.06 97/02/11 Added new Maintenance option for NetWork statistics. B.06 97/02/10 Spaces and other special characters are no longer permitted in mailbox and mailing list names. B.06 97/02/07 Network-info screen added; keeps online statistics for mail traffic among networks/the Internet. B.06 97/02/04 License file now supports dates>1999. B.05 96/10/16 "forward to user" on users screen can now be set to a local mailing list. B.03 96/06/04 Some values were toggled when mailbox renamed B.03 96/06/04 All mail was lost when mailbox was renamed B.03 96/04/22 REALNAME was not getting updated correctly when a mailbox's department changed. B.03 96/02/10 Fixed problem with DDX (CURRENT capacity was examined vs. MAXIMUM) B.03 96/02/09 Allow operations on user/maillist names which contain spaces of hours of retrying before a normal/urgent message is returned B.03 96/02/05 Added fields in "global" settings screen to configure # of hours of retrying before a normal/urgent message is returned B.03 96/01/30 Fixed bug in settings of autolist field in user screen B.02 95/08/15 Enhanced mailing-lists screen - in particular to handle lists to be used with a listserver. B.02 95/07/13 Type-secured field now initialized when list is created; to '0N' if owner is $PUBLIC or $SYSTEM, else '0Y'. B.02 95/07/12 1) Can't add/rename a user of same name as a mailing list. 2) Can't add a list of same name as a user. B.02 95/07/11 Problem with "forward to user": when a user is renamed any occurrences of user as "forward to user" in another user's record is also renamed. Also, a user can't be deleted if he appears as a "forward to user" on another user's record. B.02 95/07/10 Added "# display lines" to user-default and user screens B.02 95/06/22 Added new statistical items to known-host screen B.01 95/03/01 Options added to HPDesk gateway config B.01 95/03/01 Menu interface option added to user mailbox screens B.01 95/03/01 If department is specified and it doesn't exist already, it's added to mail-names but no mail-list entry was added. A.06 94/07/25 Wasn't allowing program to be run from all users with type=M (wasn't parsing session names in aliases) A.06 94/04/19 Mail status detail screen has a 'delete' key to delete a selected outbound message A.06 94/01/28 Each Screen Print comes out as a separaate spoolfile now; no need to exit program before screen would print A.06 94/01/20 Major renovations; control options split into two screens to accomodate many new fields A.06 94/01/20 New options under database status for outbound mail status; summary and detailed information in separate screens **** ******** SMTPSERV PROGRAM B.09 05/35/06 Added Civar MAXMSGSIZE, which if set, globally limits the number of bytes an incoming message can be (value is the actual upper limit on #bytes) B.09 05/23/06 Added check to eliminate empty messages B.09 05/04/06 Added Civar NETMAIL_INBOUNDHOST which can be set to an IP address (format 00n.00n.00n.00n - i.e. 192.001.002.010) and if set will restrict ALL inbound email (SMTP) connections except those from that IP address B.09 04/11/05 JCW NOWINDOWSCHARSET now also filters =?UTF-8?, =?KS_C?, =?ISO-8859-1?, =?KOI8-R? subject lines B.09 04/10/21 Added NETMAILNOVRFY jcw, which if set (>0) will disable the SMTP "VRFY" command altogether B.09 04/05/12 Modified log'msg routine (SMTPINLG/OULG,DESKINLG/OULG) routine to retry opening log file before giving up (previously it tried one time, and if the file open failed, data wasn't logged) B.09 04/04/21 Forced SMTP-FROM="<>" within the Received: header for messages with a null return path (purportedly bounce/DSN emails) B.09 04/04/19 Added a check to verify the HELO hostname for forgeries and bogus names B.09 04/03/04 Modified errorlog to log DNS hostname of sending system if available; only log SMTP HELO hostname if no DNS name B.09 04/03/03 Added checks of sending machine's DNS hostname against the REFUSE/REFUSEL files B.09 03/12/23 New JCW NOWINDOWSCHARSET when set in the background job will now filter all incoming messages with =?windows-####? in the subject line (i.e. subjects using non-english character set) Obviously, do not use this filter if you receive messages from sites using these character sets, though NetMail/3000 does not support the character set conversion in subject lines anyway; such messages would only be readable by POP clients that had the other character sets installed B.09 03/11/21 Modification to reduce the number of netmail/3000 error message emails generated caused by routine SMTP/network disconnects B.09 03/03/27 Fixed a bug in comparing strings in the refuse/includes files B.09 03/03/25 Added logging to errorlog file of matches in INCLUDES,INCLUDEL, REFUSE, and REFUSEL (modified arpcat00 required also) B.09 03/02/10 Wasn't checking SMTP-FROM or From/Sender fields in the message against the various excludes (domain name checks) B.09 03/02/05 Delete orphaned(incomplete) message if QUIT received before entire message processed (some M$ Outlook clients quit at ANY invalid recipient even after some valid recipients have been recorded; but before the message contents have been sent). B.09 02/07/11 Added several new entries to the extlog file; we now log info records (start with a "?") in the file, denoting -when a blocked host connects, a info record is logged showing that the host is blocked and by which list -whenever a message id is allocated while processing messages -if other exclusion reasons are hit while processing the message (invalid domain, no reverse dns, forged domains) B.09 02/07/09 Expanded delays for slowing down blocked mail servers (includes 1 second delay between each LINE of the smtp welcome banner, and 10-15 second pauses before returning any responses) **Only to servers blocked by whatever real-time blacklists are defined on the site. Fixed defect that sometimes deleted incoming mail after it was received and logged Fixed defect where RCPT TO: responses were affirmative for local users even when incoming message was going to be refused as spam(blocked site or domain). Now all users that utilize blacklists on their mailboxes generate 5xx permanent errors in reponse to RCPT TOs from spamming machines; hopefully this will cause some addresses to be removed from future spam runs. B.09 02/07/08 Modified all references to EXTLOG so file is opened and closed around each access, so the file (circular file) can be printed or copied while the background job is running B.09 02/06/28 Fixes to exclusion logic so if a host is specifically INCLUDED (+IP address in excludes.data) any mail will be allowed from that host, including items that would normally bounce like no-reverse-dns or forged-domain flagging Added better refuse messages for remote mailers when mail is refused for their host not having reverse-dns, or the domain of their message flagged as forged. Added a new CIVar to the netmail job, REFUSE_TLD, which contains a list of country codes. When a server connects and has reverse-dns that identifies it in that country, all mail is refused. [Recommended to block korea (.kr), brazil (.br), argentina (.ar), russia (.ru) and others where almost ALL outbound mail is spam] B.09 02/06/05 You can now set VERIFYDOMAINS, which in addition to the behaviour when =1, now ALSO will cause email from ANY server that does NOT have any reverse-dns entry to be blocked CIVAR WK_DOMAINS can be set in the job stream to contain a list of email domain names. Mail received from any address in this list of domains (i.e. aol.com, hp.com) MUST come from a mail server which has a reverse-DNS name which matches this domain (i.e. spam from joeblow@aol.com coming from anything but an AOL.COM mail server will be blocked) B.08 01/10/31 Added internal calls to DBXBEGIN/DBXEND to dynamically rollback the database state in case the program is aborted B.07 01/08/24 Blacklist filtering is configurable per mailbox. Globally define which lists will be used in the job stream as before, but the "advertise" attribute in the user-mailbox screen now also defines whether blacklist filtering will be enabled for that mailbox. Values now are: Y = advertise (allow queries for this mailbox) AND DON'T use blacklist filtering N = don't advertise and DON'T use blacklist filtering C = advertise AND use blacklist filtering defined in job H = don't advertise and use blacklist filtering defined in background job B.07 01/08/23 All blacklist names are now configurable via CIVars in the netmail/desklink job stream. Old (hard-coded) names are used if no user-defined names provided via CIVars. Each name, if provided MUST begin with a "." and provide the fully qualif- ied domain suffix to use in DNS lookups. Examples: !setvar NETMAIL_ORCA_LIST ".dul.orca.bc.ca" !setvar NETMAIL_MAPS_LIST ".rbl.maps.vix.com" !setvar NETMAIL_ORBS_LIST ".orbs.dorkslayers.com" !setvar NETMAIL_IMRSS_LIST ".relays.mail-abuse.org" !setvar NETMAIL_DSSL_LIST ".dssl.imrss.org" (these are all the default values if not overridden) Also, contact URLS can be provided for each of these blacklists (which are included in mail rejection messages) to point people to who to contact. It is recommended that you use these if you completely redirect the services to different services. Examples: !setvar NETMAIL_ORCA_CONTACT "www.orbl.org" !setvar NETMAIL_MAPS_CONTACT "www.ordb.org" !setvar NETMAIL_ORBS_CONTACT "www.dorkslayers.com" !setvar NETMAIL_IMRSS_CONTACT "www.mail-abuse.org" !setvar NETMAIL_DSSL_CONTACT "www.imrss.org" B.07 01/06/11 ORBS blacklist went offline; new replacement linked in (orbs.gst-group.co.uk) B.07 01/03/22 Fixed problem parsing headers in incoming messages; sometimes parser didn't stop at first blank line and interpreted what looked like headers inside mail message bodies B.07 01/03/13 Due to naming changes at ORBS.ORG, updated internal server name used in ORBS blacklist lookups B.07 00/03/21 Modified to parse Importance: high headers that M$Exchange uses for urgent messages B.07 00/03/09 Modified internal locking strategy to increase performance and concurrency. Adding records to MESSAGES set now locks on INDEX vs entire dataset. B.07 99/12/20 New JCW; throttlepop - if >0, then we force a delay on every socket read after a message exceeds 1Mb coming in (delay equal to 'throttlepop' tenths of seconds per read) B.07 99/09/20 Now interprets MSMail/Exchange's X-MSMAIL-PRIORITY: HIGH header an equivalent to PRIORITY: URGENT B.07 99/08/31 Added workaround to trim trailing "]"s off inbound email address as Exchange likes to do when forwarding messages. Was causing domain lookups to fail if verifydomains was enabled B.07 99/06/02 Added new trusted-gateway routing option "R"=Y w/no source routing, and "A"=F w/no source routing B.07 99/06/01 Added connection counter (default 300) which will cause each process to terminate after 300 connections (another process will automatically take it's place). Added due to some internal socket resource problems in MPE causing processes to hang after a number of socket errors (IPC Error 64s) Can add :SETJCW MAXCONNECTIONS=nnnn to change the limit B.07 98/11/19 Errorlog file records excluded hosts and the message now tells you why it was excluded. Error messages returned to the mail sender also note the source of their exclusion. B.07 98/11/19 Added ability to select any/none of three different online dynamic blacklists to filter spam B.07 98/08/17 Corrected error message that sometimes reported "domain in refuse file" for invalid (no DNS) domains B.07 98/08/14 Corrected error message that reported a spam relay-attempt for e-mail addressed to an invalid HPDesk address B.07 98/06/25 Fixed problem with SMTP Server processes sometimes "eating" sockets (I.e. allocating them and not properly closing them out) B.07 98/06/25 If "relaypop" jcw is set >0, then the mail server will not relay mail (i.e. accept an incoming message for delivery to ANY non- local mailbox) UNLESS: 1) the recipient has a local forwarded mailbox 2) the recipient is identified in the ALIASES file w/a redirected non-local address 3) the sending host's IP address is in the "RELAY.NETMAIL" file or 4) the sending host has logged into a pop mailbox (i.e. successfully checked their mail) on this system within the last 'relaypop' hours Any other attempts to relay are met with error messages to the sending host, entries logged in the errorlog file, and an urgent mail message sent to the mail administrator B.07 98/06/10 Added support for "relaypop" jcw; if set (to #hours>0 and <24) then NO (SMTP) mail messages will be "relayed" unless they are submitted by a POP mail user that has checked their mail within the last 'relaypop' hours. This setting completely eliminates the possibility of your system being used to relay spam by outside person(s), but if you have POP users, they must get used to 'checking their mail' before sending mail. This setting does NOT affect incoming mail to local (or forwarded) recipients or ANY outbound mail from NetMail users. B.07 98/06/04 Fixed bug parsing addresses with |