Difference between revisions of "Message tracking and recall setup and configuration"

From Messaging Server Technical Reference Wiki
Jump to: navigation, search
(Bulk update)
 
m (Bulk update)
Line 29: Line 29:
 
Where <ip-address> is the IP address where memcached accepts connections.  
 
Where <ip-address> is the IP address where memcached accepts connections.  
  
Once a server implementing the memcache protocol has been set up, every MTA in the deployment have to be configured to access it. This is done with the <code>memcache_host</code> option; there is no tracking-specific host setting at the present time. The <code>memcache_port</code> if the server is listening on a port other than 11211. It may be appropriate to set the <code>memcache_expire</code> MTA option to an appropriate value as well.  
+
Once a server implementing the memcache protocol has been set up, every MTA in the deployment have to be configured to access it. This is done with the [[memcache_host MTA option#memcache_host|<code>memcache_host</code>]] MTA option; there is no tracking-specific host setting at the present time. The [[memcache_port MTA option#memcache_port|<code>memcache_port</code>]] MTA option must be explicitly set  if the server is listening on a port other than 11211. It may be appropriate to set the [[memcache_expire MTA option#memcache_expire|<code>memcache_expire</code>]] MTA option to an appropriate value as well.  
  
Tracking/recall is enabled within the MTA by setting the <code>tracking_mode</code> MTA option to 1. Logging tracking identifiers is good idea; this is controlled by the <code>log_tracking</code> MTA option.  
+
Tracking/recall is enabled within the MTA by setting the [[tracking_mode MTA option#tracking_mode|<code>tracking_mode</code>]] MTA option to 1. Logging tracking identifiers is good idea; this is controlled by the [[log_tracking MTA option#log_tracking|<code>log_tracking</code>]] MTA option.  
  
 
Taken together, and assuming the memcache server is listening on the default port, the MTA option settings should look something like this:  
 
Taken together, and assuming the memcache server is listening on the default port, the MTA option settings should look something like this:  
 +
 +
<font color="mediumblue">
 +
msconfig> '''show memcache_host'''
 +
role.mta.memcache_host ''memcache-server-ip''
 +
msconfig> '''show tracking_mode'''
 +
role.mta.tracking_mode 1
 +
msconfig> '''show log_tracking'''
 +
role.mta.log_tracking 1
 +
</font>
 +
or in legacy configuration:
 
   
 
   
 
  <font color="mediumblue">         
 
  <font color="mediumblue">         
Line 51: Line 61:
  
 
<li>
 
<li>
  Channels that perform final delivery must be marked with the <code>trackingdelivered</code> channel option. This includes not only ims-ms and tcp_lmtpcs channels, but also the bitbucket and filter_discard channels.  
+
  Channels that perform final delivery must be marked with the [[tracking*, notracking* Channel Options#trackingdelivered|<code>trackingdelivered</code>]] channel option. This includes not only [[ims-ms channels#ims-ms_channels|<code>ims-ms</code>]] and [[LMTP client TCPIP channels#LMTP_client_TCPIP_channels|<code>tcp_lmtpcs</code> channels]], but also the <code>bitbucket</code> and [[filter_discard channel#filter_discard_channel|<code>filter_discard</code>]] channels.  
 
</li>
 
</li>
  
 
<li>
 
<li>
  Any channel that relays messages to systems outside the deployment must be marked with the <code>trackingrelayed</code> channel option. Of course this includes the tcp_local channel, but would also include, say a special tcp_aol channel set up to handle mail to the aol.com domain.  
+
  Any channel that relays messages to systems outside the deployment must be marked with the [[tracking*, notracking* Channel Options#trackingrelayed|<code>trackingrelayed</code>]]  channel option. Of course this includes the <code>tcp_local</code> channel, but would also include, say a special <code>tcp_aol</code> channel set up to handle mail to the aol.com domain.  
 
</li>
 
</li>
  
 
<li>
 
<li>
  Finally, any channel that relays message to other MTAs inside the deployment where tracking is enabled. This would typically be something like a tcp_intranet channel.  
+
  Finally, any channel that relays message to other MTAs inside the deployment where tracking is enabled. This would typically be something like a [[Typical TCPIP channels and servers#Typical_TCPIP_channels_and_servers|<code>tcp_intranet</code> channel]].  
 
</li>
 
</li>
  
Line 80: Line 90:
 
   mtqp-daemon
 
   mtqp-daemon
 
  </font>
 
  </font>
Although somewhat counterintuitive, the "smtp" channel option is required to indicate this is a channel with an associated server. Various optional channel options can also be specified, including:  
+
Although somewhat counterintuitive, the "[[smtp, smtp_cr, smtp_crlf, smtp_crorlf, smtp_lf, nosmtp, lmtp, lmtp_cr, lmtp_crlf, lmtp_crorlf, lmtp_lf Channel Options#smtp|<code>smtp</code>]]" channel option is required to indicate this is a channel with an associated server. Various optional channel options can also be specified, including:  
  
 
<ul>
 
<ul>
Line 148: Line 158:
 
</dl>
 
</dl>
  
Additionally, the server recognizes and will honor the TCP/IP channel-specific options <code>LOG_CONNECTION</code>, <code>TRACE_LEVEL</code>, <code>MAX_THREADS</code>, and <code>IGNORE_BAD_CERT</code>.  
+
Additionally, the server recognizes and will honor the [[TCPIP-channel-specific options#TCPIP-channel-specific_options|TCP/IP channel-specific options]]<code>LOG_CONNECTION</code>, <code>TRACE_LEVEL</code>, <code>MAX_THREADS</code>, and <code>IGNORE_BAD_CERT</code>.  
  
Finally, a dispatcher service definition for the MTQP server is also required:  
+
Finally, a [[Dispatcher#Dispatcher|Dispatcher]]  service definition for the MTQP server is also required:  
 
   
 
   
 
  <font color="mediumblue">         
 
  <font color="mediumblue">         
Line 169: Line 179:
 
* [[memcache_port MTA option#memcache_port|memcache_port MTA Option]]
 
* [[memcache_port MTA option#memcache_port|memcache_port MTA Option]]
 
* [[memcache_expire MTA option#memcache_expire|memcache_expire MTA Option]]
 
* [[memcache_expire MTA option#memcache_expire|memcache_expire MTA Option]]
 +
* [[tracking_mode MTA option#tracking_mode|tracking_mode MTA Option]]
 +
* [[log_tracking MTA option#log_tracking|log_tracking MTA Option]]
 
* [[tracking*, notracking* Channel Options#trackinginternal|trackinginternal Option]]
 
* [[tracking*, notracking* Channel Options#trackinginternal|trackinginternal Option]]
 
[[Category: Reference]]
 
[[Category: Reference]]
 
[[Category: Discussion]]
 
[[Category: Discussion]]

Revision as of 09:52, 24 June 2015


There are three parts to setting up message tracking and recall capabilities:

  1. Setting up a shared memcached server (or other software that implements the memcache protocol in a compatible fashion) to store tracking/recall information.
  2. Configuring all the MTAs in the deployment to enable tracking and recall.
  3. Setting up Message Tracking and Query Protocol (MTQP) servers on every MTA and possibly additional hosts.

Memcache server setup

Setting up memcached or compatible server is beyond the scope of this document. That said, note that mecmached setup is very simple: Configuration consists of a small number of options on the command line. Indeed, it's often possible to simply say:

        
  memcached -l <ip-address>

Where <ip-address> is the IP address where memcached accepts connections.

Once a server implementing the memcache protocol has been set up, every MTA in the deployment have to be configured to access it. This is done with the memcache_host MTA option; there is no tracking-specific host setting at the present time. The memcache_port MTA option must be explicitly set if the server is listening on a port other than 11211. It may be appropriate to set the memcache_expire MTA option to an appropriate value as well.

Tracking/recall is enabled within the MTA by setting the tracking_mode MTA option to 1. Logging tracking identifiers is good idea; this is controlled by the log_tracking MTA option.

Taken together, and assuming the memcache server is listening on the default port, the MTA option settings should look something like this:


msconfig> show memcache_host
role.mta.memcache_host memcache-server-ip
msconfig> show tracking_mode
role.mta.tracking_mode 1
msconfig> show log_tracking
role.mta.log_tracking 1

or in legacy configuration:

        
  MEMCACHE_HOST=<memcache-server-ip>
  TRACKING_MODE=1
  LOG_TRACKING=1

MTA Channel Setup

The next step is to tell the tracking subsystem the semantics of the various channels in the deployment using the trackinginternal, trackingrelayed, and trackingdelivered. The general rules are:

  1. Internal processing channels such as process, reprocess, conversion, and defragment require no special decoration.
  2. Channels that perform final delivery must be marked with the trackingdelivered channel option. This includes not only ims-ms and tcp_lmtpcs channels, but also the bitbucket and filter_discard channels.
  3. Any channel that relays messages to systems outside the deployment must be marked with the trackingrelayed channel option. Of course this includes the tcp_local channel, but would also include, say a special tcp_aol channel set up to handle mail to the aol.com domain.
  4. Finally, any channel that relays message to other MTAs inside the deployment where tracking is enabled. This would typically be something like a tcp_intranet channel.

Note that for tracking and recall to work messages relayed to external systems MUST be handled by a different channel than message relayed to other MTAs inside the deployment. Meeting this requirement may require additional configuration changes.

The MTRK SMTP extension defined in RFC 3885 is used to transfer tracking/recall information from one MTA to another. Use of this extension MUST be enabled on SMTP connections between MTAs inside the deployment. It also must be enabled on SUBMIT channels used by tracking/recall-enabled clients.

Since MTRK is an SMTP extension, its use is negotiated by the SMTP client and server. So the simplest way to activate this extension is to put the trackingclient and trackingserver on the defaults channel. However, if you wish to avoid use of this extension with systems outside the deployment, a more targeted approach is needed: Place trackingclient on every tcp_* channel that has trackinginternal set. Then place trackingserver on every channel that accepts messages from other hosts within the deployment. (Note that once again this may require configuration changes to separate internal and external traffic.)

MTQP server setup

An extended version of the MTRP protocol specified in RFC 3887 is used to perform tracking and recall operations. If only tracking is desired a single MTQP server can be used for the entire deployment and can be run on any host. However, in order to recall messages an MTQP server must be running on every host with an MTA, and if "total recall" is enabled MTQP servers are required on all store hosts as well.

The MTQP server leverages existing MTA facilities which require a channel. Note that the MTQP channel doesn't sent or receive messages and has no rewrite rules or associated queue. The specification of such a channel is very simple:

        
  mtqp smtp
  mtqp-daemon

Although somewhat counterintuitive, the "smtp" channel option is required to indicate this is a channel with an associated server. Various optional channel options can also be specified, including:

  • maytlsserver or musttlsserver are used to control whether SSL/TLS is allowed or required, respectively, before tracking/recall operations may be performed.
  • slave_debug enables MTQP server debugging.
  • maysaslserver may be specified to enable user authentication. User authentication is required in order to use the MTRACK and/or MRECALL commands.
  • disconnectbadauthlimit, disconnectbadcommandlimit, and disconnectcommandlimit have their usual meanings.

Some MTQP-server-specific options are also available:

TOTAL_RECALL (boolean 0 or 1, default 0)
If set, this option enables recall of messages from the store. Note that if this is used it will result in messages being deleted from recipient's folder irrespective of whether or not they have been read. Use of this option should only be undertaken with a full understanding of the possible consequences.
AUTH_DEBUG (debug token list, default "")
This option is used to specify authentication debugging tokens.
COMMAND_TIMEOUT (integer seconds, default 60)
This option specifies the time the server will wait for a command before disconnecting.
STATUS_TIMEOUT (integer seconds, default 60)
This option specifies the time the server will wait for a status write to the client before disconnecting.
CUSTOM_BANNER_STRING (string, default "")
This option specifies the name the server uses to announce itself.

Additionally, the server recognizes and will honor the TCP/IP channel-specific optionsLOG_CONNECTION, TRACE_LEVEL, MAX_THREADS, and IGNORE_BAD_CERT.

Finally, a Dispatcher service definition for the MTQP server is also required:

        
  [SERVICE=MTQP]
  PORT=1038
  IMAGE=IMTA_BIN:mtqp
  LOGFILE=IMTA_LOG:mtqp_server.log
  STACKSIZE=2048000

See also: