Sending DSNs MDNs Alt Channel
From MsgServerDocWiki
How do I send DSNs and MDNs by using an alternate channel?
This article describes how to configure an MTA to send delivery status notifications (DSNs) and message disposition notifications (MDNs) through an alternate channel and why it is worth configuring.
Contents |
[edit] Problem description
A typical problem which happens to be seen with customers is that messages are not delivered out to the Internet "fast enough", this means that messages are stored in the outgoing queue before they have a chance to get delivered, because all threads which deliver messages are busy. One can configure the MTA for more outgoing processes and threads, but this sometimes helps only for a short term.
Typically legitimate messages which are sent by authenticated users to their recipients in the internet are delivered fast: the recipients' addresses are correct, there exist no delivery problems etc. But it may happen, that these messages do not get a chance to get delivered immediately, as all delivery threads are busy on attempting to deliver messages, which are more difficult to get delivered.
These messages are delivery status notifications. They are generated by the MTA itself, upon failure to deliver messages to local users. If an MTA accepts a message for delivery, it is mandated by the relevant standards, to either deliver a message, or to return a failure notification to a message sender if it can't do so.
Therefore, the MTA should make the best effort to not to accept messages which it knows it will not deliver, but it can always happen that it accepts messages, and cannot deliver them. Sending delivery notifications can be problematic these days if these are responses to spam messages, because very often senders of spam messages are in fake domains, which have MXes (as per the DNSs) which cannot be connected to. The MTA will spend time trying to connect to such servers for minutes, utilizing threads which should be delivering legitimate mail instead. When it fails to connect there, it will treat this situation as temporary, and will queue these erroneous messages for later delivery, and if the backoff on a channel is set for frequent retries, the problematic behavior will repeat often.
This problem may be noticed especially in places which have set the grace period feature (which is set by default BTW), and do not discard spam immediately. They will have a lot of notifications about spam messages not being able to be delivered because of users being over quota.
While the DSN generation must not and cannot be switched off (it is mandated by relevant standards, and there is some percentage of legitimate DSNs), there are means to limit influence of problems with delivering DSNs to other types of messages, especially those sent by authenticated users. These means are to configure the MTA to send DSNs via a separate outgoing channel, and associate that channel with a separate process pool. That process pool will be busy trying to deliver the notifications, while the SMTP_POOL will be available for the legitimate mail. If problems described above occur, they will apply to DSNs only, while the users will not notice delays in their communication.
[edit] Configuration on a single MTA, without caring about the DSNs coming from the backend
The MTA can be reconfigured to send DSNs it generates via a separate channel. There are several pieces of configuration to achieve that.
[edit] New rewrite rules
There is a new rewrite rule needed for each top-level domain, so that DSNs to these domains are rewritten to the tcp_notify channel, which will be created in the subsequent steps. These rules can be best put to a separate file (in my case, internet-dsn.rules), and they are along the lines of:
.AC $U%$H$D@tcp_notify-daemon$Mprocess_dsn .AD $U%$H$D@tcp_notify-daemon$Mprocess_dsn .AE $U%$H$D@tcp_notify-daemon$Mprocess_dsn .AF $U%$H$D@tcp_notify-daemon$Mprocess_dsn .AG $U%$H$D@tcp_notify-daemon$Mprocess_dsn .AI $U%$H$D@tcp_notify-daemon$Mprocess_dsn
internet.rules can be used as a template.
[edit] Changes to imta.cnf
To configure the DSNs to be queued to an alternate channel, one can put the following keywords in the defaults:
defaults .. notificationchannel process_dsn dispositionchannel process_dsn
(the same channel for both DSNs and MDNs is used, to simplify the configuration). And create the process_dsn chanel:
process_dsn process_dsn-daemon
One needs to create a new channel, let's call it tcp_notify which will be used to deliver the DSNs to the internet (in imta.cnf):
tcp_notify smtp mx single_sys remotehost inner subdirs 20 maxjobs 10 pool NOTIFY_POOL backoff "pt4h" "pt8h" notices 1 loopcheck tcp_notify-daemon
there are a few interesting things in this definition:
- there is quite rare delivery retry interval (4 hours, and then every 8 hours). This is reasonable because if a DSN is not delivered for the first time, it is unlikely to be delivered at all. There is no point of retrying it often, especially a DSN delivery attempt can consume a thread for a long time
- similarly, there is no point of keeping the undeliverable DSNs in the queue too long. They will be kept for one day only
- this channel will use a separate process pool NOTIFY_POOL. The actual number of processes (maxjobs and job_limit, these values should be equal as nothing else uses that pool), depends on the capabilities of the machine.
- loopcheck should be enabled on this channel, as MX records for spammers' domains sometimes point at 127.0.0.1
Finally, the rewrite rules available in internet-dsn.rules need to be imported:
<IMTA_TABLE:internet-dsn.rules
That line must be put to imta.cnf before the one which imports internet.rules
[edit] Changes to job_controller.cnf
The process pool needs to be configured in the job_controller.cnf:
[POOL=NOTIFY_POOL] job_limit=10
[edit] Changes to tcp_notify_option
One can also decrease some protocol level timeouts for the tcp_notify channel. As it has been told, it is likely that the MTA cannot connect to MX machines for spammers' domains, so it does not make sense to attempt connecting for too long (and hence "locking" threads). For instance, one could set:
STATUS_MAIL_RECEIVE_TIME=1
so that the server waits for only one minute for a successful connection to a remote host. As the configuration file is related to the tcp_notify only, other connections will not be affected.
[edit] Two-tier deployment, DSNs generated on the backend should be associated with the tcp_notify on the front-end, too
If LMTP is used for mailbox delivery, vast majority of DSNs (including those related to overquota users) are generated on the front-end, so it is not necessary to recognize DSNs generated on the backend. If, however, SMTP is used, such DSNs will be generated on the back-end, too and when relayed to the front-end, they will be recognized as legitimate mail from users. It is better, that the MTA can recognize the DSNs coming from the backend as if they were DSNs it generates itself, and associate them with tcp_notify so that the same characteristics are applied, and so that legitimate mail is not affected if the number of such DSNs is high.
The trick to achieve this is, to configure the back-end so that it sends DSNs through its own tcp_notify channel, and configure that channel to connect to an alternate port on the front-end. The front-end would associate that port with its own tcp_notify channel and send messages arriving via that port as if they were its own DSNs.
The architecture in case of the two-tier deployment is presented on the figure below (machines are blue, channels are dark blue). The back-end system (on the left) either sends DSNs generated in the process_dsn channel to the front-end, port 2025, through its tcp_notify channel, or sends other mail to the front-end, port 25, through tcp_local. The front-end uses tcp_intranet as a source channel form the mail from the back-end if it comes through port 25, or uses tcp_notify if mail comes on port 2025. That mail, along with DSNs generated by the front-end in process_dsn, is relayed to the Internet via its tcp_notify channel.
[edit] The back-end configuration
One needs to configure the back-end as described above (in the "Configuration on a single MTA, without caring about the DSNs coming from the backend" section). the tcp_notify should differ in two things
- there should be daemon front-end-domain-or-ip put on tcp_notify, just like in tcp_local, so that all mail is relayed through the front-end
- tcp_notify should be configured with port 2025 (2025 is the alternate port on the front-end)
[edit] The front-end configuration
The front-end should be configured like was described in the "Configuration on a single MTA, without caring about the DSNs coming from the backend" section, with following additions:
[edit] A new dispatcher instance
There should be dispatcher instance configured to listen on port 2025 in dispatcher.cnf, and it should be associated with the tcp_nofity channel.
[SERVICE=SMTP] PORT=2025 IMAGE=IMTA_BIN:tcp_smtp_server LOGFILE=IMTA_LOG:tcp_smtp_server.log STACKSIZE=2048000 PARAMETER=CHANNEL=tcp_notify
[edit] Rewrite rules
One more set of reqrite rules is needed, so that mail coming through tcp_notify is also sent with tcp_notify. These rules should look along the lines of:
.AC $U%$H$D@tcp_notify-daemon$Mtcp_notify .AD $U%$H$D@tcp_notify-daemon$Mtcp_notify .AE $U%$H$D@tcp_notify-daemon$Mtcp_notify .AF $U%$H$D@tcp_notify-daemon$Mtcp_notify .AG $U%$H$D@tcp_notify-daemon$Mtcp_notify .AI $U%$H$D@tcp_notify-daemon$Mtcp_notify
again, internet.rules can be used as a pattern. The rules can be put to a separate file (say, internet-notify.rules), which should be included in imta.cnf:
<IMTA_TABLE:internet-notify.rules
(again, before the inclusion of internet.rules)
[edit] Permissions on connecting to the alternate port
Whoever can connect to port 2025 on the MTA, can send mail to the Internet. While it may be disallowed for the Internet users, one may also want to configure the MTA to allow only the back-end to connect to that port, to prevent malicious local users, who are able to connect to port 2025, from relaying mail and thus bypassing whatever access control is set on port 25.
Such permissions can be set in mappings, in the PORT_ACCESS mapping table:
*|*|2025|*|* $C$|BE_IP;$2|$Y$E *|*|2025|*|* $N
where BE_IP is a new mapping table which defines the IP numbers which can connect to port 2025, for instance:
BE_IP 10.1.4.6 $Y 10.1.4.7 $Y 127.0.0.1 $Y * $N
Answered by: User:Wchem
Date initially answered: May 5, 2007
Categories: FAQ | MTA
