Job Controller operation

The Job Controller does not process or deliver messages itself, but rather keeps track of message files, and creates and manages channel  jobs to process those messages.

Upon receipt of an incoming message from any source, the MTA channel  that is handling the receipt of the message determines the destination,  enqueues the message, and sends a request to the Job Controller to  execute the next channel. The Job Controller will then initiate a channel job, if one is needed (that is, if there is not such a channel  job already running, or if there are not "enough" jobs for  that channel). Channel jobs, in turn, ask for and receive from the Job Controller the name of which message they should process next. When there are multiple messages to process, a channel process may end up  running for "awhile", in a cycle of  asking the Job Controller for a message  and then processing (delivering) it, and then asking for  another, etc.. And if the number of messages eligible for immediate delivery attempts is sufficiently "high", the Job  Controller will initiate more than one channel job to work in parallel,  each delivering a subset of the messages.  Internally, the Job Controller maintains a data structure of a set of queues of messages awaiting delivery attempts. This data structure is referred to as the queue cache database. New messages are inserted into this data structure sorted onto queues according to their  destination channel, in some cases also sorted by their destination  domain name, and further sorted according to  message processing priority. Additional queues are maintained (one for each destination channel) of those messages that have already had at least one  unsuccessful delivery attempt, and which are waiting for another  delivery attempt.

The Job Controller configuration establishes processing pools;  each such pool has a limit   on how many processes may  execute in it simultaneously (and a pool may optionally be configured  with restrictions on times of day or days of the week in which it may  execute processes). Each channel is constrained (via the   channel option) to run  in one such pool, and may optionally be further constrained on how many  processes it may run simultaneously within the pool. Multiple channels may be configured to run in the same pool, if it is desired for those channels to share (contend  for) the same pool of process slots.

The Job Controller tracks how many processes each channel has running (and in the case of multithreaded channels specifically written to  operate with the Job Controller by letting the Job Controller initiate  delivery threads, the Job Controller tracks how many threads are  running). When there are "enough" messages  eligible for an immediate delivery attempt,  the Job Controller will initiate a new  delivery thread, or whole new delivery process, as needed (if the  configured limits on such jobs have not yet been reached).

The Job Controller will also "cycle" channel jobs, aging out (expiring)  sufficiently old channel jobs  and then creating new channel  jobs (as needed) to take their place. Thus channel jobs, even for busy channels, have a limited life-span. That this is a built-in aspect of the Job Controller both increases robustness in the face of unexpected  problems, and ensures that updates to the MTA configuration, and changes to user  and domain data in LDAP, will propagate through to affect channel jobs  automatically, with bounded delay.

As part of the Job Controller&#x27;s housekeeping and self-maintenance of its internal message queueing data structures, the Job Controller will  periodically do a disk scan of the MTA queue area,  to detect any  message files  omitted from its in-memory queues  and reconcile its  in-memory lists with what is physically present on disk.

So to summarize: the Job Controller&#x27;s primary responsibilities are to maintain internal queues of which messages need delivery attempts and  when, to initiate channel jobs to attempt those deliveries as needed,  and to hand over to channel jobs the name of which message the job  should attempt to process next.

See also:
 * max_life_askwork Option
 * mtpriority_policy MTA Option
 * job_limit Option
 * job_pool
 * pool Option
 * maxjobs Option
 * threaddepth Option
 * max_life_time Option
 * synch_time Option
 * max_cache_messages Option
 * Job Controller operation under stress
 * Job Controller priority-based processing
 * Job Controller
 * Channels