Cnbuild utility

From Messaging Server Technical Reference Wiki
Jump to: navigation, search

Legacy configuration: Compile the MTA configuration, alias, mapping, conversion, system wide filter, circuit check, and option files, and various configutil parameters, as well as (optionally) the reverse "database", general "database", and forward "database", and also (as of MS 7.0) the Job Controller configuration file, the Dispatcher configuration file, and TCP/IP channel option files; generate an image file (suitable for memory mapping by MTA processes).

Unified Configuration: Compile confdesc.xml as well as (optionally) the reverse "database", general "database", and forward "database"; generate an image file (suitable for memory mapping by MTA processes).

Syntax

  imsimta cnbuild

imsimta cnbuild Command Switches
Switch Default
-image_file=file-spec -image_file=IMTA_TABLE:advanced/config_data
-maximum -nomaximum
-option_file=file-spec -option_file=IMTA_TABLE:option.dat
-remove -noremove
-sizes -nosizes
-statistics -nostatistics
-check -nocheck
-synonyms -nosynonyms
-xml_config -noxml_config

Restrictions

Must have superuser privileges, or be logged in as the MTA user (see the user option in restricted.cnf) in order to use this utility.

Description

With a legacy configuration, the cnbuild utility compiles the textual configuration, option, mapping, conversion, system wide filter, circuit check, and alias files, and various configutil parameters (see Basic configuration settings relevant to alias LDAP lookups and Basic configuration settings relevant to domain LDAP lookups), and (depending upon the setting of the use_text_databases MTA option) also optionally the reverse "database", general "database", and forward "database", and (as of MS 7.0) also the Job Controller configuration file, Dispatcher configuration file, and TCP/IP channel option files (for SMTP channels or LMTP client channels), and generates an image file (suitable for memory mapping by MTA processes). With a Unified Configuration, most of these files have been consolidated: when using a Unified Configuration, the cnbuild utility compiles the confdesc.xml file, and (depending upon the setting of the use_text_databases MTA option) also optionally the the reverse "database", general "database", and forward "database", and generates an image file (suitable for memory mapping by MTA processes). The resulting image is the file formerly named by the imta_config_data option of the MTA tailor file; as of MS 7.0.5, such MTA tailor file options have been replaced by consistent locations, and the image file is simply IMTA_TABLE:advanced/config_data.

Whenever a component of the MTA (e.g., a channel program) must read any possibly compiled configuration component, it first checks to see whether a compiled configuration image file exists, and if so, the running program uses that image. There are five exceptions to this rule. The first is imsimta cnbuild itself, which for obvious reasons always reads the text files and never tries to use an image form of the configuration data. The remaining four exceptions are imsimta test -domain_map, imsimta test -rewrite, imsimta test -mapping, and imsimta test -x400 which can all be instructed with the -image_file switch to use a different compiled configuration file. This facility in imsimta test -rewrite is useful for testing changes prior to compiling them.

One reason for compiling configuration information is simple: performance. Precompiling the MTA configuration allows MTA processes to quickly load that precompiled configuration and then get immediately to work, rather than each new MTA process needing to read the MTA configuration and build the in-memory representation of the configuration itself, before starting its own work. Note that MTA design includes short-lived, transient processes (e.g., most channel delivery job processes), as well as long-lived processes (e.g., Dispatcher and Job Controller), so avoiding the overhead of each MTA process, short-lived or not, repeating the same configuration-processing work reduces overhead.

A second, and nowadays often more important, reason for compiling configuration information is for convenience in configuration management and updating: due to the fact that when a compiled configuration exists the MTA processes will normally refer to it preferentially, the MTA administrator can make changes to the underlying configuration files at leisure, and optionally perform some testing of such changes (e.g., using a -noimage_file switch with test utilities) prior to compiling the updated configuration and having the changes become "live"; meantime, the MTA continues running with the older, compiled configuration.

Once you begin to use a compiled configuration, it will be necessary to recompile the configuration every time changes are made to the settings and files comprising the source of the compiled configuration. For a Unified Configuration, this is primarily confdesc.xml (normally modified using the msconfig utility); if the use_text_databases MTA option has been set, then also potentially the reverse "database" replacement text file, the general "database" replacement text file, and the forward "database" replacement text file. Specifically, these are the files

  • CONFIGROOT/confdesc.xml,
  • CONFIGROOT/reverse.txt,
  • CONFIGROOT/general.txt, and
  • CONFIGROOT/forward.txt.

For a legacy configuration, this includes various relevant configutil parameters (see Basic configuration settings relevant to alias LDAP lookups and Basic configuration settings relevant to domain LDAP lookups) and the following files: the MTA configuration file (or any files referenced by it, such as internet.rules), the MTA system alias file, the MTA mapping file, the MTA option file, the MTA conversion file, system wide filter file, or the circuit check configuration file; if the use_text_databases MTA option has been set, then also the reverse "database" replacement text file, the general "database" replacement text file, the forward "database" replacement text file; as of MS 7.0, also the Job Controller configuration file, the Dispatcher configuration file, and TCP/IP channel option files (affecting SMTP channels and LMTP client channels). Specifically, these are the files which, prior to MS 7.0.5, were pointed at by the MTA Tailor file options imta_config_file, imta_alias_file, imta_mapping_file, imta_option_file, imta_conversion_file, imta_system_filter_file, imta_reverse_data, imta_general_data, imta_forward_data, respectively, or nowadays are simply:

  • CONFIGROOT/imta.cnf,
  • CONFIGROOT/aliases,
  • CONFIGROOT/mappings,
  • CONFIGROOT/option.dat,
  • CONFIGROOT/conversions,
  • CONFIGROOT/imta.filter,
  • CONFIGROOT/reverse.txt,
  • CONFIGROOT/general.txt, and
  • CONFIGROOT/forward.txt,

as well as the file CONFIGROOT/circuitcheck.cnf.

Until such time as the configuration is recompiled, changes to any of these files will not be visible to the MTA. Furthermore, even after recompiling the configuration, note that changes to the MTA compiled configuration will not be seen by running MTA processes unless and until the MTA configuration is reloaded via the imsimta reload utility, or until the process expires and is replaced by a new process. In the case of transient, short-lived processes, (e.g., most channel delivery jobs) their own natural quick expiration may lead to them being replaced by new processes soon enough that any lag in making use of the new MTA configuration is relatively unimportant; however, for significant changes impacting the Job Controller, or the Dispatcher or its server processes (SMTP server processes, SMTP SUBMIT server processes, LMTP server processes), after performing imsimta cnbuild also consider performing imsimta reload, when the change needs to take effect "immediately" and imsimta reload will suffice. (Note that imsimta reload only updates certain parts of the precompiled MTA configuration; in particular, it does not reload the channel definitions and rewrite rules portions of the MTA configuration as they are complex and intertwined and require a full re-read of the configuration. Some additional cases of configuration changes relevant to the Job Controller can instead be modified "live", without requiring a restart of the Job Controller, via the imsimta cache -change utility.) A full restart of the MTA (especially restarting the Job Controller) should be avoided on production systems unless truly necessary as a restart interrupts and disrupts message delivery; however, a imsimta restart * to restart the various servers underneath the Dispatcher causes only brief disruption in acceptance of incoming messages, so is acceptable when enqueue processing changes beyond those covered by imsimta reload need to take effect "immediately" upon incoming messages.

Note that updates to the tlds.txt file (introduced in MS 7.0.5) do not require use of imsimta cnbuild to take effect; instead, updates to tlds.txt are incorporated by use of the chbuild utility.

See Compiling the MTA configuration for further details on the use of compiled configurations.

If imsimta cnbuild spots a syntactic error in one of the files that it is attempting to use to build the compiled configuration, it will be reported, typically as an error of the form

time-stamp: Error in mm_init -- detail

where the detail provides additional, specific detail about the particular error. For instance (with output wrapped here for display convenience; in reality it would appear all on one line):


09:28:26.15: Error in mm_init -- duplicate host in channel table -- host.domain. 
com -- line #45 in file IMTA_CONFIG_FILE 

Note that imsimta cnbuild does not know specifics of the syntax and semantics of the Dispatcher configuration file or Job Controller configuration file, or channel option files, so generally errors in the option settings in those files---other than egregiously, obviously broken syntax problems---will not be reported by the utility and instead will get reported by the component in question when it attempts to begin running.

As of MS 6.3p1, errors compiling the configuration (in particular, mm_init errors) will cause imsimta cnbuild to exit with a non-zero status. Prior to that version, while such errors would certainly cause imsimta cnbuild to exit with error text and without making a new compiled configuration, the exit status was nevertheless zero in many cases.

There is also a secondary, "tuning" use of imsimta cnbuild. When the MTA is building an in-memory representation of its configuration (whether that is to generate a compiled configuration, or whether in the absence of any compiled configuration MTA processes are each reading the MTA configuration and building their own private representations of the MTA configuration), it starts by assuming certain table sizes, and if those table sizes are not adequate, iteratively increases those sizes until reaching "big enough" tables in memory to hold the current configuration. The starting points for such table sizes are controlled by internal size MTA options . Using imsimta cnbuild with its -statistics switch provides information on how "close" the current internal size MTA options are to the sizes needed for the current MTA configuration, and using imsimta cnbuild with the set of switches -noimage_file -maximum -option_file will generate a new MTA option file with internal size MTA options set appropriately for the current MTA configuration. This sort of tuning permits more efficient compilation of the MTA configuration---though it is not strictly necessary (since the MTA will interatively resize, if necessary, on-the-fly while reading its configuration).

Since the MTA will resize its internal table sizes as needed, errors about exceeding table sizes are normally seen only if the MTA's more-or-less "hard" limits on resizing are reached. (The limits are established by the maximum.dat file and/or "hard" limits in the code.) And since the MTA's "hard" limits are very generous, exceeding the limits is usually an indication of either a configuration error of a type that has confused the MTA about the intended meaning of certain configuration inputs (for instance, an extraneous blank line in the rewrite rules, causing the MTA to attempt to interpret all remaining material as channel definitions), or configuration choices involving poor use of MTA facilities that would be better handled in an alternate manner (such as attempting to hard code many thousands of mapping table entries, rather than using a few general entries that do general database callouts> for the specific fields). In particular, reaching the limits specified in the normal maximum.dat file is usually an indication of poor configuration choices; you should contact Oracle if you believe you wish to exceed those limits, as you may be better served by alternate configuration tactics.

Finally, a command such as


# imsimta cnbuild -noimage_file -option_file=file-spec

may be used to list current option values (as specified in the current MTA option file, plus default values) in the file specified as file-spec. To list (most) default option values, irrespective of override values specified in the current MTA option file, the current MTA option file must be "moved aside" before the above command is issued. (For instance, temporarily rename the MTA option file, then issue the above command, then rename the MTA option file back to its normal location.)

Switches

-check, -nocheck (default)

-checkmeans to check the structure of the confdesc.xml file.

-image_file[=file-spec], -noimage_file

By default, imsimta cnbuild creates as output the image file named CONFIGROOT/advanced/config_data; (prior to MS 7.0.5, the file located via the imta_config_data MTA Tailor option). With the -image_file switch, an alternate file name may be specified. When the -noimage_file switch is specified, imsimta cnbuild does not produce an output image file. This switch is used in conjunction with the -⁠option_file switch to produce as output an option file which specifies table sizes adequate to hold the configuration required by the processed input files.

-maximum, -nomaximum (default)

When -maximum is specified, the file SERVERROOT/lib/maximum.dat is read in addition to the internal size MTA options (which in legacy configuration would be stored in the MTA option.dat file, located prior to MS 7.0.5 via the imta_option_file MTA Tailor file option). The maximum.dat file specifies near maximum table sizes but does not change any other MTA option settings. Only use the -maximum switch if the current table sizes are inadequate. The -noimage_file and -option_file switches should always be used in conjunction with this switch---it makes no sense to output the enormous configuration that is produced by -maximum, but it does make sense to use -maximum to get past size restrictions in order to set properly adjusted MTA option values (in legacy configuration, build an MTA option file with properly adjusted option values) so that a properly sized configuration can be built with a subsequent imsimta cnbuild invocation.

-option_file[=file-spec], -nooption_file (default)

imsimta cnbuildcan optionally set various internal size MTA options (in legacy configuration, produce an option file that contains correct table sizes) to hold the configuration that was just compiled (plus a little room for growth). The -option_file switch causes this file to be output. By default, this file is named CONFIGROOT/option.dat (or prior to MS 7.0.5, was located via the imta_option_file MTA Tailor file option). The value on the -option_file switch may be used to specify an alternate file name. If the -⁠nooption_file switch is given, then no option file will be output. imsimta cnbuild always reads any MTA options previously set, whether set as Unified Configuration MTA options or set in the MTA option file, CONFIGROOT/option.dat; use of the -⁠nooption_file switch will not alter this behavior. However, use of the -maximum switch causes imsimta cnbuild to also read MTA options from the file CONFIGROOT/advanced/maximum.dat This maximum.dat file specifies near maximum table sizes. Only use the -maximum switch if the current table sizes are inadequate, and only use it to generate new MTA option settings (set either as new values for MTA options in Unified Configuration, or by building a new option.dat file in legacy configuration). The -noimage_file switch should always be specified when -maximum is specified since a maximum-size image would be truly enormous and extremely wasteful.

-remove, -noremove (default)

The -remove switch causes removal of any existant compiled configuration; i.e., removes the file CONFIGROOT/advanced/config_data (located prior to MS 7.0.5 via the imta_config_data MTA Tailor file option).

-sizes, -nosizes (default)

The -sizes switch instructs imsimta cnbuild to output information on the sizes of uncompiled MTA tables.

-statistics, -nostatistics (default)

The -statistics switch instructs imsimta cnbuild to output information on how much of the various tables in the compiled configuration were actually used to store data. These numbers give a rough measurement of the efficiency of the compilation, and may indicate whether or not an additional rebuild with the -option_file switch is needed. Specifying -⁠statistics effectively forces -noimage_file.

-synonyms, -nosynonyms (default)

-synonymsmeans to output a list of MTA option and channel option synonyms; that is, output a list of the MTA options and channel option marked as aliases in confdesc.xml:


spamfilter_config_file -> spamfilter1_config_file 
brightmail1_config_file -> spamfilter1_config_file 
brightmail_config_file -> spamfilter1_config_file 
...additional aliased MTA options...
733 -> percents 
822 -> sourceroute 
brightmail -> sourcespamfilter1 
channelfilter -> destinationfilter 
...additional aliased channel keywords...

-xml_config, -noxml_config (default)

-xml_configdirects cnbuild to compile a Unified Configuration; -noxml_config directs cnbuild to compile a legay configuration.

Examples

#1

# imsimta cnbuild
# #     Depending upon what has changed, consider issuing:
# # imsimta reload
# # imsimta restart *

This is the standard command used on UNIX to regenerate a compiled configuration. After compiling the configuration, restart any programs which may need to reload the new configuration; e.g., the SMTP server should be restarted. (Note that the * character may need shell-quoting so that it will be passed through the shell properly to the MTA utility.)

#2

# imsimta cnbuild -noimage_file -option_file -maximum
# imsimta cnbuild

Use these two UNIX commands when you encounter the infamous "No room in table" MTA error message.


See also: