Store Transaction Log Format

From Messaging Server Technical Reference Wiki
Jump to: navigation, search


Message store XML transaction logging is enabled by setting the messagetrace.activate option to transactlog. Both the MTA and store have legacy transaction log formats that are deprecated in favor of the XML format. For a discussion of the MTA XML logging format, see the log_format MTA option.

Each log entry is a self-closing XML entity with a two-letter entity name that contains attributes (also with two-letter names). New attributes may be added and the attributes may be re-ordered in any patch release, so use of an XML-aware parser is recommended. The content of the log is intended to be extended in a backwards-compatible format (with possible exceptions for a major release), unlike the legacy store messagetrace and server log formats which are unstable. Due to the size of transaction log files on busy systems, use of a SAX-style parser is recommended.

To control what actions are logged, use the actions option (unified configuration only). To control what attributes are logged, use the actionattributes option. Note that the MTA uses a different mechanism to control what is logged; see the Transaction logging MTA options section.

XML Log Attributes Always Present in Store Transaction log

The following attributes appear on all Store Transaction Log entries and are thus not mentioned in the event-specific descriptions below:

  • pi - Process id (integer). Note that the MTA uses a different format for this attribute documented in the log_process option.
  • sn - service name (e.g., imap, pop, imquotacheck)
  • ts - time stamp. Both MTA & store use ISO 8601 format as of 8.0.2; but in 8.0 the store used a legacy timestamp format.

XML Log Common Attributes

The following attributes may appear on several XML log entry event types with largely consistent meaning. When these are mentioned in the 'Common' attribute list for an event type, they will be included unless disabled by the actionattributes option.

  • ma - IMAP Mailbox name (internal form)
  • mi - Message id
  • om - Source mailbox for copy/rename (ma is destination mailbox)
  • si - session id (IMAP & POP): a unique integer identifier for a client session/connection.
  • tr - transport information (MTA & store). Prior to 8.0.2, the store only included the client's address and port in this field.
  • us - User name: the canonial authorization user identity (the permanent identity of the primary mail account being accessed). For more information on user identifiers see User Identifiers. Can also be [unauthenticated] when appropriate. For the ac action, the string "Admin" is used when this can't be determined (typically for mboxutil).

XML Log Entity Names and Specific Attributes:

ac - Access Control Change (IMAP only). Attributes include:

  • Common: ma, si, us
  • ao - Old ACL using permanent user identifiers. New in 8.0.2.
  • an - New ACL using permanent user identifiers. New in 8.0.2.
  • nt - Old and new ACL with ':' delimiter (Messaging Server prior to 8.0.2 only).

bm - Big Memory Allocation Event (new in MS 8.0.2.2). Attributes include:

  • nt - Big memory function (malloc, calloc, realloc)
  • sz - Bytes allocated
  • fn - Source filename of allocation
  • ln - Source line number of allocation

cp - Copy Message Event (new in MS 8.0.2.2). Attributes include:

  • Common: ma, om, si
  • mc - number of copied messages
  • sz - total size of copied messages
  • su - source IMAP UID set for copy operation
  • uc - destination IMAP UID set for copy operation
  • uv - IMAP UIDVALIDITY for destination
  • nt - Error message on copy failure (omitted on success, new in MS 8.1.0.5.0)

co - Socket Connection (open/close). Attributes include:

  • Common: si, tr
  • ac - Action code. First letter is 'O' for connection open and 'C' for connection close. Subsequent letters are extensible flags. See MTA transaction log entry format for the meaning of the subsequent flags for the MTA. Subcodes that can be used by the MMP and store include:
    • D - Closed due to DNS RBL
    • I - Closed due to internal/config error
    • L - Closed due to connection limit
    • P - Closed due to broken pipe
    • R - Closed due to connection reset
    • S - Closed due to socket error
    • T - Closed due to timeout
    • W - Closed due to TCP Access wrap filter
    • F - Closed due to force kill, imsconnutil -k
  • at - Store only: will be 'ssl' if SSL was used at connection open time or empty string if SSL was not used.
  • br - bytes received during connection (new in MS 8.0.2).
  • bs - bytes sent during connection (new in MS 8.0.2).
  • fu - flag update page scan count (new in MS 8.0.2.2). A large number indicates potentially significant server CPU consumed by this user's client.
  • nm - number of mailboxes selected during session (new in MS 8.0.2, imap/pop only).
  • nt - In 8.0, contains unstructured information about the connection at connection close. Removed in MS 8.0.2 in favor of separate attributes.
  • rr - Reason connection was rejected (new in 8.0.2.1, MMP only)
  • sb - Search body count (new in MS 8.0.2.2). This counts the number of messages mapped for searching purposes by this user. This does not count searches performed by ISS, DSE or elastic search.
  • sd - Session duration with HHH:MM:SS format (new in MS 8.0.2).
  • td - Time spent on DNS RBL lookups in milliseconds (new in MS 8.0.2.1, MMP only).

ex - Expunge Action (store IMAP expunge/expire). Attributes include:

  • Common: ma, mi, si
  • mc - Messages in mailbox (post-expunge). Prior to MS 8.0.2 this attribute combined me with the pre-expunge message count using a '/' delimiter.
  • me - Messages changed (for ex action, messages expunged). New in MS 8.0.2.
  • mi - Message Id. Note that when this attribute is enabled, a separate expunge log entry is created for each message. If this attribute is not enabled, then only one expunge entry is created for each expunge operation.
  • no - Node name (local host name or remote client IP & port).

fc - Flag Change Action (store +/-flags). New in MS 8.0.2.2.

  • Common: ma, si
  • ac - Action code; one of "S" for set, "C" for clear or "R" for replace.
  • fl - Flag list (space delimited)
  • me - Number of messages changed
  • sq - IMAP modification sequence for this change
  • uc - UIDs changed in IMAP uid set format
  • uv - IMAP UIDVALIDITY for mailbox

fe - Fetch Message Action (POP & IMAP only). Attributes include:

  • Common: ma, mi, si, us
  • fd - Fetch decoding (b64, qp or omitted) (8.0.2 IMAP only)
  • fo - Offset to message part in stored message (8.0.2 IMAP only)
  • fp - Fetch offset into message part (8.0.2 IMAP only)
  • om - Alternate for ma code (8.0.1 POP only)
  • sz - Actual bytes fetched. For 8.0.2 this is a number. For earlier 8.0 versions, this instead contains a string combining: fetch start offset ":" fetch data size or "Binary:" followed by the offset into the message, the offset into the decoded data and the fetch data size (IMAP only).
  • ui - IMAP UID for message (8.0.2 IMAP only)

li - Login/Authenticate Action (store/MMP). Attributes include:

  • Common: si, tr, us
  • ae - Integer authentication error code; see nt for description. Omitted if not known (MMP only, new in MS 8.0.2.1)
  • at - Authentication Type (SASL mechanism name, ssl-port-cert, anonymous or plaintext)
  • bd - badness delay (seconds) before next authentication attempt (MMP only, new in MS 8.0.2.1)
  • cs - Ciphersuite used followed by TLS version. If SSL/TLS is not used, this will be 'noSSL'.
  • nt - Authentication Error or Reply
  • ph - Proxy host name from mailHost or affinity config (MMP only, new in MS 8.0.2.1).
  • pt - Proxy transport information (MMP only, new in MS 8.0.2.1)
  • ua - User authentication identity. This is the user whose password is used to authenticate; which differs from us when administrative proxy authentication is used (new in MS 8.0.2).
  • uo - Original user identity. This is the identity originally specified by the client prior to canonicalization (MMP only, new in MS 8.0.2.1)

lo - Logout action; (POP-only, only if poplogmboxstat is set). Attributes include:

  • Common: si, tr, us
  • ct - Unix timestamp of POP login.
  • mc - Number of messages not marked for deletion.
  • sz - Total bytes in messages not marked for deletion.

ma - Message Append Action. Attributes include:

  • Common: ma, mi, si, us
  • cx - alternate name for session identifier (MS 8.0.1 only).
  • sz - Total bytes in the appended message.
  • ui - IMAP UID for message
  • uv - IMAP UIDVALIDITY for message

is - ISC Convertion Action. Attributes include:

  • Common: us
  • ma - The user-folderId information .
  • ui - IMAP UID for message
  • sz - Total bytes in the message.
  • ez - Total bytes sent to ElasticSearch
  • it - Indexing of document in Elasticsearch response time in microseconds

mc, md, mr - Mailbox Create, Delete, Rename Actions (IMAP only). Attributes include:

  • Common: ma, om, si, us
  • fi - partition name (classic store only)
  • rt - Mailbox Rename duration (introduced in 8.1.0.1)

ms, mu - Mailbox Subscribe, Unsubscribe Actions (IMAP only). Attributes include:

  • Common: ma, si, us
  • fi - namespace (IMAP2bis only)

qc - Quota Change (IMAP only). Attributes include:

  • Common: si, us
  • ur - Quota Root
  • dq - Disk storage quota (number in KB)

qe - Quota Exceeded Action (quotacheck tool only). Attributes include:

  • Common: us
  • dq - Disk storage quota (number in KB)
  • du - Disk storage usage (number in KB)
  • mq - Message count quota (number)
  • mc - Message count used (number)
  • qt - Overquota Trigger (numeric percentage)
  • qr - Quota Rule Name ('General' if not using a rule file)

se - Search, Sort or Thread Mailbox (IMAP only). Attributes include: (new in 8.0.2.2)

  • Common: ma, si
  • mc - Number of matching messages
  • nm - Number of mailboxes searched (only counts local mailboxes when remote shared folders are present).
  • nt - Error message on search failure (omitted on success)
  • sb - Search body count. This counts the number of messages mapped for searching purposes for this search. This does not count searches performed by ISS, DSE or Elasticsearch.
  • sf - Search flags (Q=Message sequence search, U=Uid search, I=iss, D=dse, E=elastic, C=classic, T=Thread, X=Context, S=Sort, M=multi-mailbox search)
  • td - Time passed during search in milliseconds (omitted if 0)

sl - Select Mailbox (IMAP only). Attributes include:

  • Common: ma, si, tr, us


See also: