Difference between revisions of "Sieve redis extension"

From Messaging Server Technical Reference Wiki
Jump to: navigation, search
m (Bulk update)
m (Bulk update)
 
Line 2: Line 2:
 
__NOTOC____NOEDITSECTION__
 
__NOTOC____NOEDITSECTION__
  
As of the 8.0.2.3 release, the MTA supports a private "<code>redis</code>" operator, which may be used to perform both actions and tests. (Configuration of MTA connections to redis is required.) This nonstandard extension is available without any special capability declaration; no "<code>require</code>" action is needed for its use.  By default, the "<code>redis</code>" operator may be used in any Sieve script, but its availability may be selectively disabled via the [[enable_sieve_redis MTA option#enable_sieve_redis|<code>enable_sieve_redis</code>]] MTA option. This extension is provided so that Sieve filters can access and manipulate data using the redis protocol.  
+
As of the 8.0.2.3 release, the MTA supports a private "<code>redis</code>" operator, which may be used to perform both actions and tests. (Configuration of MTA connections to Redis is required.) This nonstandard extension is available without any special capability declaration; no "<code>require</code>" action is needed for its use.  By default, the "<code>redis</code>" operator may be used in any Sieve script, but its availability may be selectively disabled via the [[enable_sieve_redis MTA option#enable_sieve_redis|<code>enable_sieve_redis</code>]] MTA option. This extension is provided so that Sieve filters can access and manipulate data using the redis protocol.  
  
 
The various Sieve "<code>redis</code>" operations share a set of common  nonpositional parameters:  
 
The various Sieve "<code>redis</code>" operations share a set of common  nonpositional parameters:  
Line 110: Line 110:
 
   
 
   
 
   redis :add &#x5b;:tableprefix prefix-string&#x5d; &#x5b;:duplicate&#x5d;
 
   redis :add &#x5b;:tableprefix prefix-string&#x5d; &#x5b;:duplicate&#x5d;
         &#x5b;:timeout timeout-number&#x5d; &#x5b;:set&#x5d; &#x5b;:sortedset&#x5d;
+
         &#x5b;:timeout timeout-number&#x5d; &#x5b;:set &#x7c; :sortedset&#x5d;
 
         &#x5b;:score score&#x5d; key-string value-string
 
         &#x5b;:score score&#x5d; key-string value-string
 
   
 
   
Line 163: Line 163:
 
  if redis :adjustup 1 "entry" "10" { ... handle entry greater than 10 ... }
 
  if redis :adjustup 1 "entry" "10" { ... handle entry greater than 10 ... }
 
  </font>
 
  </font>
redis "<code>:adjustup</code>" operations are performed on reevaluations but the adjustment amount is changed to <code>0</code>.  "<code>:duplicate</code>" causes the adjustment amount to be retained on reevaluations.  
+
Redis "<code>:adjustup</code>" operations are performed on reevaluations but the adjustment amount is changed to <code>0</code>.  "<code>:duplicate</code>" causes the adjustment amount to be retained on reevaluations.  
 
   
 
   
 
   
 
   
Line 171: Line 171:
 
Append the specified value to the entry with the specified key. The entry must already exist. Returns TRUE if the operation is successful, FALSE if it is not.  
 
Append the specified value to the entry with the specified key. The entry must already exist. Returns TRUE if the operation is successful, FALSE if it is not.  
  
redis "<code>:append</code>" operations are skipped and return success unconditionally on reevaluations by default. "<code>:duplicate</code>" forces the operation to be performed on all reevaluations.  
+
Redis "<code>:append</code>" operations are skipped and return success unconditionally on reevaluations by default. "<code>:duplicate</code>" forces the operation to be performed on all reevaluations.
 +
 +
 +
  redis :expire &#x5b;:tableprefix prefix-string&#x5d;
 +
        &#x5b;:duplicate&#x5d; key-string
 +
 +
Sets the expiration time in seconds for the entry with the specified key. A time of 0 will cause the entry to be be deleted immediately. Negative values will cause the entry to be preserved indefinitely.Remove the entry with the specified key. Returns TRUE if the operation is successful, FALSE if it is not. Note that the expiration time for sets and sorted sets can be set without specifying any additional parameters.
 +
 
 +
Redis "<code>:expire</code>" operations are skipped and return success unconditionally on reevaluations by default. "<code>:duplicate</code>" forces the operation to be performed on all reevaluations.  
 
   
 
   
 
   
 
   
 
   redis :fetch &#x5b;:tableprefix prefix-string&#x5d;
 
   redis :fetch &#x5b;:tableprefix prefix-string&#x5d;
         &#x5b;MATCH-TYPE&#x5d; &#x5b;COMPARATOR&#x5d; &#x5b;:set&#x5d; &#x5b;:sortedset&#x5d;
+
         &#x5b;MATCH-TYPE&#x5d; &#x5b;COMPARATOR&#x5d; &#x5b;:set &#x7c; :sortedset&#x5d;
 
         &#x5b;:withscores&#x5d; key-string &#x5b;test-string&#x5d;
 
         &#x5b;:withscores&#x5d; key-string &#x5b;test-string&#x5d;
 
   
 
   
Line 195: Line 203:
 
         &#x5b;:timeout timeout-value&#x5d;  key-string
 
         &#x5b;:timeout timeout-value&#x5d;  key-string
 
   
 
   
Implements a generic reservation capability, where a specified key-string can be repeatedly reserved up to a specified quota. :release releases a reservation previously created by :reserve. The associated redis entry contains a record of how many reservations are held by each process on the host. Outstanding reservations associated with nonexistent processes will be automatically deteled.  
+
Implements a generic reservation capability, where a specified key-string can be repeatedly reserved up to a specified quota. :release releases a reservation previously created by :reserve. The associated Redis entry contains a record of how many reservations are held by each process on the host. Outstanding reservations associated with nonexistent processes will be automatically deteled.  
  
 
This operation always functions as a test, returning TRUE if the reservatation has been releaves succesfully, FALSE otherwise.  
 
This operation always functions as a test, returning TRUE if the reservatation has been releaves succesfully, FALSE otherwise.  
  
redis reservation operations are performed on reevaluations.  
+
Redis reservation operations are performed on reevaluations.  
 
   
 
   
 
   
 
   
Line 234: Line 242:
 
   
 
   
 
   redis :store &#x5b;:tableprefix prefix-string&#x5d;
 
   redis :store &#x5b;:tableprefix prefix-string&#x5d;
         &#x5b;:timeout timeout-value&#x5d; &#x5b;:set&#x5d; &#x5b;:sortedset&#x5d;
+
         &#x5b;:timeout timeout-value&#x5d; &#x5b;:set &#x7c; :sortedset&#x5d;
 
         &#x5b;:score score&#x5d; key-string value-string
 
         &#x5b;:score score&#x5d; key-string value-string
 
   
 
   

Latest revision as of 07:26, 17 May 2020


As of the 8.0.2.3 release, the MTA supports a private "redis" operator, which may be used to perform both actions and tests. (Configuration of MTA connections to Redis is required.) This nonstandard extension is available without any special capability declaration; no "require" action is needed for its use. By default, the "redis" operator may be used in any Sieve script, but its availability may be selectively disabled via the enable_sieve_redis MTA option. This extension is provided so that Sieve filters can access and manipulate data using the redis protocol.

The various Sieve "redis" operations share a set of common nonpositional parameters:

:tableprefix prefix-string
Unlike MeterMaid, each redis server provides a single storage area; in effect a single "table". Multiple tables can be implemented by adding a prefix to the key. ":tableprefix" provides a convenient way to specify such a prefix while making it clear that the string is a prefix and not logically part of the key.
:timeout timeout-number
Specifies the timeout value in seconds for the entry being created or updated. The default timeout value is 0, which means the entry never times out.
:quota quota-int
Specifies the maximum number of counts to permit during the quota timeout period. This corresponds to MeterMaid throttle table quota setting.
:quotatimeout quota-timeout-int
Specifies the duration in seconds of the period over which counts are recorded. This corresponds to MeterMaid throttle table quota_time setting.
:penalize
Corresponds to MeterMaid throttle table option penalize setting.
:limit
Normally :throttle operations update the throttle even when the throttle engages; this is appropriate when throttling an external activity that occurs outside the control of Messaging Server. The addition of :limit causes a :throttle test to only update when the throttle does not engage; this is appropriate when a throttle is being used to limit some internal activity that doesn't occur when the throttle engages. Any operation failure is also handled in a failsafe manner when :limit is specified - server down, network failure, wrong data type, etc. - to return TRUE rather than the usual FALSE.
:set
New in MS 8.1.0.3. Specifies that the data associated with the key is a Redis set.
:sortedset
New in MS 8.1.0.3. Specifies that the data associated with the key is a Redis sorted set.
:score score-number
New in MS 8.1.0.3. Specifies the score for an entry in a Redis sorted set. The default score is 0.
:duplicate
It may be necessary to evaluate Sieves more than once, e.g., when a message is sent to multiple recipients and the Sieve employs an envelope "to" test. Additionally, even if a particular Sieve script does not contain such a test, it may nevertheless be necessary to reevaluate it if a previous script was reevaluated and that action changed one or more of the inputs to the current script.
It is undesirable in most cases to reevaluate redis actions that perform increment, decrement, or append operations. As such, most redis operations are skipped when a script is reevaluated. (The exact behavior is noted in each section below.)
However, there are cases, e.g., when using recipient-specific data, when reevaluation is the correct action. The ":duplicate" nonpositional parameter provides this capability: if specified, the associated redis operation will be performed on both initial and subsequent evaluations.
MATCH-TYPE and COMPARATOR
See RFC 5228 for information about MATCH-TYPE and COMPARATOR parameters.

The redis Sieve extension provides the following operations:


  redis :add [:tableprefix prefix-string] [:duplicate]
        [:timeout timeout-number] [:set | :sortedset]
        [:score score] key-string value-string

Add a new entry with the specified key, value and timeout. The operation only suceeds if the entry does not already exist. The operation returns TRUE if the entry is added successfully, FALSE if not. If :set or :sortedset are specified add the specified value to the set. The set will be created if it doesn't already exist. A score of 0 is used for entries assed to sorted sets if no other value is specified.

For example:


      if not redis :add "a" "b" { ... handle failure ... }

redis ":add" operations are skipped and return success unconditionally on script reevaluations by default. ":duplicate" forces the operation to be performed on all reevaluations.


  redis :adjustdown adjustment-value [:tableprefix prefix-string]
        [:duplicate] [MATCH-TYPE] [COMPARATOR]
        [:quotatimeout quotatimeout-numeric [:penalize]]
        [:timeout timeout-value] [:sortedset] key-string [test-string]

Decrement the entry with the specified key by the specified adjustment value. The entry must contain an unsigned decimal string, The adjustment value can be either an integer or a string; if it is a string it must contain an optionally signed decimal value. Negative adjustment value are allowed.

The specified entry will be created (with a value of 0) prior to adjustment if it doesn't already exist. The timeout value is only used if the entry has to be created.

Redis ":adjustdown" operations can be applied to throttle entries. If this is done, the appropriate quota parameters must be specified in case the entry needs to be created.

New in MS 8.1.0.3, Redis ":adjustdown" operations can be applied to members of sorted sets by specifying the :sortedset parameter. If this is done test-string is interpreted as the name of the set member.

The adjusted value is returned as an unsigned decimal string if neither MATCH-TYPE, COMPARATOR, nor test-string are specified. However, if any of these parameters are specified this operation functions as a test with updated value of the entry being the tested value. MATCH-TYPE and COMPARATOR default to :value "lt" and "i;ascii-numeric" respectively. For example:


      if redis :adjustdown 1 "entry" "10" {
        ... handle entry less than 10 ... }

redis ":adjustdown" operations are performed on reevaluations but the adjustment amount is changed to 0. ":duplicate" causes the adjustment amount to be retained on reevaluations.


  redis :adjustup adjustment-value [:tableprefix prefix-string]
        [:duplicate] [MATCH-TYPE] [COMPARATOR]
        [:quotatimeout quotatimeout-numeric [:penalize]]
        [:timeout timeout-value] [:sortedset] key-string [test-string]

Increment the entry with the specified key by the specified adjustment value. The entry must contain an unsigned decimal string. The adjustment value can be either an integer or a string; if it is a string it must contain an optionally signed decimal value. Negative adjustments values are allowed.

The specified entry will be created (with a value of 0) prior to adjustment if it doesn't already exist. The timeout value is only used if the entry has to be created.

Redis ":adjustup" operations can be applied to throttle entries. If this is done, the appropriate quota parameters must be specified in case the entry needs to be created.

New in MS 8.1.0.3, Redis ":adjustup" operations can be applied to members of sorted sets by specifying the :sortedset parameter. If this is done test-string is interpreted as the name of the set member.

The adjusted value is returned as an unsigned decimal string if if neither MATCH-TYPE, COMPARATOR, nor test-string are specified. However, if any of these parameters are specified this operation functions as a test with updated value of the entry being the tested value. MATCH-TYPE and COMPARATOR default to :value "gt" and "i;ascii-numeric" respectively. For example:


if redis :adjustup 1 "entry" "10" { ... handle entry greater than 10 ... }

Redis ":adjustup" operations are performed on reevaluations but the adjustment amount is changed to 0. ":duplicate" causes the adjustment amount to be retained on reevaluations.


  redis :append [:tableprefix prefix-string]
        [:duplicate] key-string value-string

Append the specified value to the entry with the specified key. The entry must already exist. Returns TRUE if the operation is successful, FALSE if it is not.

Redis ":append" operations are skipped and return success unconditionally on reevaluations by default. ":duplicate" forces the operation to be performed on all reevaluations.


  redis :expire [:tableprefix prefix-string]
        [:duplicate] key-string

Sets the expiration time in seconds for the entry with the specified key. A time of 0 will cause the entry to be be deleted immediately. Negative values will cause the entry to be preserved indefinitely.Remove the entry with the specified key. Returns TRUE if the operation is successful, FALSE if it is not. Note that the expiration time for sets and sorted sets can be set without specifying any additional parameters.

Redis ":expire" operations are skipped and return success unconditionally on reevaluations by default. ":duplicate" forces the operation to be performed on all reevaluations.


  redis :fetch [:tableprefix prefix-string]
        [MATCH-TYPE] [COMPARATOR] [:set | :sortedset]
        [:withscores] key-string [test-string]

Fetches the value of the entry with the specified key.

The entry's value is simply returned as a string if neither MATCH-TYPE, COMPARATOR, nor test-string are specified. An empty string will be returned if the specified entry doesn't exist.

However, if any of the three parameters are specified this operation functions as a test with current value of the entry being the tested value. MATCH-TYPE and COMPARATOR default to :is and "i;ascii-casemap" respectively. The test always fails if the entry does not exist. This can be to test for the existence of an entry:


if redis :fetch :matches "entry" "*" { ... entry exists ... }

The members of Redis sets and sorted sets may be fetched by specifying the corresponding parameter. Multiple members are returned as lists. If :withscores is specified member-score pairs are returned.

redis ":fetch" operations are simply repeated on reevaluations.


  redis :release [:duplicate] [:tableprefix prefix-string]
        [:timeout timeout-value]  key-string

Implements a generic reservation capability, where a specified key-string can be repeatedly reserved up to a specified quota. :release releases a reservation previously created by :reserve. The associated Redis entry contains a record of how many reservations are held by each process on the host. Outstanding reservations associated with nonexistent processes will be automatically deteled.

This operation always functions as a test, returning TRUE if the reservatation has been releaves succesfully, FALSE otherwise.

Redis reservation operations are performed on reevaluations.


  redis :remove [:tableprefix prefix-string]
        [:duplicate] key-string

Remove the entry with the specified key. Returns TRUE if the operation is successful, FALSE if it is not. Note that sets and sorted sets can be removed without specifying any additional parameters.

redis ":remove" operations are skipped and return success unconditionally on reevaluations by default. ":duplicate" forces the operation to be performed on all reevaluations.


  redis :replace [:tableprefix prefix-string]
        [:timeout timeout-value] key-string value-string

Update the value and timeout of an entry. The entry must already exist. Returns TRUE if the operation is successful, FALSE if it is not.

redis ":replace" operations are simply repeated on reevaluations.


  redis :reserve :quota quota-numeric [:duplicate]
        [:tableprefix prefix-string]
        [:timeout timeout-value]  key-string

Implements a generic reservation capability, where a specified key-string can be repeatedly reserved up to a specified quota. Each reserve operation must be matched by a corresponding release. The associated redis entry contains a record of how many reservations are held by each process on the host. Outstanding reservations associated with nonexistent processes will be automatically deteled.

Note that since there is no server-side awareness of entry semantics the quota parameter must be specified in every reserve call in case the entry needs to be created. If the entry already exists the quota parameter value will be checked against the corresponding value stored in the entry.

Each reserve operation increments the reservation count by 1, and should be matched by a corresponding release operation.

This operation always functions as a test, returning TRUE if the reservation is successful, FALSE otherwise.

redis reservation operations are performed on reevaluations.


  redis :store [:tableprefix prefix-string]
        [:timeout timeout-value] [:set | :sortedset]
        [:score score] key-string value-string

Creates a new entry or updates an existing entry with the specified key, value and timeout. Returns TRUE if the operation is successful, FALSE if it is not.

Store operations are identical to add operations for sets and sorted sets.

redis ":store" operations are simply repeated on reevaluations.


  redis :throttle :quota quota-numeric [:duplicate]
        :quotatimeout quotatimeout-numeric [:penalize] [:limit]
        [:tableprefix prefix-string]
        [:timeout timeout-value] [MATCH-TYPE] [COMPARATOR]
        [:adjustup adjustment-value] [:adjustdown adjustment-value]
        key-string [test-string] 

Implements the MeterMaid throttle capability. See the MeterMaid documentation for details of throttle semantics. Note that since there is no server-side awareness of entry semantics the quota and quotatimeout parameters must be specified in every throttle call in case the entry needs to be created. If the entry already exists the parameter values will be checked against the corresponding values stored in the entry.

The throttle value is incremented by default. Either ":adjustup" or ":adjustdown" can be used to specify an alternate adjustment value. (Note that in this case these nonpositional parameters function as modifiers, not operation specifiers.)

New in MS 8.0.2.3, ":limit" can be specified to prevent the throttle from being adjusted if it engages and to behave in a failsafe fashion.

This operation always functions as a test. If neither MATCH-TYPE, COMPARATOR, nor test-string are specified TRUE is returned if the throttle is engaged, FALSE if it is not engaged or an error occurs. If one of these three parameters is provided the throttle is adjusted and then Sieve tests are applied to the adjusted value. The default MATCH-TYPE is :value "gt" and the default COMPARATOR is "i;ascii-numeric".

redis ":throttle" operations are performed on reevaluations but the adjustment amount is changed to 0. ":duplicate" causes the adjustment amount to be retained on script reevaluations.


See also: