Key Tests

Reference Documentation

Platform
Intel® PAC
Napatech SmartNIC
Content Type
Reference Information
Capture Software Version
Link™ Capture Software 12.10
Napatech Software Suite: Key Tests

This section specifies the syntax and semantics for the Key tests.

Supported Options Per Feature Set

Supported Feature Sets
N-ANL4 N-ANL5
N-ANL6
N-ANL7
N-ANL8
N-ANL9
N-ANL10
  X

The syntax for the Key tests is shown below:

<KeyTest>           :== 'Key' '(' <Identifier> [ ',' <KeyTestOptionList> ] ')' <EqualNotEqualTest> <KeySetIdOrAny>
<KeyTestOptionList> :== <KeyTestOption> [ ',' <KeyTestOption> ]
<KeyTestOption>     :== ( 'ColorInfo' '=' <TrueFalseValue> ) |
                        ( 'KeyID' '=' <8-bit decimal value> ) |
                        ( 'CounterSet' '=' <CounterSet> ) |
                        ( 'FieldAction' '=' <FieldAction> )
<CounterSet>        :== 'CSA' | 'CSB'
<FieldAction>       :== 'Swap' | 'None'
<KeySetIdOrAny>     :== <KeySetId> | 'ANY' | 'MISS' | 'UNHANDLED'

The Key test is a generalization of the KeyMatch Test, where instead of testing on single fields in a packet, tests are conducted on a key definition that can contain multiple fields from a packet. Key test is used for both the Key matcher functionality and Flow matcher functionality; the sections below will specify the similarities and differences.

Key matcher

  • ColorInfo: Enables color information when combining key tests. Default: True.
  • KeyID: N/A
  • CounterSet: N/A
  • FieldAction: N/A

A key test for the Key matcher functionality matches incoming packets against a key definition, which is tied to keys entered through the KeyList command. The KeyList command ties key data is to a KeySetId. If the key is found the returned value of the key test is a KeySetId. If the returned KeySetId equals the right-hand-side (RHS) of the test, then the test evaluates to true, otherwise false. If the key was not found, the key test returns MISS, which evaluates to true when RHS is MISS or ANY. Every possible returned value of a key test evaluates to true with ANY. UNHANDLED is not applicable for the Key matcher functionality.

When using the Key matcher, the KeySetId used on the RHS expression must either be tied to the key type of the key definition used on the left-hand-side (LHS), or not be tied to any key type. In the latter case (i.e. no keys have yet been added to the database with that KeySetId) the key set id will be tied to the key type at this time.

Key tests can be combined using the 'AND' and 'OR' operators. If both keys in a combined Key test expression matches a packet and both keys have color information attached to them, the resulting color will be the bitwise OR of the two colors. The stream info of a key is internally stored in the upper 7 bits of the color. Only one key can contribute to the stream info and therefore the color contribution of one of the keys must be turned off if both of them carry stream information. That is what the "ColorInfo" option in the Key test is for. If one key type carries stream information and the other key type carries color information, care should be taken to not set the upper 7 color bits on any keys of the latter key type as this could yield unexpected results.

Note: As also noted in the KeyMatch Test section, it is important, that if you have multiple assign commands that only differs with Key tests, then the key definitions used must be the same. You cannot test one key definition in one filter and another key definition in another filter, unless the two filters are mutually exclusive. If the filters are not mutually exclusive, you will then have to include both key definitions in both filter expressions. The 'ANY' keyword can be used in this case to specify a "dummy" Key statement that will always evaluate to true. See example below.

Note: Another limitation is that it is only possible to do at most two lookups per packet. It is thus not possible to have more than two key tests in one filter (assign statement).

Flow matcher

  • ColorInfo: N/A
  • KeyID: Required option for Flow matcher. Default: Key matcher functionality is used instead.
  • CounterSet: Chooses which counter set shall be incremented when a packet is matched. Default: CSA.
  • FieldAction: Enables swapping of field. Default: None.

A key test for the Flow matcher functionality matches incoming packets against a key definition, which is tied to keys entered through flow learn records ( NtFlow_t ). The tie between key tests and and flow learn records is the KeyID option and the flow learn record field keyId. When set, the KeyID option is also the signal that the Flow matcher functionality should be used for the key test, rather than the Key matcher functionality.

Flow learn records also have the field keySetId, which ties to the right-hand-side (RHS) of the key test expression. If the key is found with the correct KeyID the key test returns the KeySetId set by the flow learn record. If the returned KeySetId equals the RHS of the test, then the test evaluates to true, otherwise false. If the key was not found the key test returns MISS. ANY is not applicable for the Flow matcher functionality.

During periods with high incoming data rate, it may in rare situation happen that the Flow matcher is unable to perform lookup for a packet, in which case the key test will return UNHANDLED. By having a filter where the key test return value is compared to UNHANDLED, you can catch all packets for which the Flow Matcher wasn't able to perform a lookup.

The CounterSet option specifies which set (A or B) of counters in the flow info record should be modified. Note that when multiple filters include the key tests, for example hit, miss, and unhandled filters, the CounterSet value will overwrite previous values, so the last assign command dictates the option value.

The FieldAction option will cause the key test to perform its test on swapped fields. See KeyType command - FieldSizes for more information.

Note: Where the Key matcher functionality is able to perform two lookups per filter, the Flow matcher functionality is only able to perform a single lookup for a single packet. Only key tests from the highest priority filter matching a packet will be executed. This is also why there are no use cases for ANY.

Examples

This section presents several examples of Key test usage.

Key matcher - Port matching

This example demonstrates how to use the Key matcher to match packets based on layer 4 destination port values. The keys are tagged with 2 different key sets (3 or 4) in order to differentiate between TCP and UDP ports.

// Define Key Type
KeyType[Name=DstPortKT] = {16}
// Create key definition
KeyDef[Name=DstPort; KeyType=DstPortKT] = (Layer4Header[0]/16)
// Add a number of tcp port values to key set 3
KeyList[KeyType=DstPortKT; KeySet=3] = (80),(21),(22)
// Add a number of udp port values to key set 4
KeyList[KeyType=DstPortKT; KeySet=4] = (53), (5353)
// Add a range of udp port values to key set 4
KeyList[KeyType=DstPortKT; KeySet=4] = ((27000..27015))
// Set up filter matching tcp ports in key set 3
Assign[StreamId=0] = Layer4Protocol == TCP and Key(DstPort) == 3
// Set up filter matching udp ports in key set 4
Assign[StreamId=1] = Layer4Protocol == UDP and Key(DstPort) == 4

Key matcher - Single IPv4 address

In this example a single key definition is used to match on the source address of IPv4 frames. Color information is added to the keys a key in the database will set the descriptor color when it matches in the Key test:

// Define Key Type
KeyType[name=KT_IPv4; ColorInfo=True] = {32}
// Create key definition
KeyDef[Name=SrcIPv4; KeyType=KT_IPv4] = (Layer3Header[12]/32)
// Add some keys with color information
KeyList[KeySet = 7; Color = 23; KeyType = KT_IPv4] = ( [192.168.0.2] ), ( [192.168.0.3] ), ( [192.168.0.4]; Color = 24 )
KeyList[KeySet = 8; Color = 25; KeyType = KT_IPv4] = ( [192.168.1.20] ), ( [192.168.1.21] ), ( [192.168.1.24]; Color = 26 )
// Set up filters
Assign[StreamId=0] = Layer3Protocol == IPv4 AND Key(SrcIPv4) == 7
Assign[StreamId=1] = Layer3Protocol == IPv4 AND Key(SrcIPv4) == 8

Key matcher - Source and destination IPv6 addresses

In this example we are interested in matching both the source and destination addresses of IPv6 frames. Furthermore we want both the SA and DA to be matched against a collection of keys belonging to the same key type. This would be useful in scenarios where one wants to match bidirectional traffic where an address appears in both the SA or DA fields in the incoming frames. In this example upstream frames from the same client IP <-> remote IP address pair should end up in the same stream as when they are reversed in the downstream direction. This is done by applying a different hash recipe based on whether the traffic is upstream or downstream. It is assumed that both up- and downstream traffic arrives on the same port. When we cannot use the port number to logically separate the packets they must be separated based on whether SA or DA matches an ip address in the database. Both the SA and DA must be matched at that same time when a frame arrives. It is necessary to signal this condition/scenario when defining the key type. This is where the 'Access = Full' option comes into play.

// Define fields
Define IPv6Src = Field(Layer3Header[8]/128)
Define IPv6Dst = Field(Layer3Header[24]/128)

// Define Up- and Downstream hash recipes
Define HashUp = Hash(HashWord0_3=IPv6Src, HashWord4_7=IPv6Dst)
Define HashDown = Hash(HashWord0_3=IPv6Dst, HashWord4_7=IPv6Src)

// Define Key Type
KeyType[name=KT_IPv6; Access = Full; ColorInfo=True] = {128}

// Define keys
KeyDef[name=SrcIPv6; KeyType=KT_IPv6] = (IPv6Src)
KeyDef[name=DstIPv6; KeyType=KT_IPv6] = (IPv6Dst)

// Add a masked key
KeyList[KeySet = 8; Color = 25; KeyType = KT_IPv6] = ( {[ffff:ffff:ffff:ffff:ffff:ffff:ffff:0000]:[d7a0:4e95:0:0:0:0:af:0000]} )

// Set up filters
Assign[StreamId=(0..31); Hash=HashUp] = Port == 0 AND Layer3Protocol == IPv6 AND (Key(SrcIPv6) == 8 AND Key(DstIPv6) == ANY)
Assign[StreamId=(0..31); Hash=HashDown] = Port == 0 AND Layer3Protocol == IPv6 AND (Key(SrcIPv6) == ANY AND Key(DstIPv6) == 8)

As mentioned in the note above, since the filters are not mutually exclusive (same port), both keys must use the same keys in both filters, hence the use of the ANY keyword.

Key matcher - Inner IPv6 address

This example shows how to filter GTP-U packets where the tunneled packet is IPv6 and the source or destination address matches a key. Coloring is used to identify each "subscriber" on the tunneled network:

// Define key type
KeyType[Name=KT; Access=Full; ColorInfo=True] = {128}
// Define keys for source- and destination address
KeyDef[Name=SrcKD; KeyType=KT] = (InnerLayer3Header[8]/128)
KeyDef[Name=DstKD; KeyType=KT] = (InnerLayer3Header[24]/128)
// Set up filter
Assign[StreamId=0] = TunnelType == GTPv1-U and InnerLayer3Protocol == IPv6 and (Key(SrcKD) == 3 or Key(DstKD) == 3)
// Keys can now be added
KeyList[KeyType=KT; Color=23] = ([1234:abcd:0:0:0:0:0:c0ed])

Key matcher - Flow matching

This example shows how to implement a flow matching scheme where a composite key is used. The key is composed of VLAN Id, SA, DA as well as layer 4 source and destination port. Since field sizes must be a multiple of 8 bits and the VLAN Id is 12 bits, it is extracted as a 16 bit field and a 12 bit mask applied to it in the 'KeyDef' definition. It is assumed that up- and downstream traffic enters the adapter on the same port. Here two keys are defined: one for upstream and one for downstream traffic. Because up- and downstream traffic enters the same port, both keys will be used in the filters, so the "Access=Full" option must be set on the key type. The downstream key has its source and destination fields swapped. The keys have color information attached to them. Two hash recipes are used (up- and downstream) to ensure that packets from the same flow end up in the same stream:

// Define fields
Define VlanId = Field(FirstVLAN[2]/16)
Define IPv4Src = Field(Layer3Header[12]/32)
Define Ipv4Dst = Field(Layer3Header[16]/32)
Define DstPort = Field(Layer4Header[0]/16)
Define SrcPort = Field(Layer4Header[2]/16)

// Define Up- and Downstream hash recipes
Define HashUp = Hash(HashWord0_3=IPv4Src, HashWord4_7=IPv4Dst, HashWord8=SrcPort, HashWord9=DstPort)
Define HashDown = Hash(HashWord0_3=IPv4Dst, HashWord4_7=IPv4Src, HashWord8=DstPort, HashWord9=SrcPort)

// Define Key Type
KeyType[name=KT_4TupleV4; Access=Full; ColorInfo=True] = {16, 32, 32, 16, 16}

// Define keys
KeyDef[name=KD_TupleV4Up; KeyType=KT_4TupleV4] = ({0xFFF:VlanId}, IPv4Src, IPv4Dst, SrcPort, DstPort)
KeyDef[name=KD_TupleV4Down; KeyType=KT_4TupleV4] = ({0xFFF:VlanId}, IPv4Dst, IPv4Src, DstPort, SrcPort)

// Add single key
KeyList[KeySet = 7; Color = 23; KeyType = KT_4TupleV4] = (0x234, [192.168.0.2], [192.168.0.1], 0x0dc0, 'b10), (436, 0xaf3322de, 0xadbeef60, 0x0dc0, 0xde00; Color = 24) // Color overridden
// Add masked key
KeyList[KeySet = 7; Color = 25; KeyType = KT_4TupleV4] = ( {0xf00:400}, {[ff.ff.00.00]:[192.169.0.0]}, {[ff.ff.00.00]:[192.170.0.0]}, 0xde00, 0x0dff )

// Set up filters
Assign[StreamId=(0..31); Hash=HashUp] = Layer3Protocol == IPv4 AND Key(KD_4TupleV4Up) == 7 AND Key(KD_4TupleV4Down) == ANY
Assign[StreamId=(0..31); Hash=HashDown] = Layer3Protocol == IPv4 AND Key(KD_4TupleV4Up) == ANY AND Key(KD_4TupleV4Down) == 7

Note that because the two filters are not mutually exclusive, two lookups are necessary per filter and so we must use the ANY keyword since both keys must be present in both filters.

Key matcher - NAT Mapping

This example assumes a setup involving a NAT router, where both sides are monitored by an aggregation switch sending traffic into one port on the adapter. On the LAN side a 5-tuple hash distributes flows to a number of streams. The goal is to have the corresponding packets on the WAN side sent to the same streams. On the WAN side hashing cannot be used, because the NAT router mangles the source IP address and port on the outgoing packets. The packets would consequently not end up in the same streams. It is assumed that the port mapping in the NAT router can be learned by the application responsible for configuring the adapter. The following key sets are used:

  • IPv4/TCP : 3
  • IPv4/UDP : 4
  • IPv6/TCP : 5
  • IPv6/UDP : 6

The NTPL for this example looks as follows:

// Set up hashing on the LAN side
HashMode(Priority=1; Layer3Type=IP) = Hash5TupleSorted

// Define key types and keys for IPv4 and IPv6 packets
KeyType[Name=KT_IPv4; Access=Full; StreamInfo=True] = {32, 16, 16}
KeyDef[Name=KD_IPv4Up; KeyType=KT_IPv4] = (Layer3Header[16]/32, Layer4Header[0]/16, Layer4Header[2]/16)
KeyDef[Name=KD_IPv4Down; KeyType=KT_IPv4] = (Layer3Header[12]/32, Layer4Header[2]/16, Layer4Header[0]/16)
KeyType[Name=KT_IPv6; Access=Full; StreamInfo=True] = {128, 16, 16}
KeyDef[Name=KD_IPv6Up; KeyType=KT_IPv6] = (Layer3Header[24]/128, Layer4Header[0]/16, Layer4Header[2]/16)
KeyDef[Name=KD_IPv6Down; KeyType=KT_IPv6] = (Layer3Header[8]/128, Layer4Header[2]/16, Layer4Header[0]/16)

// Assign key match filters for the WAN side (IPv4 and IPv6)
Assign[StreamId=Key] = Layer3Protocol == IPv4 AND Layer4Protocol == TCP AND (Key(KD_IPv4Up) == 3 or Key(KD_IPv4Down) == 3)
Assign[StreamId=Key] = Layer3Protocol == IPv4 AND Layer4Protocol == UDP AND (Key(KD_IPv4Up) == 4 or Key(KD_IPv4Down) == 4)
Assign[StreamId=Key] = Layer3Protocol == IPv6 AND Layer4Protocol == TCP AND (Key(KD_IPv6Up) == 5 or Key(KD_IPv6Down) == 5)
Assign[StreamId=Key] = Layer3Protocol == IPv6 AND Layer4Protocol == UDP AND (Key(KD_IPv6Up) == 6 or Key(KD_IPv6Down) == 6)

// Assign an "All" filter for the LAN packets
Assign[StreamId=(0..63);Priority=1] = All

// Add keys as the NAT mapping is learned e.g. TCP: (192.168.0.3-90.88.12.34):(2383-80) –> (<NAT-IP>-90.88.12.34):(4560-80)
KeyList[KeyType=KT_IPv4;KeySet=3;StreamId=45] = ([90.88.12.34], 4560, 80)

Flow matcher - Source & destination swapping

This example shows how to set up a bi-directional IPv4 TCP Flow matcher in NTPL. The KeyDef defines a 5-tuple consisting of IPv4 source and destination addresses, IP protocol field, and TCP source and destination ports. The filters are set up such that UNHANDLED packets are send to stream ID 0. Packets that MISS are on port 0 send to stream ID 1 to be learned, or are dropped if received on port 1. Packets that hit a learned flow are dropped and the relevant flow statistic counters are incremented. Stream ID 0 and 1 should be handled by an user application.

Because KeyType kt and KeyDef kd in the NTPL below is defined as swappable fields, the key test option FieldAction=Swap becomes available. When FieldAction=Swap is set, the swappable fields will be swapped before key lookup. In this example IPv4 addresses and TCP ports are swapped:

  • FieldAction=None: Key data = (IP Protocol, IPv4 src addr, IPv4 dst addr, TCP src port, TCP dst port)
  • FieldAction=Swap: Key data = (IP Protocol, IPv4 dst addr, IPv4 src addr, TCP dst port, TCP src port)

The result is that packets from port 0 and port 1 belonging to the same flow will end up with identical keys. This single key principle enables the flow matcher to perform stateful flow supervision and collect flow statistics from the two directions individually.

KeyType[Name=kt] = {sw_32_32, sw_16_16}
KeyDef[Name=kd; KeyType=kt; IpProtocolField=Outer] = (Layer3Header[12]/32/32, Layer4Header[0]/16/16)
DefineMacro("FilterCheck", "Port==$1 and Layer3Protocol==IPv4 and Layer4Protocol==TCP")
Assign[StreamId=0] = FilterCheck(0) and Key(kd, KeyID=1) == UNHANDLED
Assign[StreamId=0] = FilterCheck(1) and Key(kd, KeyID=1, FieldAction=Swap) == UNHANDLED
Assign[StreamId=1] = FilterCheck(0) and Key(kd, KeyID=1) == MISS
Assign[StreamId=Drop] = FilterCheck(1) and Key(kd, KeyID=1, FieldAction=Swap) == MISS
Assign[StreamId=Drop] = FilterCheck(0) and Key(kd, KeyID=1, CounterSet=CSA) == 4
Assign[StreamId=Drop] = FilterCheck(1) and Key(kd, KeyID=1, CounterSet=CSB, FieldAction=Swap) == 4

See Flow matcher feature for more Flow matcher examples and flowmatch/flowmatch_example_main.cpp for a C++ code example.