net/streamidstatistics/streamidstatistics_example.c

Reference Documentation

Platform
Intel® PAC
Napatech SmartNIC
Content Type
Reference Information
Capture Software Version
Link™ Capture Software 12.10
Napatech Software Suite: net/streamidstatistics/streamidstatistics_example.c
net/streamidstatistics/streamidstatistics_example.c

Description

This source file show the procedure needed to get stream-id statistics that can be correlated with what the application receive.

The following NTAPI functions are used:

Note
This example does not work with the NT4E-STD accelerator

Prerequisites

A Napatech capture accelerator is needed to run this example. The ntservice.ini must have at least one HostBuffersRx defined. Below is an example of a minimum ini-file. It will create a 32MB RX hostbuffer from NUMA node 0.

[System]
TimestampFormat = NATIVE
[Adapter0]
AdapterType = NT20E
BusId = 00:0a:00.00
HostBuffersRx = [1,32,0]

Program flow

The following is required to perform real-time analysis on packets:

  • #include/nt.h - Applications/Tools only need to include nt.h to obtain prototypes, macros etc. from NTAPI.
  • NT_Init(NTAPI_VERSION) - Initialize the NTAPI library. NTAPI_VERSION is a define that describes the version of the API described in the header files included by nt.h. NT_Init() will ask the NTAPI library to convert return data to the NTAPI_VERSION if possible. This will ensure that applications can run on NTAPI libraries of newer versions.
  • NT_ConfigOpen() - Open a config stream in order to setup filter using the NT_NTPL() command and stream-id configuration.
  • NT_StatOpen() - Open a statistics stream which will be used to get stream-id statistics.
  • NT_NTPL() to inactivate the stream-id to ensure that no packets are forwarded to the stream-id until we are ready to handle them. A time stamp of when the stream-id is in-activated is returned. This time stamp is important because it tells us when to stop discarding packets.
  • NT_StatRead() is called asking for a statistics reset when the next statistics update is performed.
  • NT_NetRxOpen() - Open a stream. The stream ID must match the one used when creating the filter using the NT_NTPL() command. A stream doesn't return data until traffic is assigned to it by creating a filter. Stream IDs might be shared between other streams and it is possible to make several filters to one stream ID. Each filter can have a unique color in the ASSIGN. The "color" of the ASSIGN can be used to mark packets making it possible for the stream to determine if the packets it receives via NT_NetRxGet() as based on its assign or if the packet belongs to the other streams that also share the hostbuffer.
  • NT_NTPL() - Assign traffic to a stream by creating a filter using a manually chosen stream ID. The stream ID must match the one used NT_NetRxOpen().
  • NT_NTPL() to activate the stream-id because we are now ready to handle the traffic. It is important that wait with this step until all NT_NTPL() has been issued.
  • NT_NetRxGet() and NT_NetRxRelease() - Receive and release packets. Packets received with a NT_NET_GET_PKT_TIMESTAMP() older than the time stamp returned in the "in-activation" ste are discarded newer packets are counted.
  • When 10 packets has been received the stream-id is inactivated via NT_ConfigWrite() and the application will continue NT_NetRxGet() and NT_NetRxRelease() until NT_STATUS_TIMEOUT is returned.
  • Delete the stream-id assignments via NT_NTPL().
  • NT_NetRxClose() - Close the network stream.
  • Get stream-id statistics via NT_StatRead() and check that the amount of packets counted by the application and statistics stream match.
  • Close all other streams, NT_StatClose(), NT_ConfigClose().
  • NT_Done() - Close down the NTAPI library.
  • NT_ExplainError() - Explain an error code returned by NTAPI functions.

Code

/*
*
* Copyright 2023 Napatech A/S. All Rights Reserved.
*
* 1. Copying, modification, and distribution of this file, or executable
* versions of this file, is governed by the terms of the Napatech Software
* license agreement under which this file was made available. If you do not
* agree to the terms of the license do not install, copy, access or
* otherwise use this file.
*
* 2. Under the Napatech Software license agreement you are granted a
* limited, non-exclusive, non-assignable, copyright license to copy, modify
* and distribute this file in conjunction with Napatech SmartNIC's and
* similar hardware manufactured or supplied by Napatech A/S.
*
* 3. The full Napatech Software license agreement is included in this
* distribution, please see "NP-0405 Napatech Software license
* agreement.pdf"
*
* 4. Redistributions of source code must retain this copyright notice,
* list of conditions and the following disclaimer.
*
* THIS SOFTWARE IS PROVIDED "AS IS" WITHOUT ANY WARRANTIES, EXPRESS OR
* IMPLIED, AND NAPATECH DISCLAIMS ALL IMPLIED WARRANTIES INCLUDING ANY
* IMPLIED WARRANTY OF TITLE, MERCHANTABILITY, NONINFRINGEMENT, OR OF
* FITNESS FOR A PARTICULAR PURPOSE. TO THE EXTENT NOT PROHIBITED BY
* APPLICABLE LAW, IN NO EVENT SHALL NAPATECH BE LIABLE FOR PERSONAL INJURY,
* OR ANY INCIDENTAL, SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES WHATSOEVER,
* INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, CORRUPTION OR
* LOSS OF DATA, FAILURE TO TRANSMIT OR RECEIVE ANY DATA OR INFORMATION,
* BUSINESS INTERRUPTION OR ANY OTHER COMMERCIAL DAMAGES OR LOSSES, ARISING
* OUT OF OR RELATED TO YOUR USE OR INABILITY TO USE NAPATECH SOFTWARE OR
* SERVICES OR ANY THIRD PARTY SOFTWARE OR APPLICATIONS IN CONJUNCTION WITH
* THE NAPATECH SOFTWARE OR SERVICES, HOWEVER CAUSED, REGARDLESS OF THE THEORY
* OF LIABILITY (CONTRACT, TORT OR OTHERWISE) AND EVEN IF NAPATECH HAS BEEN
* ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. SOME JURISDICTIONS DO NOT ALLOW
* THE EXCLUSION OR LIMITATION OF LIABILITY FOR PERSONAL INJURY, OR OF
* INCIDENTAL OR CONSEQUENTIAL DAMAGES, SO THIS LIMITATION MAY NOT APPLY TO YOU.
*
*
*/
/**
* @example net/streamidstatistics/streamidstatistics_example.c
* @section streamidstatistics_example_description Description
*
* This source file show the procedure needed to get stream-id statistics that can be correlated with what the application receive.
*
* The following NTAPI functions are used:
* - @ref NT_Init()
* - @ref NT_StatOpen()
* - @ref NT_ConfigOpen()
* - @ref NT_ConfigWrite()
* - @ref NT_StatRead()
* - @ref NT_NetRxOpen()
* - @ref NT_NTPL()
* - @ref NT_NetRxGet()
* - @ref NT_NET_GET_PKT_DESCRIPTOR_TYPE()
* - @ref NT_NET_GET_PKT_TIMESTAMP()
* - @ref NT_NET_GET_PKT_WIRE_LENGTH()
* - @ref NT_NET_GET_PKT_L2_PTR()
* - @ref NT_NetRxRelease()
* - @ref NT_NetRxClose()
* - @ref NT_ConfigClose()
* - @ref NT_StatClose()
* - @ref NT_Done()
* - @ref NT_ExplainError()
*
* @note
* This example does not work with the NT4E-STD accelerator
*
* <hr>
* @section streamidstatistics_example_prerequisites Prerequisites
* A Napatech capture accelerator is needed to run this example. The ntservice.ini must
* have at least one HostBuffersRx defined. Below is an example of a
* minimum ini-file. It will create a 32MB RX hostbuffer from NUMA
* node 0.
* @code
* [System]
* TimestampFormat = NATIVE
*
* [Adapter0]
* AdapterType = NT20E
* BusId = 00:0a:00.00
* HostBuffersRx = [1,32,0]
* @endcode
*
* @section streamidstatistics_example_flow Program flow
* @{
* The following is required to perform real-time analysis on packets:
* - \#include/nt.h - Applications/Tools only need to include @ref
* nt.h to obtain prototypes, macros etc. from NTAPI.
* - @ref NT_Init(@ref NTAPI_VERSION) - Initialize the NTAPI
* library. @ref NTAPI_VERSION is a define that describes the version
* of the API described in the header files included by @ref
* nt.h. @ref NT_Init() will ask the NTAPI library to convert return data
* to the @ref NTAPI_VERSION if possible. This will ensure that
* applications can run on NTAPI libraries of newer versions.
* - @ref NT_ConfigOpen() - Open a config stream in order to setup
* filter using the @ref NT_NTPL() command and stream-id configuration.
* - @ref NT_StatOpen() - Open a statistics stream which will be used
* to get stream-id statistics.
* - @ref NT_NTPL() to inactivate the stream-id to ensure that no
* packets are forwarded to the stream-id until we are ready to handle
* them. A time stamp of when the stream-id is in-activated is returned. This
* time stamp is important because it tells us when to stop discarding packets.
* - @ref NT_StatRead() is called asking for a statistics reset when the next
* statistics update is performed.
* - @ref NT_NetRxOpen() - Open a stream. The stream ID must match the
* one used when creating the filter using the @ref NT_NTPL()
* command. A stream doesn't return data until traffic is assigned
* to it by creating a filter. Stream IDs might be shared between
* other streams and it is possible to make several filters to one
* stream ID. Each filter can have a unique color in the ASSIGN. The
* "color" of the ASSIGN can be used to mark packets making it
* possible for the stream to determine if the packets it receives
* via @ref NT_NetRxGet() as based on its assign or if the packet belongs
* to the other streams that also share the hostbuffer.
* - @ref NT_NTPL() - Assign traffic to a stream by creating a filter
* using a manually chosen stream ID. The stream ID must match the
* one used @ref NT_NetRxOpen().
* - @ref NT_NTPL() to activate the stream-id because we are now
* ready to handle the traffic. It is important that wait with this
* step until all @ref NT_NTPL() has been issued.
* - @ref NT_NetRxGet() and @ref NT_NetRxRelease() - Receive and release packets. Packets received
* with a @ref NT_NET_GET_PKT_TIMESTAMP() older than the time stamp returned in the "in-activation"
* ste are discarded newer packets are counted.
* - When 10 packets has been received the stream-id is inactivated via @ref NT_ConfigWrite() and
* the application will continue @ref NT_NetRxGet() and @ref NT_NetRxRelease() until @ref NT_STATUS_TIMEOUT
* is returned.
* - Delete the stream-id assignments via @ref NT_NTPL().
* - @ref NT_NetRxClose() - Close the network stream.
* - Get stream-id statistics via @ref NT_StatRead() and check that the amount of packets
* counted by the application and statistics stream match.
* - Close all other streams, @ref NT_StatClose(), @ref NT_ConfigClose().
* - @ref NT_Done() - Close down the NTAPI library.
* - @ref NT_ExplainError() - Explain an error code returned by NTAPI functions.
*
*<hr>
* @section streamidstatistics_example_code Code
* @}
*/
// Include this in order to access the Napatech API
#include <nt.h>
#include <inttypes.h>
#if defined(WIN32) || defined(WIN64)
#define snprintf(dst, ...) _snprintf_s((dst), _countof(dst), __VA_ARGS__)
#endif
int main(void)
{
int numPackets=0; // The number of packets received
int numBytes=0; // The number of bytes received (wire length)
char tmpBuffer[20]; // Buffer to build filter string
char errorBuffer[NT_ERRBUF_SIZE]; // Error buffer
int status; // Status variable
NtNetStreamRx_t hNetRx; // Handle to the RX stream
NtConfigStream_t hCfgStream; // Handle to a config stream
NtStatStream_t hStatStream; // Handle to a statistics stream
NtStatistics_t stat; // Statistics data
NtNtplInfo_t ntplInfo; // Return data structure from the NT_NTPL() call.
NtNetBuf_t hNetBuf; // Net buffer container. Packet data is returned in this when calling NT_NetRxGet().
uint64_t inactiveTime=0; // Time when the stream-id was in-activated
uint32_t ntplid=0; // The NT_NTPL ntplid returned from the Assign[]
// Initialize the NTAPI library and thereby check if NTAPI_VERSION can be used together with this library
if ((status = NT_Init(NTAPI_VERSION)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_Init() failed: %s\n", errorBuffer);
return -1;
}
// Open a config stream to assign a filter to a stream ID.
if ((status = NT_ConfigOpen(&hCfgStream, "TestStream")) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_ConfigOpen() failed: %s\n", errorBuffer);
return -1;
}
if ((status = NT_StatOpen(&hStatStream, "Stat")) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_StatOpen() failed: %s\n", errorBuffer);
return -1;
}
// Inactivate the streamid 1 to ensure that no packets are forwarded to it
if ((status = NT_NTPL(hCfgStream, "Setup[State=Inactive]=StreamId==1", &ntplInfo, NT_NTPL_PARSER_VALIDATE_NORMAL)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NTPL() failed: %s\n", errorBuffer);
fprintf(stderr, ">>> NTPL errorcode: %X\n", ntplInfo.u.errorData.errCode);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]);
return -1;
}
// Save the inactive time so we can discard packets older than this time because
// they are of no interest.
inactiveTime = ntplInfo.ts;
// Reset statistics
stat.u.query_v3.poll=0; // Wait for a new set
stat.u.query_v3.clear=1; // Clear statistics
if ((status = NT_StatRead(hStatStream, &stat)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_StatRead() failed: %s\n", errorBuffer);
return -1;
}
// Get a stream handle with the hostBuffer mapped to it. NT_NET_INTERFACE_PACKET specify that we will receive data packet-by-packet
if ((status = NT_NetRxOpen(&hNetRx, "TestStream", NT_NET_INTERFACE_PACKET, 1, -1)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetRxOpen() failed: %s\n", errorBuffer);
return -1;
}
// Assign traffic to stream ID 1 and mask all traffic matching the assign statement color=7.
if ((status = NT_NTPL(hCfgStream, "Assign[streamid=1;color=7] = All", &ntplInfo, NT_NTPL_PARSER_VALIDATE_NORMAL)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NTPL() failed: %s\n", errorBuffer);
fprintf(stderr, ">>> NTPL errorcode: %X\n", ntplInfo.u.errorData.errCode);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]);
return -1;
}
// Store the ntplid so we can delete the Assign as part of the cleanup process
ntplid=ntplInfo.ntplId;
// Activate the stream-id
printf("Activating the stream-id\n");
if ((status = NT_NTPL(hCfgStream, "Setup[State=Active]=StreamId==1", &ntplInfo, NT_NTPL_PARSER_VALIDATE_NORMAL)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NTPL() failed: %s\n", errorBuffer);
fprintf(stderr, ">>> NTPL errorcode: %X\n", ntplInfo.u.errorData.errCode);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]);
return -1;
}
printf("Waiting to get packets\n");
// Dump packet info. Stop when 10 packets has been received
while (numPackets < 10) {
if ((status = NT_NetRxGet(hNetRx, &hNetBuf, 100000)) != NT_SUCCESS) {
if ((status == NT_STATUS_TIMEOUT) || (status == NT_STATUS_TRYAGAIN)) {
// Timeouts are ok, we just need to wait a little longer for a packet
continue;
}
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetRxGet() failed: %s\n", errorBuffer);
return -1;
}
// We got a packet. Check if the timestamp is newer than when the NTPL assign command was applied
if (NT_NET_GET_PKT_TIMESTAMP(hNetBuf) >= inactiveTime) {
// Increment the number of packets processed.
numPackets++;
// Increment the bytes received
numBytes+=NT_NET_GET_PKT_WIRE_LENGTH(hNetBuf);
printf("Got %d packets so far\r", numPackets);
}
// Release the current packet
if ((status = NT_NetRxRelease(hNetRx, hNetBuf)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetRxGet() failed: %s\n", errorBuffer);
return -1;
}
}
// Deactivate the stream-id
printf("\nDeactivating the stream-id\n");
if ((status = NT_NTPL(hCfgStream, "Setup[State=Inactive]=StreamId==1", &ntplInfo, NT_NTPL_PARSER_VALIDATE_NORMAL)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NTPL() failed: %s\n", errorBuffer);
fprintf(stderr, ">>> NTPL errorcode: %X\n", ntplInfo.u.errorData.errCode);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]);
return -1;
}
printf("Continue to get packets until NT_STATUS_TIMEOUT... This will take a while.\n");
while (1) {
if ((status = NT_NetRxGet(hNetRx, &hNetBuf, 1000)) != NT_SUCCESS) {
if (status == NT_STATUS_TIMEOUT) {
break;
}
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetRxGet() failed: %s\n", errorBuffer);
return -1;
}
// We got a packet. Check if the timestamp is newer than when the NTPL assign command was applied
if (NT_NET_GET_PKT_TIMESTAMP(hNetBuf) >= inactiveTime) {
// Increment the number of packets processed.
numPackets++;
// Increment the bytes received
numBytes+=NT_NET_GET_PKT_WIRE_LENGTH(hNetBuf);
if ((numPackets%10) == 0) {
printf("Got %d packets so far\r", numPackets);
}
}
// Release the current packet
if ((status = NT_NetRxRelease(hNetRx, hNetBuf)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetRxGet() failed: %s\n", errorBuffer);
return -1;
}
}
printf("Got %d packets so far\n", numPackets);
// Delete the filter
snprintf(tmpBuffer, 20, "delete=%d", ntplid);
if ((status = NT_NTPL(hCfgStream, tmpBuffer, &ntplInfo, NT_NTPL_PARSER_VALIDATE_NORMAL)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NTPL() failed: %s\n", errorBuffer);
fprintf(stderr, ">>> NTPL errorcode: %X\n", ntplInfo.u.errorData.errCode);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]);
return -1;
}
// Close the stream and release the hostbuffer.
NT_NetRxClose(hNetRx);
printf("Getting stream-id statistics\n");
stat.u.query_v3.poll=0; // Wait for a new set
stat.u.query_v3.clear=0; // Don't clear statistics
if ((status = NT_StatRead(hStatStream, &stat)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_StatRead() failed: %s\n", errorBuffer);
return -1;
}
// Close the config stream
if ((status = NT_ConfigClose(hCfgStream)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_ConfigClose() failed: %s\n", errorBuffer);
return -1;
}
// Close the statistics stream
if ((status = NT_StatClose(hStatStream)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_StatClose() failed: %s\n", errorBuffer);
return -1;
}
printf("Done\n");
printf("APP received : %d packets, %d bytes. \n", numPackets, numBytes);
printf("Stream-id received: %" PRIu64 " packets, %" PRIu64" bytes. \n", stat.u.query_v3.data.stream.streamid[1].forward.pkts, stat.u.query_v3.data.stream.streamid[1].forward.octets);
printf("Stream-id dropped : %" PRIu64 " packets, %" PRIu64" bytes. \n", stat.u.query_v3.data.stream.streamid[1].drop.pkts, stat.u.query_v3.data.stream.streamid[1].drop.octets);
// Close down the NTAPI library
return 0;
}