net/streamidstatistics/streamidstatistics_example.c

Reference Documentation

product_line_custom
Napatech SmartNIC
category
Reference Information

Description

This source file show the procedure needed to get stream-id statistics that can be correlated with what the application receive.

Note:

This example does not work with the NT4E-STD accelerator

Prerequisites

A Napatech capture accelerator is needed to run this example. The ntservice.ini must have at least one HostBuffersRx defined. Below is an example of a minimum ini-file. It will create a 32MB RX hostbuffer from NUMA node 0. [System] TimestampFormat = NATIVE [Adapter0] AdapterType = NT20E BusId = 00:0a:00.00 HostBuffersRx = [1,32,0]

Program flow

The following is required to perform real-time analysis on packets:
  • #include/nt.h - Applications/Tools only need to include nt.h to obtain prototypes, macros etc. from NTAPI.

  • NT_Init(NTAPI_VERSION) - Initialize the NTAPI library. NTAPI_VERSION is a define that describes the version of the API described in the header files included by nt.h. NT_Init() will ask the NTAPI library to convert return data to the NTAPI_VERSION if possible. This will ensure that applications can run on NTAPI libraries of newer versions.

  • NT_ConfigOpen() - Open a config stream in order to setup filter using the NT_NTPL() command and stream-id configuration.

  • NT_StatOpen() - Open a statistics stream which will be used to get stream-id statistics.

  • NT_NTPL() to inactivate the stream-id to ensure that no packets are forwarded to the stream-id until we are ready to handle them. A time stamp of when the stream-id is in-activated is returned. This time stamp is important because it tells us when to stop discarding packets.

  • NT_StatRead() is called asking for a statistics reset when the next statistics update is performed.

  • NT_NetRxOpen() - Open a stream. The stream ID must match the one used when creating the filter using the NT_NTPL() command. A stream doesn't return data until traffic is assigned to it by creating a filter. Stream IDs might be shared between other streams and it is possible to make several filters to one stream ID. Each filter can have a unique color in the ASSIGN. The "color" of the ASSIGN can be used to mark packets making it possible for the stream to determine if the packets it receives via NT_NetRxGet() as based on its assign or if the packet belongs to the other streams that also share the hostbuffer.

  • NT_NTPL() - Assign traffic to a stream by creating a filter using a manually chosen stream ID. The stream ID must match the one used NT_NetRxOpen().

  • NT_NTPL() to activate the stream-id because we are now ready to handle the traffic. It is important that wait with this step until all NT_NTPL() has been issued.

  • NT_NetRxGet() and NT_NetRxRelease() - Receive and release packets. Packets received with a NT_NET_GET_PKT_TIMESTAMP() older than the time stamp returned in the "in-activation" ste are discarded newer packets are counted.

  • When 10 packets has been received the stream-id is inactivated via NT_ConfigWrite() and the application will continue NT_NetRxGet() and NT_NetRxRelease() until NT_STATUS_TIMEOUT is returned.

  • Delete the stream-id assignments via NT_NTPL().

  • NT_NetRxClose() - Close the network stream.

  • Get stream-id statistics via NT_StatRead() and check that the amount of packets counted by the application and statistics stream match.

  • Close all other streams, NT_StatClose(), NT_ConfigClose().

  • NT_Done() - Close down the NTAPI library.

  • NT_ExplainError() - Explain an error code returned by NTAPI functions.

Code

/* * %NT_SOFTWARE_LICENSE% */ /** * @example net/streamidstatistics/streamidstatistics_example.c * @section streamidstatistics_example_description Description * * This source file show the procedure needed to get stream-id statistics that can be correlated with what the application receive. * * The following NTAPI functions are used: * - @ref NT_Init() * - @ref NT_StatOpen() * - @ref NT_ConfigOpen() * - @ref NT_ConfigWrite() * - @ref NT_StatRead() * - @ref NT_NetRxOpen() * - @ref NT_NTPL() * - @ref NT_NetRxGet() * - @ref NT_NET_GET_PKT_DESCRIPTOR_TYPE() * - @ref NT_NET_GET_PKT_TIMESTAMP() * - @ref NT_NET_GET_PKT_WIRE_LENGTH() * - @ref NT_NET_GET_PKT_L2_PTR() * - @ref NT_NetRxRelease() * - @ref NT_NetRxClose() * - @ref NT_ConfigClose() * - @ref NT_StatClose() * - @ref NT_Done() * - @ref NT_ExplainError() * * @note * This example does not work with the NT4E-STD accelerator * * <hr> * @section streamidstatistics_example_prerequisites Prerequisites * A Napatech capture accelerator is needed to run this example. The ntservice.ini must * have at least one HostBuffersRx defined. Below is an example of a * minimum ini-file. It will create a 32MB RX hostbuffer from NUMA * node 0. * @code * [System] * TimestampFormat = NATIVE * * [Adapter0] * AdapterType = NT20E * BusId = 00:0a:00.00 * HostBuffersRx = [1,32,0] * @endcode * * @section streamidstatistics_example_flow Program flow * @{ * The following is required to perform real-time analysis on packets: * - \#include/nt.h - Applications/Tools only need to include @ref * nt.h to obtain prototypes, macros etc. from NTAPI. * - @ref NT_Init(@ref NTAPI_VERSION) - Initialize the NTAPI * library. @ref NTAPI_VERSION is a define that describes the version * of the API described in the header files included by @ref * nt.h. @ref NT_Init() will ask the NTAPI library to convert return data * to the @ref NTAPI_VERSION if possible. This will ensure that * applications can run on NTAPI libraries of newer versions. * - @ref NT_ConfigOpen() - Open a config stream in order to setup * filter using the @ref NT_NTPL() command and stream-id configuration. * - @ref NT_StatOpen() - Open a statistics stream which will be used * to get stream-id statistics. * - @ref NT_NTPL() to inactivate the stream-id to ensure that no * packets are forwarded to the stream-id until we are ready to handle * them. A time stamp of when the stream-id is in-activated is returned. This * time stamp is important because it tells us when to stop discarding packets. * - @ref NT_StatRead() is called asking for a statistics reset when the next * statistics update is performed. * - @ref NT_NetRxOpen() - Open a stream. The stream ID must match the * one used when creating the filter using the @ref NT_NTPL() * command. A stream doesn't return data until traffic is assigned * to it by creating a filter. Stream IDs might be shared between * other streams and it is possible to make several filters to one * stream ID. Each filter can have a unique color in the ASSIGN. The * "color" of the ASSIGN can be used to mark packets making it * possible for the stream to determine if the packets it receives * via @ref NT_NetRxGet() as based on its assign or if the packet belongs * to the other streams that also share the hostbuffer. * - @ref NT_NTPL() - Assign traffic to a stream by creating a filter * using a manually chosen stream ID. The stream ID must match the * one used @ref NT_NetRxOpen(). * - @ref NT_NTPL() to activate the stream-id because we are now * ready to handle the traffic. It is important that wait with this * step until all @ref NT_NTPL() has been issued. * - @ref NT_NetRxGet() and @ref NT_NetRxRelease() - Receive and release packets. Packets received * with a @ref NT_NET_GET_PKT_TIMESTAMP() older than the time stamp returned in the "in-activation" * ste are discarded newer packets are counted. * - When 10 packets has been received the stream-id is inactivated via @ref NT_ConfigWrite() and * the application will continue @ref NT_NetRxGet() and @ref NT_NetRxRelease() until @ref NT_STATUS_TIMEOUT * is returned. * - Delete the stream-id assignments via @ref NT_NTPL(). * - @ref NT_NetRxClose() - Close the network stream. * - Get stream-id statistics via @ref NT_StatRead() and check that the amount of packets * counted by the application and statistics stream match. * - Close all other streams, @ref NT_StatClose(), @ref NT_ConfigClose(). * - @ref NT_Done() - Close down the NTAPI library. * - @ref NT_ExplainError() - Explain an error code returned by NTAPI functions. * *<hr> * @section streamidstatistics_example_code Code * @} */ #include <nt.h> #include <inttypes.h> #if defined(WIN32) || defined (WIN64) //#define snprintf _snprintf #define snprintf(a, b, c, d) _snprintf_s((a), _countof(a), (b), (c), (d)) #endif int main(void) { int numPackets=0; // The number of packets received int numBytes=0; // The number of bytes received (wire length) char tmpBuffer[20]; // Buffer to build filter string char errorBuffer[NT_ERRBUF_SIZE]; // Error buffer int status; // Status variable NtNetStreamRx_t hNetRx; // Handle to the RX stream NtConfigStream_t hCfgStream; // Handle to a config stream NtStatStream_t hStatStream; // Handle to a statistics stream NtStatistics_t stat; // Statistics data NtNtplInfo_t ntplInfo; // Return data structure from the NT_NTPL() call. NtNetBuf_t hNetBuf; // Net buffer container. Packet data is returned in this when calling NT_NetRxGet(). uint64_t inactiveTime=0; // Time when the stream-id was in-activated uint32_t ntplid=0; // The NT_NTPL ntplid returned from the Assign[] // Initialize the NTAPI library and thereby check if NTAPI_VERSION can be used together with this library if((status = NT_Init(NTAPI_VERSION)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_Init() failed: %s\n", errorBuffer); return -1; } // Open a config stream to assign a filter to a stream ID. if((status = NT_ConfigOpen(&hCfgStream, "TestStream")) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_ConfigOpen() failed: %s\n", errorBuffer); return -1; } if((status = NT_StatOpen(&hStatStream, "Stat")) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_StatOpen() failed: %s\n", errorBuffer); return -1; } // Inactivate the streamid 1 to ensure that no packets are forwarded to it if((status = NT_NTPL(hCfgStream, "Setup[State=Inactive]=StreamId==1", &ntplInfo, NT_NTPL_PARSER_VALIDATE_NORMAL)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NTPL() failed: %s\n", errorBuffer); fprintf(stderr, ">>> NTPL errorcode: %X\n", ntplInfo.u.errorData.errCode); fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]); fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]); fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]); return -1; } // Save the inactive time so we can discard packets older than this time because // they are of no interest. inactiveTime = ntplInfo.ts; // Reset statistics stat.cmd=NT_STATISTICS_READ_CMD_QUERY_V2; stat.u.query_v2.poll=0; // Wait for a new set stat.u.query_v2.clear=1; // Clear statistics if((status = NT_StatRead(hStatStream, &stat)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_StatRead() failed: %s\n", errorBuffer); return -1; } // Get a stream handle with the hostBuffer mapped to it. NT_NET_INTERFACE_PACKET specify that we will receive data packet-by-packet if((status = NT_NetRxOpen(&hNetRx, "TestStream", NT_NET_INTERFACE_PACKET, 1, -1)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NetRxOpen() failed: %s\n", errorBuffer); return -1; } // Assign traffic to stream ID 1 and mask all traffic matching the assign statement color=7. if((status = NT_NTPL(hCfgStream, "Assign[streamid=1;color=7] = All", &ntplInfo, NT_NTPL_PARSER_VALIDATE_NORMAL)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NTPL() failed: %s\n", errorBuffer); fprintf(stderr, ">>> NTPL errorcode: %X\n", ntplInfo.u.errorData.errCode); fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]); fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]); fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]); return -1; } // Store the ntplid so we can delete the Assign as part of the cleanup process ntplid=ntplInfo.ntplId; // Activate the stream-id printf("Activating the stream-id\n"); if((status = NT_NTPL(hCfgStream, "Setup[State=Active]=StreamId==1", &ntplInfo, NT_NTPL_PARSER_VALIDATE_NORMAL)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NTPL() failed: %s\n", errorBuffer); fprintf(stderr, ">>> NTPL errorcode: %X\n", ntplInfo.u.errorData.errCode); fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]); fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]); fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]); return -1; } printf("Waiting to get packets\n"); // Dump packet info. Stop when 10 packets has been received while(numPackets < 10) { if((status = NT_NetRxGet(hNetRx, &hNetBuf, 100000)) != NT_SUCCESS) { if((status == NT_STATUS_TIMEOUT) || (status == NT_STATUS_TRYAGAIN)) { // Timeouts are ok, we just need to wait a little longer for a packet continue; } // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NetRxGet() failed: %s\n", errorBuffer); return -1; } // We got a packet. Check if the timestamp is newer than when the NTPL assign command was applied if(NT_NET_GET_PKT_TIMESTAMP(hNetBuf) >= inactiveTime) { // Increment the number of packets processed. numPackets++; // Increment the bytes received numBytes+=NT_NET_GET_PKT_WIRE_LENGTH(hNetBuf); printf("Got %d packets so far\r", numPackets); } // Release the current packet if((status = NT_NetRxRelease(hNetRx, hNetBuf)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NetRxGet() failed: %s\n", errorBuffer); return -1; } } // Deactivate the stream-id printf("\nDeactivating the stream-id\n"); if((status = NT_NTPL(hCfgStream, "Setup[State=Inactive]=StreamId==1", &ntplInfo, NT_NTPL_PARSER_VALIDATE_NORMAL)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NTPL() failed: %s\n", errorBuffer); fprintf(stderr, ">>> NTPL errorcode: %X\n", ntplInfo.u.errorData.errCode); fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]); fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]); fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]); return -1; } printf("Continue to get packets until NT_STATUS_TIMEOUT... This will take a while.\n"); while(1) { if((status = NT_NetRxGet(hNetRx, &hNetBuf, 1000)) != NT_SUCCESS) { if(status == NT_STATUS_TIMEOUT) { break; } // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NetRxGet() failed: %s\n", errorBuffer); return -1; } // We got a packet. Check if the timestamp is newer than when the NTPL assign command was applied if(NT_NET_GET_PKT_TIMESTAMP(hNetBuf) >= inactiveTime) { // Increment the number of packets processed. numPackets++; // Increment the bytes received numBytes+=NT_NET_GET_PKT_WIRE_LENGTH(hNetBuf); if((numPackets%10) == 0) { printf("Got %d packets so far\r", numPackets); } } // Release the current packet if((status = NT_NetRxRelease(hNetRx, hNetBuf)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NetRxGet() failed: %s\n", errorBuffer); return -1; } } printf("Got %d packets so far\n", numPackets); // Delete the filter snprintf(tmpBuffer, 20, "delete=%d", ntplid); if((status = NT_NTPL(hCfgStream, tmpBuffer, &ntplInfo, NT_NTPL_PARSER_VALIDATE_NORMAL)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NTPL() failed: %s\n", errorBuffer); fprintf(stderr, ">>> NTPL errorcode: %X\n", ntplInfo.u.errorData.errCode); fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]); fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]); fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]); return -1; } // Close the stream and release the hostbuffer. NT_NetRxClose(hNetRx); printf("Getting stream-id statistics\n"); stat.cmd=NT_STATISTICS_READ_CMD_QUERY_V2; stat.u.query_v2.poll=0; // Wait for a new set stat.u.query_v2.clear=0; // Don't clear statistics if((status = NT_StatRead(hStatStream, &stat)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_StatRead() failed: %s\n", errorBuffer); return -1; } // Close the config stream if((status = NT_ConfigClose(hCfgStream)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_ConfigClose() failed: %s\n", errorBuffer); return -1; } // Close the statistics stream if((status = NT_StatClose(hStatStream)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_StatClose() failed: %s\n", errorBuffer); return -1; } printf("Done\n"); printf("APP received : %d packets, %d bytes. \n", numPackets, numBytes); printf("Stream-id received: %" PRIu64 " packets, %" PRIu64" bytes. \n", stat.u.query_v2.data.stream.streamid[1].forward.pkts, stat.u.query_v2.data.stream.streamid[1].forward.octets); printf("Stream-id dropped : %" PRIu64 " packets, %" PRIu64" bytes. \n", stat.u.query_v2.data.stream.streamid[1].drop.pkts, stat.u.query_v2.data.stream.streamid[1].drop.octets); // Close down the NTAPI library NT_Done(); return 0; }