net/capture/capture_example.c

Reference Documentation

product_line_custom
IntelĀ® PAC
Napatech SmartNIC
category
Reference Information
Napatech Software Suite: net/capture/capture_example.c
net/capture/capture_example.c

Description

This source file is an example of how to do capture to disk via the segment interface using NTAPI.

The following NTAPI functions are used:

Prerequisites

A Napatech capture accelerator is need to run this example. The ntservice.ini must have at least one HostBuffersRx defined. Below is an example of a minimum ini-file. It will create a 32MB RX hostbuffer from NUMA node 0.

[System]
TimestampFormat = NATIVE
[Adapter0]
AdapterType = NT20E
BusId = 00:0a:00.00
HostBuffersRx = [1,32,0]

Program flow

The following is required to perform capture of segments to disk:

  • #include/nt.h - Applications/Tools only need to include nt.h to obtain prototypes, macros etc. from NTAPI.
  • NT_Init(NTAPI_VERSION) - Initialize the NTAPI library. NTAPI_VERSION is a define that describes the version of the API described in the header files included by nt.h. NT_Init() will ask the NTAPI library to convert return data to the NTAPI_VERSION if possible. This will ensure that applications can run on NTAPI libraries of newer versions.
  • NT_NetRxOpen() - Open the stream using a stream ID. The stream ID must match the one used when creating the filter.
  • NT_NTPL() - Assign traffic to the stream. A stream does not return data until traffic is assigned to it by a filter. Stream IDs can be shared between other streams.
  • NT_NetRxRead() - Get the file header. A NT file header must be written to the beginning of the file after the last NT_NTPL() call has been made. Set NtNetRx_s.cmd=NT_NETRX_READ_CMD_GET_FILE_HEADER and issue the NT_NetRxRead() call. The fileheader is returned in NtNetRx_s.u.fileheader.data.
  • Create the capture file and write NT header. Use the OS specific functions to create a new capture file.
  • Optional step. Wait until we start seeing segments that are hit by the NTPL assign command. This is done to avoid getting segments that are not fully classified by the stream. NT_NetRxGet() is called with a timeout of 1000ms and will return NT_STATUS_TIMEOUT in case nothing is received within 1000ms and will return NT_SUCCESS when a segment is returned. Segments with NT_NET_GET_SEGMENTLENGTH()==0 can be returned so it is needed to check for the segment length before using data within the segment. The NT_NET_GET_SEGMENT_TIMESTAMP() macro can still be used on the empty segments. Return values different from that is an indication of an error. Segments that are prior to the expected time are released via NT_NetRxRelease().
  • NT_NetRxGet(), write to file and NT_NetRxRelease() - Receive segments, write to disk and release segments. The Segment macros are used to find the segment and length and timestamp of the segment:
  • NT_NetRxClose() - Close the stream when terminating. This will close the stream and release the NTPL assignment made on the hostbuffer.
  • Close captured file
  • NT_Done() - Close down the NTAPI library.

Code

/*
*
* Copyright 2020 Napatech A/S. All Rights Reserved.
*
* 1. Copying, modification, and distribution of this file, or executable
* versions of this file, is governed by the terms of the Napatech Software
* license agreement under which this file was made available. If you do not
* agree to the terms of the license do not install, copy, access or
* otherwise use this file.
*
* 2. Under the Napatech Software license agreement you are granted a
* limited, non-exclusive, non-assignable, copyright license to copy, modify
* and distribute this file in conjunction with Napatech SmartNIC's and
* similar hardware manufactured or supplied by Napatech A/S.
*
* 3. The full Napatech Software license agreement is included in this
* distribution, please see "NP-0405 Napatech Software license
* agreement.pdf"
*
* 4. Redistributions of source code must retain this copyright notice,
* list of conditions and the following disclaimer.
*
* THIS SOFTWARE IS PROVIDED "AS IS" WITHOUT ANY WARRANTIES, EXPRESS OR
* IMPLIED, AND NAPATECH DISCLAIMS ALL IMPLIED WARRANTIES INCLUDING ANY
* IMPLIED WARRANTY OF TITLE, MERCHANTABILITY, NONINFRINGEMENT, OR OF
* FITNESS FOR A PARTICULAR PURPOSE. TO THE EXTENT NOT PROHIBITED BY
* APPLICABLE LAW, IN NO EVENT SHALL NAPATECH BE LIABLE FOR PERSONAL INJURY,
* OR ANY INCIDENTAL, SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES WHATSOEVER,
* INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, CORRUPTION OR
* LOSS OF DATA, FAILURE TO TRANSMIT OR RECEIVE ANY DATA OR INFORMATION,
* BUSINESS INTERRUPTION OR ANY OTHER COMMERCIAL DAMAGES OR LOSSES, ARISING
* OUT OF OR RELATED TO YOUR USE OR INABILITY TO USE NAPATECH SOFTWARE OR
* SERVICES OR ANY THIRD PARTY SOFTWARE OR APPLICATIONS IN CONJUNCTION WITH
* THE NAPATECH SOFTWARE OR SERVICES, HOWEVER CAUSED, REGARDLESS OF THE THEORY
* OF LIABILITY (CONTRACT, TORT OR OTHERWISE) AND EVEN IF NAPATECH HAS BEEN
* ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. SOME JURISDICTIONS DO NOT ALLOW
* THE EXCLUSION OR LIMITATION OF LIABILITY FOR PERSONAL INJURY, OR OF
* INCIDENTAL OR CONSEQUENTIAL DAMAGES, SO THIS LIMITATION MAY NOT APPLY TO YOU.
*
*
*/
/**
* @example net/capture/capture_example.c
* @section capture_example_description Description
*
* This source file is an example of how to do capture to disk via the
* segment interface using NTAPI.
*
* The following NTAPI functions are used:
* - @ref NT_Init()
* - @ref NT_NetRxOpen()
* - @ref NT_NTPL()
* - @ref NT_NetRxRead()
* - @ref NT_NetRxGet()
* - @ref NT_NET_GET_SEGMENT_PTR()
* - @ref NT_NET_GET_SEGMENT_LENGTH()
* - @ref NT_NET_GET_SEGMENT_TIMESTAMP()
* - @ref NT_NetRxRelease()
* - @ref NT_NetRxClose()
* - @ref NT_Done()
* - @ref NT_ExplainError()
*
* @section capture_example_prerequisites Prerequisites
* A Napatech capture accelerator is need to run this example. The ntservice.ini
* must have at least one HostBuffersRx defined. Below is an example
* of a minimum ini-file. It will create a 32MB RX hostbuffer from
* NUMA node 0.
* @code
* [System]
* TimestampFormat = NATIVE
*
* [Adapter0]
* AdapterType = NT20E
* BusId = 00:0a:00.00
* HostBuffersRx = [1,32,0]
* @endcode
*
* @section capture_example_flow Program flow
* @{
* The following is required to perform capture of segments to disk:
* - \#include/nt.h - Applications/Tools only need to include @ref
* nt.h to obtain prototypes, macros etc. from NTAPI.
* - @ref NT_Init(@ref NTAPI_VERSION) - Initialize the NTAPI
* library. @ref NTAPI_VERSION is a define that describes the version
* of the API described in the header files included by @ref
* nt.h. NT_Init() will ask the NTAPI library to convert return data
* to the @ref NTAPI_VERSION if possible. This will ensure that
* applications can run on NTAPI libraries of newer versions.
* - @ref NT_NetRxOpen() - Open the stream using a stream ID.
* The stream ID must match the one used when creating the
* filter.
* - @ref NT_NTPL() - Assign traffic to the stream. A stream does not
* return data until traffic is assigned to it by a filter. Stream
* IDs can be shared between other streams.
* - NT_NetRxRead() - Get the file header. A NT file header must be
* written to the beginning of the file after the last NT_NTPL()
* call has been made. Set @ref NtNetRx_s::cmd "NtNetRx_s.cmd"=@ref
* NT_NETRX_READ_CMD_GET_FILE_HEADER and issue the NT_NetRxRead()
* call. The fileheader is returned in @ref
* NtNetRxFileHeader_s::data
* "NtNetRx_s.u.fileheader.data".
* - Create the capture file and write NT header. Use the OS specific
* functions to create a new capture file.
* - Optional step. Wait until we start seeing segments that are hit
* by the NTPL assign command. This is done to avoid getting
* segments that are not fully classified by the stream.
* NT_NetRxGet() is called with a timeout of 1000ms and will return
* NT_STATUS_TIMEOUT in case nothing is received within 1000ms and
* will return NT_SUCCESS when a segment is returned. Segments with NT_NET_GET_SEGMENTLENGTH()==0
* can be returned so it is needed to check for the segment length before using data within
* the segment. The NT_NET_GET_SEGMENT_TIMESTAMP() macro can still be used on the empty segments.
* Return values different from that is an indication of an error. Segments that
* are prior to the expected time are released via NT_NetRxRelease().
* - NT_NetRxGet(), write to file and NT_NetRxRelease() - Receive
* segments, write to disk and release segments. The @ref
* SegmentMacros are used to find the segment and length and
* timestamp of the segment:
* - @ref NT_NET_GET_SEGMENT_PTR() - Get a pointer to the segment.
* - @ref NT_NET_GET_SEGMENT_LENGTH() - Get length of the segment to store.
* - @ref NT_NET_GET_SEGMENT_TIMESTAMP() - The time the segment was delivered.
* - @ref _nt_net_build_pkt_netbuf() and @ref _nt_net_get_next_packet() are used to traverse
* packets inside a segment. This is usefull if inspection is needed before saving the
* segment.
* - NT_NetRxClose() - Close the stream when terminating.
* This will close the stream and release the NTPL assignment made on the hostbuffer.
* - Close captured file
* - @ref NT_Done() - Close down the NTAPI library.
*
*<hr>
* @section capture_example_code Code
* @}
*/
#include <nt.h>
#if defined(WIN32) || defined (WIN64)
#define snprintf _snprintf
#endif
struct ntpcap_ts_s {
uint32_t sec;
uint32_t usec;
};
struct ntpcap_hdr_s {
struct ntpcap_ts_s ts;
uint32_t caplen;
uint32_t wirelen;
};
int main(void)
{
FILE* hf; // File handle
int numSegments = 0; // The number of segments received
int numPackets = 0; // The number of packets received
uint64_t numBytes = 0; // The number of bytes received
uint64_t numBytesWire = 0; // The number of bytes received on the wire
char tmpBuffer[20]; // Buffer to build filter string
char errorBuffer[NT_ERRBUF_SIZE]; // Error buffer
int status; // Status variable
NtNetStreamRx_t hNetRx; // Handle to the RX stream
NtConfigStream_t hCfgStream; // Handle to a config stream
NtNtplInfo_t ntplInfo; // Return data structure from the NT_NTPL() call.
NtNetBuf_t hNetBuf; // Net buffer container. Segment data is returned in this when calling NT_NetRxGet().
NtNetRx_t readCmd; // NetRx read command structure.
struct NtNetBuf_s pktNetBuf; // Packet netbuf structure.
struct ntpcap_ts_s* pTs;
// Initialize the NTAPI library and thereby check if NTAPI_VERSION can be used together with this library
if ((status = NT_Init(NTAPI_VERSION)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_Init() failed: %s\n", errorBuffer);
return -1;
}
// Open a config stream to assign a filter to a stream ID.
if ((status = NT_ConfigOpen(&hCfgStream, "TestStream")) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_ConfigOpen() failed: %s\n", errorBuffer);
return -1;
}
// Assign traffic to stream ID 1 and mask all traffic matching the assign statement color=7.
if ((status = NT_NTPL(hCfgStream, "Assign[streamid=1;color=7] = port == 0", &ntplInfo, NT_NTPL_PARSER_VALIDATE_NORMAL)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NTPL() failed: %s\n", errorBuffer);
fprintf(stderr, ">>> NTPL errorcode: %X\n", ntplInfo.u.errorData.errCode);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]);
return -1;
}
// Used if PCAP header used
pTs = (struct ntpcap_ts_s *)(void *)&(ntplInfo.ts);
// Close the config stream
if ((status = NT_ConfigClose(hCfgStream)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_ConfigClose() failed: %s\n", errorBuffer);
return -1;
}
// Open stat stream
if ((status = NT_StatOpen(&hStat, "hStat")) != 0) {
fprintf(stderr, "Failed to create statistics stream: 0x%08X\n", status);
return -1;
}
// Reset stats
static NtStatistics_t statSet;
statSet.u.query_v3.poll = 1;
statSet.u.query_v3.clear = 1;
if ((status = NT_StatRead(hStat, &statSet))) {
fprintf(stderr, "Failed resetting statistics: 0x%08X\n", status);
return -1;
}
// Get a stream handle with stream ID 1. NT_NET_INTERFACE_SEGMENT specify that we will receive data in a segment based matter.
if ((status = NT_NetRxOpen(&hNetRx, "TestStream", NT_NET_INTERFACE_SEGMENT, 1, -1)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetRxOpen() failed: %s\n", errorBuffer);
return -1;
}
// Read the file header.
if ((status = NT_NetRxRead(hNetRx, &readCmd)) != NT_SUCCESS) {
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetRxRead() failed: %s\n", errorBuffer);
return -1;
}
// Create the capture file
if ((hf = fopen("capfile.ntcap", "w+b")) == NULL) {
perror("Failed to open capfile.ntcap");
return -1;
}
// Write the file header
fwrite(readCmd.u.fileheader.data, readCmd.u.fileheader.size, 1, hf);
// Optional step. Wait for the first packet that hit the NTPL assign command
printf("Waiting for the first segment\n");
while (1) {
if ((status = NT_NetRxGet(hNetRx, &hNetBuf, 1000)) != NT_SUCCESS) {
if ((status == NT_STATUS_TIMEOUT) || (status == NT_STATUS_TRYAGAIN)) {
// Timeouts are ok, we just need to wait a little longer for a segment
continue;
}
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetRxGet() failed: %s\n", errorBuffer);
return -1;
}
// We got a segment. Check if the timestamp is newer than when the NTPL assign command was applied
// If PCAP header configured, we need to convert the timestamps received
if ((((struct ntpcap_ts_s *)NT_NET_GET_SEGMENT_PTR(hNetBuf))->sec * 1000000 + ((struct ntpcap_ts_s *)NT_NET_GET_SEGMENT_PTR(hNetBuf))->usec) >
(pTs->sec * 1000000 + pTs->usec)) {
break; // Break out, we have received a segment that is received after the NTPL assign command was applied
}
} else
if ((((struct ntpcap_ts_s *)NT_NET_GET_SEGMENT_PTR(hNetBuf))->sec * 1000000000 + ((struct ntpcap_ts_s *)NT_NET_GET_SEGMENT_PTR(hNetBuf))->usec) >
(pTs->sec * 1000000000 + pTs->usec)) {
break; // Break out, we have received a segment that is received after the NTPL assign command was applied
}
} else {
if (NT_NET_GET_SEGMENT_TIMESTAMP(hNetBuf) > ntplInfo.ts) {
break; // Break out, we have received a segment that is received after the NTPL assign command was applied
}
}
// Release the segment as it is too "old".
if ((status = NT_NetRxRelease(hNetRx, hNetBuf)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetRxRelease() failed: %s\n", errorBuffer);
return -1;
}
}
// Write 10 segments to disk
while (1) {
if (NT_NET_GET_SEGMENT_LENGTH(hNetBuf)) {
// Optional step. Here all packets are inspected before storing
// Start by building a packet netbuf structure
{
struct ntpcap_hdr_s* pHdr = (struct ntpcap_hdr_s *)NT_NET_GET_SEGMENT_PTR(hNetBuf);
uint64_t segLength = NT_NET_GET_SEGMENT_LENGTH(hNetBuf);
uint64_t totSegmentBytes = 0;
while (totSegmentBytes < segLength) {
if (pHdr->wirelen) {
numPackets++;
numBytesWire += pHdr->wirelen;
}
totSegmentBytes += pHdr->caplen + (uint32_t)sizeof(struct ntpcap_hdr_s);
pHdr = (struct ntpcap_hdr_s *)((uint8_t *)pHdr + pHdr->caplen + sizeof(struct ntpcap_hdr_s));
}
} else {
_nt_net_build_pkt_netbuf(hNetBuf, &pktNetBuf);
do {
// Just count the amount of packets and wire length
numPackets++;
numBytesWire += NT_NET_GET_PKT_WIRE_LENGTH((&pktNetBuf));
} while (_nt_net_get_next_packet(hNetBuf, NT_NET_GET_SEGMENT_LENGTH(hNetBuf), &pktNetBuf)>0);
}
if ((status = (int)fwrite(NT_NET_GET_SEGMENT_PTR(hNetBuf), NT_NET_GET_SEGMENT_LENGTH(hNetBuf), 1, hf)) <= 0) {
perror("Failed writing segment");
return -1;
}
// Increment the number of segments processed.
numSegments++;
// Increment the bytes received
numBytes += NT_NET_GET_SEGMENT_LENGTH(hNetBuf);
printf("%016llx - Received segment of %lu bytes.\n",
(unsigned long long)NT_NET_GET_SEGMENT_TIMESTAMP(hNetBuf), NT_NET_GET_SEGMENT_LENGTH(hNetBuf));
}
// Release the current segment
if ((status = NT_NetRxRelease(hNetRx, hNetBuf)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetRxRelease() failed: %s\n", errorBuffer);
return -1;
}
if (numSegments == 10) {
break;
}
// Get the next segment
while (1) {
if ((status = NT_NetRxGet(hNetRx, &hNetBuf, 1000)) != NT_SUCCESS) {
if ((status == NT_STATUS_TIMEOUT) || (status == NT_STATUS_TRYAGAIN)) {
// Timeouts are ok, we just need to wait a little longer for a segment
continue;
}
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetRxGet() failed: %s\n", errorBuffer);
return -1;
}
break; // We got a segment
}
}
// Close the stream and release the hostbuffer. This will also remove the NTPL assignments performed.
NT_NetRxClose(hNetRx);
// Close the file
fclose(hf);
// Request stats
statSet.u.query_v3.poll = 1;
statSet.u.query_v3.clear = 0;
if ((status = NT_StatRead(hStat, &statSet)) != NT_SUCCESS) {
fprintf(stderr, "Failed reading statistics: 0x%08X\n", status);
return -1;
}
// Read drop counters for streamid 1
uint64_t totDropsPkts = statSet.u.query_v3.data.stream.streamid[1].drop.pkts;
uint64_t totDropsBytes = statSet.u.query_v3.data.stream.streamid[1].drop.octets;
// Close stat stream
NT_StatClose(hStat);
// Open a config stream to delete a filter.
if ((status = NT_ConfigOpen(&hCfgStream, "TestStream")) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_ConfigOpen() failed: %s\n", errorBuffer);
return -1;
}
// Delete the filter
snprintf(tmpBuffer, sizeof(tmpBuffer), "delete=%d", ntplInfo.ntplId);
if ((status = NT_NTPL(hCfgStream, tmpBuffer, &ntplInfo, NT_NTPL_PARSER_VALIDATE_NORMAL)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NTPL() failed: %s\n", errorBuffer);
fprintf(stderr, ">>> NTPL errorcode: %X\n", ntplInfo.u.errorData.errCode);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]);
fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]);
return -1;
}
// Close the config stream
if ((status = NT_ConfigClose(hCfgStream)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_ConfigClose() failed: %s\n", errorBuffer);
return -1;
}
printf("Drop: %16lu packets, %16lu bytes\n", totDropsPkts, totDropsBytes);
printf("Done: %16d segments, %16d packets, %16lu bytes, %16lu bytes on wire\n", numSegments, numPackets, numBytes, numBytesWire);
// Close down the NTAPI library
return 0;
}
//
// EOF
//