net/transmit_multifunction/transmit_multifunction_example.c

Reference Documentation

Platform
Intel® PAC
Napatech SmartNIC
Content Type
Reference Information
Capture Software Version
Link™ Capture Software 12.10
Napatech Software Suite: net/transmit_multifunction/transmit_multifunction_example.c
net/transmit_multifunction/transmit_multifunction_example.c

Description

This source file is an example of how to transmit data in segments using NTAPI. The example is able to transmit different packet sizes at different rates on different ports. The example shows how to create an incrementing or decrementing 32-bit pattern. When the rate is fixed, the example reuses segments to reduce the CPU load to virtually zero.

The following NTAPI functions are used:

NT_NET_SET_PKT_WIRE_LENGTH

NT_NetTxRelease()

Prerequisites

  • The ntservice.ini must have at least one HostBuffersTx defined. Below is an example of a minimum ini-file. It will create a 4MB TX hostbuffer from NUMA node 0.
[System]
TimestampFormat = NATIVE
[Adapter0]
AdapterType = NT20E2
BusId = 00:0a:00.00
HostBuffersTx = [1,4,0]

Program flow

The following is required to transmit packages:

  • #include/nt.h - Applications/Tools only need to include nt.h to obtain prototypes, macros etc. from NTAPI.
  • NT_Init(NTAPI_VERSION) - Initialize the NTAPI library. NTAPI_VERSION is a define that describes the version of the API described in the header files included by nt.h. NT_Init() will ask the NTAPI library to convert return data to the NTAPI_VERSION if possible. This will ensure that applications can run on NTAPI libraries of newer versions.
  • NT_ConfigOpen() - Open the configuration stream
  • NT_ConfigRead() - Read configuration data; used to retrieve the transmission limit on a port on a so-called 4GA adapter.
  • NT_ConfigWrite() - Write configuration data; used to set the transmission limit on a port on a so-called 4GA adapter.
  • NT_ConfigClose() - Close the configuration stream

NT_InfoOpen() - Open the information stream

  • NT_InfoRead() - Read information data. Used to decide the timestamp method, whether or not absolute Tx timing is supported, and whether or not it is enabled and the timstamp with.
  • NT_InfoClose() - Close the information stream.
  • NT_StatOpen() - Open the statistics stream.
  • NT_StatRead() - Read the statistics in order to clear the statistics.
  • NT_NetTxOpen() - Open a hostbuffer that can transmit packets to port 0.
  • NT_NetTxGet() - Get an empty segment from the wanted transmit port.
    • NT_NETTX_SEGMENT_OPTION_RAW - Parameter to NT_NetTxGet. Returns an empty segment.
    • _nt_net_build_pkt_netbuf - Returns a pointer to the first packet in the segment
    • NT_NET_SET_PKT_CLEAR_DESCR_EXT7 - Clear the packet descriptor. Assumes that an NT extended 7 packet descriptor is going to be used.
    • NT_NET_SET_PKT_DESCR_TYPE_EXT7 - Set the packet descriptor to be NT extend 7
    • NT_NET_SET_PKT_CAP_LENGTH - Set the captured length of the packet. This includes the packet data length and the length of the NT extended 7 packet descriptor
    • NT_NET_SET_PKT_WIRE_LENGTH - Set the wire length of the packet.
    • NT_NET_SET_PKT_RECALC_L2_CRC - Force the FPGA to recalculate the level 2 crc.
    • NT_NET_SET_PKT_TXNOW - Force the FPGA to transmit the packet immediately. The packet timestamp is ignored when setting txnow.
    • NT_NET_SET_PKT_TXSETCLOCK - Synchronize the TX clock when using absolut TX timing.
    • NT_TIMESTAMP_METHOD_EOF - Timestamp method status - The timestamp is placed at the end of the packet. The opposite NT_TIMESTAMP_METHOD_SOF - The timestamp is placed at the beginning og the packet.
    • _nt_net_get_next_packet - Get a pointer to the next packet in the segment. The remaining length of the segment is returned. Note: the segment must be completely filled otherwise it will not be sent and can cause the FPGA to crash.
    • NT_NET_SET_PKT_TXIGNORE - Force the FPGA to discard the packet.
    • NT_NET_GET_PKT_L2_PTR() is used to get the L2 pointer to the tx buffer, this is where the payload is placed.
    • NT_NET_SET_PKT_TIMESTAMP() is used to timestamp the packets. The packets will be transmitted based on the timing provided in the timestamps. The timestamp format must. be NT_TIMESTAMP_TYPE_NATIVE, hence TimestampFormat=NATIVE in ntservice.ini
    • NT_NET_GET_PKT_DESCR_LENGTH - Get the size of the packet descriptor.
    • NT_DESCR_EXT7_LENGTH - The sizeof the the extended 7 packet descriptor.
  • NT_NetTxRead - Read information about the tx host buffer.
  • NT_NetTxRelease() - Release the tx packet buffer. Once a tx buffer is released it will be transmitted according to the timestamp set in the packet.
  • NT_StatRead() - Read the statistics in order to see how many packets that are transmitted.
  • NT_NetTxClose() - Close the TX stream.
  • NT_StatClose() - Close the statistics stream.
  • NT_Done() - Close down the NTAPI library.

Code

/*
*
* Copyright 2023 Napatech A/S. All Rights Reserved.
*
* 1. Copying, modification, and distribution of this file, or executable
* versions of this file, is governed by the terms of the Napatech Software
* license agreement under which this file was made available. If you do not
* agree to the terms of the license do not install, copy, access or
* otherwise use this file.
*
* 2. Under the Napatech Software license agreement you are granted a
* limited, non-exclusive, non-assignable, copyright license to copy, modify
* and distribute this file in conjunction with Napatech SmartNIC's and
* similar hardware manufactured or supplied by Napatech A/S.
*
* 3. The full Napatech Software license agreement is included in this
* distribution, please see "NP-0405 Napatech Software license
* agreement.pdf"
*
* 4. Redistributions of source code must retain this copyright notice,
* list of conditions and the following disclaimer.
*
* THIS SOFTWARE IS PROVIDED "AS IS" WITHOUT ANY WARRANTIES, EXPRESS OR
* IMPLIED, AND NAPATECH DISCLAIMS ALL IMPLIED WARRANTIES INCLUDING ANY
* IMPLIED WARRANTY OF TITLE, MERCHANTABILITY, NONINFRINGEMENT, OR OF
* FITNESS FOR A PARTICULAR PURPOSE. TO THE EXTENT NOT PROHIBITED BY
* APPLICABLE LAW, IN NO EVENT SHALL NAPATECH BE LIABLE FOR PERSONAL INJURY,
* OR ANY INCIDENTAL, SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES WHATSOEVER,
* INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, CORRUPTION OR
* LOSS OF DATA, FAILURE TO TRANSMIT OR RECEIVE ANY DATA OR INFORMATION,
* BUSINESS INTERRUPTION OR ANY OTHER COMMERCIAL DAMAGES OR LOSSES, ARISING
* OUT OF OR RELATED TO YOUR USE OR INABILITY TO USE NAPATECH SOFTWARE OR
* SERVICES OR ANY THIRD PARTY SOFTWARE OR APPLICATIONS IN CONJUNCTION WITH
* THE NAPATECH SOFTWARE OR SERVICES, HOWEVER CAUSED, REGARDLESS OF THE THEORY
* OF LIABILITY (CONTRACT, TORT OR OTHERWISE) AND EVEN IF NAPATECH HAS BEEN
* ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. SOME JURISDICTIONS DO NOT ALLOW
* THE EXCLUSION OR LIMITATION OF LIABILITY FOR PERSONAL INJURY, OR OF
* INCIDENTAL OR CONSEQUENTIAL DAMAGES, SO THIS LIMITATION MAY NOT APPLY TO YOU.
*
*
*/
/**
* @example net/transmit_multifunction/transmit_multifunction_example.c
* @section transmit_multifunction_example_description Description
*
* This source file is an example of how to transmit data in segments using
* NTAPI. The example is able to transmit different packet sizes at different
* rates on different ports. The example shows how to create an incrementing
* or decrementing 32-bit pattern. When the rate is fixed, the example reuses
* segments to reduce the CPU load to virtually zero.
*
* The following NTAPI functions are used:
* - @ref NT_Init()
* - @ref NT_ConfigOpen
* - @ref NT_ConfigRead
* - @ref NT_ConfigWrite
* - @ref NT_ConfigClose
* - @ref NT_InfoOpen
* - @ref NT_InfoRead
* - @ref NT_InfoClose
* - @ref NT_StatOpen
* - @ref NT_NetTxRead
* - @ref NT_NetTxOpen()
* - @ref NT_NetTxGet()
* - @ref NT_NETTX_SEGMENT_OPTION_RAW
* - @ref _nt_net_build_pkt_netbuf
* - @ref NT_NET_SET_PKT_CLEAR_DESCR_EXT7
* - @ref NT_NET_SET_PKT_DESCR_TYPE_EXT7
* - @ref NT_NET_SET_PKT_CAP_LENGTH
* - @ref NT_NET_SET_PKT_CAP_LENGTH_NOALIGN
* - @ref NT_NET_SET_PKT_WIRE_LENGTH
* - @ref NT_NET_SET_PKT_RECALC_L2_CRC
* - @ref NT_NET_SET_PKT_TXNOW
* - @ref NT_NET_SET_PKT_TXSETCLOCK
* - @ref NT_TIMESTAMP_METHOD_EOF
* - @ref NT_NET_SET_PKT_TIMESTAMP
* - @ref _nt_net_get_next_packet
* - @ref NT_NET_SET_PKT_TXIGNORE
* - @ref NT_NET_GET_PKT_DESCR_LENGTH
* - @ref NT_DESCR_EXT7_LENGTH
* - @ref NT_NetTxRelease()
* - @ref NT_StatRead
* - @ref NT_NetTxClose()
* - @ref NT_StatClose
* - @ref NT_Done()
* - @ref NT_ExplainError()
*
* @section transmit_multifunction_example_prerequisites Prerequisites
* - The ntservice.ini must have at least one HostBuffersTx defined. Below is
* an example of a minimum ini-file. It will create a 4MB TX hostbuffer from
* NUMA node 0.
*
* @code
* [System]
* TimestampFormat = NATIVE
*
* [Adapter0]
* AdapterType = NT20E2
* BusId = 00:0a:00.00
* HostBuffersTx = [1,4,0]
* @endcode
*
* @section transmit_multifunction_example_flow Program flow
*
* The following is required to transmit packages:
* - \#include/nt.h - Applications/Tools only need to include @ref
* nt.h to obtain prototypes, macros etc. from NTAPI.
* - @ref NT_Init(@ref NTAPI_VERSION) - Initialize the NTAPI
* library. @ref NTAPI_VERSION is a define that describes the version
* of the API described in the header files included by @ref
* nt.h. NT_Init() will ask the NTAPI library to convert return data
* to the @ref NTAPI_VERSION if possible. This will ensure that
* applications can run on NTAPI libraries of newer versions.
* - @ref NT_ConfigOpen() - Open the configuration stream
* - @ref NT_ConfigRead() - Read configuration data; used to retrieve
* the transmission limit on a port on a so-called 4GA adapter.
* - @ref NT_ConfigWrite() - Write configuration data; used to set the
* transmission limit on a port on a so-called 4GA adapter.
* - @ref NT_ConfigClose() - Close the configuration stream
* - @ref NT_InfoOpen() - Open the information stream
* - @ref NT_InfoRead() - Read information data. Used to decide the timestamp method,
* whether or not absolute Tx timing is supported, and whether or not it is enabled and
* the timstamp with.
* - @ref NT_InfoClose() - Close the information stream.
* - @ref NT_StatOpen() - Open the statistics stream.
* - @ref NT_StatRead() - Read the statistics in order to clear the statistics.
* - @ref NT_NetTxOpen() - Open a hostbuffer that can transmit packets to port 0.
* - @ref NT_NetTxGet() - Get an empty segment from the wanted transmit port.
* - @ref NT_NETTX_SEGMENT_OPTION_RAW - Parameter to @ref NT_NetTxGet.
* Returns an empty segment.
* - @ref _nt_net_build_pkt_netbuf - Returns a pointer to the first
* packet in the segment
* - @ref NT_NET_SET_PKT_CLEAR_DESCR_EXT7 - Clear the packet descriptor. Assumes
* that an NT extended 7 packet descriptor is going to be used.
* - @ref NT_NET_SET_PKT_DESCR_TYPE_EXT7 - Set the packet descriptor to be NT extend 7
* - @ref NT_NET_SET_PKT_CAP_LENGTH - Set the captured length of the packet.
* This includes the packet data length and the length of the NT extended 7
* packet descriptor
* - @ref NT_NET_SET_PKT_WIRE_LENGTH - Set the wire length of the packet.
* - @ref NT_NET_SET_PKT_RECALC_L2_CRC - Force the FPGA to recalculate the level 2 crc.
* - @ref NT_NET_SET_PKT_TXNOW - Force the FPGA to transmit the packet immediately.
* The packet timestamp is ignored when setting txnow.
* - @ref NT_NET_SET_PKT_TXSETCLOCK - Synchronize the TX clock when using absolut
* TX timing.
* - @ref NT_TIMESTAMP_METHOD_EOF - Timestamp method status - The timestamp is
* placed at the end of the packet. The opposite @ref NT_TIMESTAMP_METHOD_SOF -
* The timestamp is placed at the beginning og the packet.
* - @ref _nt_net_get_next_packet - Get a pointer to the next packet in the segment.
* The remaining length of the segment is returned. <b>Note: the segment must be completely
* filled otherwise it will not be sent and can cause the FPGA to crash.</b>
* - @ref NT_NET_SET_PKT_TXIGNORE - Force the FPGA to discard the packet.
* - @ref NT_NET_GET_PKT_L2_PTR() is used to get the L2 pointer to the tx
* buffer, this is where the payload is placed.
* - @ref NT_NET_SET_PKT_TIMESTAMP() is used to timestamp the packets. The
* packets will be transmitted based on the timing provided in the
* timestamps. The timestamp format <b>must</b>. be
* NT_TIMESTAMP_TYPE_NATIVE, hence TimestampFormat=NATIVE in
* ntservice.ini
* - @ref NT_NET_GET_PKT_DESCR_LENGTH - Get the size of the packet descriptor.
* - @ref NT_DESCR_EXT7_LENGTH - The sizeof the the extended 7 packet descriptor.
* - @ref NT_NetTxRead - Read information about the tx host buffer.
* - @ref NT_NetTxRelease() - Release the tx packet buffer. Once a tx
* buffer is released it will be transmitted according to the
* timestamp set in the packet.
* - @ref NT_StatRead() - Read the statistics in order to see how many packets that are
* transmitted.
* - @ref NT_NetTxClose() - Close the TX stream.
* - @ref NT_StatClose() - Close the statistics stream.
* - @ref NT_Done() - Close down the NTAPI library.
*<hr>
* @section transmit_multifunction_example_code Code
*/
// Include this in order to access the Napatech API
#include <nt.h>
#if defined(__linux__) || defined(__FreeBSD__)
#include <unistd.h>
#include <signal.h>
#elif defined(WIN32) || defined(WIN64)
// ctrl, strings
#include <winsock2.h> // SetConsoleCtrlHandler(), for Windows 2000 Pro/Server or later; Sleep
#define strtoll _strtoi64
#endif
#include <argparse.h>
#include <string.h>
#include <assert.h>
#define MIN_PACKET_SIZE 64 // Minimum packet size to transmit
#define MAX_PACKET_SIZE 8192 // Maximum packet size to transmit
#define SEGMENT_LENGTH (1024*1024) // Segment size to transmit
// Macro to get a random number
#define RANDOM(_min_, _max_) ((uint32_t)_min_ + (uint32_t)((_max_-_min_+1) * (rand() /(RAND_MAX + 1.0))))
volatile int appRunning=1; // The application will run as long as appRunning==1
/**
* Print command line info
*/
static const char *usageText[] = {
"Syntax:\n"
"transmit_multifunction [-h][-p <port no.>][-r <tx rate>][-s <size>|r|i|d]\n"
"\nCommands:\n",
NULL};
static int opt_port = -1;
static char *opt_rate = NULL;
static char *opt_size = NULL;
/**
* Table of valid options.
*/
struct argparse_option arg_options[] = {
OPT_INTEGER('p', "port", &opt_port, "Output port", NULL, 0, 0, "port number"),
OPT_STRING( 'r', "rate", &opt_rate, "Transmit rate\n"
"0 = line rate\n"
"<rate> = <rate> Mbps\n"
"<rate>K = <rate> Kbps\n"
"<rate>M = <rate> Mbps\n"
"<rate>G = <rate> Gbps", NULL, 0, 0, "value"),
OPT_STRING( 's', "size", &opt_size, "Packet size - <value> = packet size (Size is aligned to 4 byte boundary\n"
"r = random size\n"
"i = increasing size from 64 bytes to 8192 bytes.\n"
"d = decreasing size from 8192 bytes to 64 bytes.", NULL, 0, 0, "value"),
};
/**
* The function called when user is pressing CTRL-C
*/
#if defined(WIN32) || defined (WIN64)
static BOOL WINAPI StopApplication(int sig)
#else
static void StopApplication(int sig)
#endif
{
#ifdef WIN32
// Force the application to stop.
return TRUE;
#else
if (sig == SIGINT)
// Force the application to stop.
#endif
}
static int is4GATxSupported = 0;
static int _Is4GATxSupported(NtInfoStream_t hInfo, uint8_t adapterNo)
{
NtInfo_t infoRead;
(void)memset(&infoRead, 0, sizeof(NtInfo_t));
if (NT_InfoRead(hInfo, &infoRead) != 0) {
return 0;
}
// 4GA adapters with ANL9 feature level support native TX with inline capabilities
// NOTE: 4GA adapters with ANL9 feature level doesn't support
// NT_NET_SET_PKT_DESCR_TYPE_EXT7 descriptor type
return 1;
}
return 0;
}
//
// Function to set the transmission rate/limit of a port on a 4GA
// adapter. (This program uses the packets' time stamp to control the
// transmission rate for older Napatech adapters.
//
static int _TxPortRateLimit(int port, uint64_t new, uint64_t *pOld)
{
NtConfigStream_t hConfig = NULL;
uint64_t oldRateLimit = 0;
int status = NT_ConfigOpen(&hConfig, "tx_port_rate_limit");
if (status) {
char errorBuffer[1024];
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_ConfigOpen failed: %s\n", errorBuffer);
return status;
}
NtConfig_t configRead;
(void)memset(&configRead, 0, sizeof(configRead));
configRead.u.portSettings_v2.portNo = (uint8_t)port;
//
// Set port transmission limit in two steps:
// First read the port's current settings,
//
status = NT_ConfigRead(hConfig, &configRead);
//
// and then set the new transmission limit.
//
if (status == NT_SUCCESS) {
oldRateLimit = configRead.u.portSettings_v2.data.txPortRateLimit;
status = NT_ConfigWrite(hConfig, &configRead);
}
if (status) {
char errorBuffer[1024];
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_ConfigRead/Write failed: %s\n", errorBuffer);
}
(void)NT_ConfigClose(hConfig);
if (status == NT_SUCCESS && pOld != NULL)
*pOld = oldRateLimit;
return status;
}
//
// Function that checks whether the packets in the TX host buffer have
// reached the adapter's FPGA.
//
static int _NetTxDataCheckHbInfo(NtNetStreamTx_t* phNetTx, bool bWaitDone)
{
do {
NtNetTx_t ntNetTx;
const int status = NT_NetTxRead(*phNetTx, &ntNetTx);
if (status == NT_SUCCESS) {
assert(ntNetTx.u.hbInfo.numHostBuffers == 1);
const size_t nHbSizeTotal = ntNetTx.u.hbInfo.aHostBuffer[0].size;
const size_t nHbSizeAvail = ntNetTx.u.hbInfo.aHostBuffer[0].available;
const size_t nHbSizeRel = ntNetTx.u.hbInfo.aHostBuffer[0].released;
const size_t nHbSizeDeq = ntNetTx.u.hbInfo.aHostBuffer[0].dequeued;
if (nHbSizeAvail == nHbSizeTotal && nHbSizeRel == 0 && nHbSizeDeq == 0) {
return 0; // zero int is success
}
} else {
char errorBuffer[1024];
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetTxRead failed: %s\n", errorBuffer);
return -1; // negative int is error
}
if (bWaitDone) {
#if defined(__linux__) || defined(__FreeBSD__)
(void)usleep(1000); // pause to avoid busy waiting
#elif defined(WIN32) || defined(WIN64)
Sleep(1); // 1 milli-sec
#endif
}
} while (bWaitDone && appRunning);
return 1; // positive int means `not done`
}
//
//
//
int main(int argc, const char *argv[])
{
char errorBuffer[NT_ERRBUF_SIZE]; // Error buffer
int status; // Status variable
NtNetStreamTx_t hNetTx; // Handle to the TX stream
NtNetBuf_t hNetBufTx; // Net buffer container. Used when getting a transmit segment
struct NtNetBuf_s pktNetBuf; // Net buffer container. Used when getting a transmit buffer
NtStatStream_t hStatStream; // Statistics stream handle
NtStatistics_t hStat; // Stat handle.
uint32_t i; // Counter
uint32_t payload = 0; // Payload content/counter
uint32_t currentPacketSize = MIN_PACKET_SIZE; // Packet size. Default set to MIN_PACKET_SIZE
uint32_t txPacketSize; // Size of next packet to transmit
uint32_t segmentsSent = 0; // Number of segments sent
uint64_t packetsSent = 0; // Number of packets sent.
uint32_t txPort = 0; // Transmit port
uint32_t txSize = MIN_PACKET_SIZE; // Transmit packet size
uint32_t txType = 0; // Transmit type - Increasing, decreasing or random size
uint64_t txRate = 0; // Transmit rate
NtInfoStream_t hInfo; // Handle to an info stream
NtInfo_t infoRead; // Info data
uint64_t ts=0; // Timestamp counter
uint64_t tsR=0; // The 10ns roundup time that the adapter is actually to run
int firstPacket = 0; // First packet to transmit
enum NtTimestampMethod_e timestampMethod; // Timestamp method
enum NtTxTimingMethod_e txTiming; // TX mode - Do we use absolut or relative mode
int timestamp64BitSupport; // Is 64 bit timestamp supported
uint32_t maxPacketsInSegment; // Max number of packets we want in the segment
uint32_t maxPacketsInAdapter; // Max number of packets we want in the adapter
struct argparse argparse;
#define NUM_MB 10 /* Number of segments to pre-fill/reuse */
void *seg_hdr_ptr[NUM_MB];
uint32_t packetsInSegment[NUM_MB]; // Number of packets in current segment
const int min_hb_size_in_mb = NUM_MB;
uint32_t reuse_idx;
printf("\nNapatech example: Segment based transmit\n");
printf("----------------------------------------\n\n");
(void)memset(seg_hdr_ptr, 0, sizeof(seg_hdr_ptr)); /* Make ptrs NULL */
(void)memset(packetsInSegment, 0, sizeof(packetsInSegment));
// Register ctrl+c handler so we are able to stop again
#if defined(WIN32) || defined (WIN64)
SetConsoleCtrlHandler((PHANDLER_ROUTINE)StopApplication, TRUE);
#else
struct sigaction newaction; // Ctrl+c signal handler container
memset(&newaction, 0, sizeof(newaction));
newaction.sa_handler = StopApplication;
if (sigaction(SIGINT, &newaction, NULL) < 0) {
fprintf(stderr, "Failed to register SIGINT sigaction.\n");
exit(-1);
}
#endif
argparse_init(&argparse, arg_options, usageText, 0);
argparse_parse(&argparse, argc, argv);
if (opt_port != -1) {
txPort = (uint32_t)opt_port;
}
if (opt_rate != NULL) {
int len;
char *endptr;
txRate=strtoll(opt_rate, &endptr, 0);
len = (int)strlen(endptr);
if (len > 1) {
fprintf(stderr, "\"-r %s\" is not valid.\n", opt_rate);
return 1;
}
else if (len == 0) {
// value is in megabit;
txRate*=1000000;
}
else {
switch (endptr[0]) {
case 'K':
// Rate is in kilo bit per second.
txRate*=1000;
break;
case 'M':
// Rate is in Mega bit per second.
txRate*=1000000;
break;
case 'G':
// Rate is in Mega bit per second.
txRate*=1000000000;
break;
default:
fprintf(stderr, "\"-r %s\" is not valid.\n", opt_rate);
return 1;
}
}
}
if (opt_size != NULL) {
if (strlen(opt_size) > 1) {
// Align packet size to uint32_t
txSize = (atoi(opt_size)+3)&~3;
txType = 0;
if (txSize < MIN_PACKET_SIZE || txSize > MAX_PACKET_SIZE) {
fprintf(stderr, ">>> Error: Illegal packet size: %d\n", txSize);
exit(1);
}
}
else {
switch (opt_size[0]) {
case 'i':
txType = 'i';
break;
case 'd':
txType = 'd';
break;
case 'r':
txType = 'r';
break;
default:
txType = 'r';
break;
}
}
}
if (txRate < 10000000 && txRate != 0) {
// When transmitting slowly we don't want to fill the adapter with data as we
// need to wait until all data are transmitted. When transmitting slower
// than 10M we want to fill the segments partly and we do only want to send the
// segments to the adapter when some the data is transmitted. If sending all the
// segments to the adapter, they will be queued internally in the adapter.
// We only want <maxPacketsInSegment> packets in each segment and we only want
// the send the segment to the adapter when adapter has less than <maxPacketsInAdapter>
// packets waiting to be transmitted.
maxPacketsInSegment = (uint32_t)(txRate / 1000);
maxPacketsInAdapter = 3 * maxPacketsInSegment;
}
else {
// We are going to transmit fast, so there are no limits. Just fill the
// segments and send them to the adapter as quicky as possible.
maxPacketsInSegment = 0xFFFFFFFF;
maxPacketsInAdapter = 0xFFFFFFFF;
}
// Print the settings used.
printf("Transmit port: %d\n", txPort);
if (txRate == 0) {
printf("Transmit Rate: Line rate\n");
}
else {
printf("Transmit Rate: %llu bits/second\n", (unsigned long long int)txRate);
}
if (txType == 0) {
printf("Transmit size: %d bytes\n\n", txSize);
printf("The program will pre-fill/reuse %d TX segments to save CPU.\n", NUM_MB);
}
else {
switch (txType) {
case 'i':
printf("Transmit size: Increasing from %d bytes to %d bytes\n\n", MIN_PACKET_SIZE, MAX_PACKET_SIZE);
break;
case 'd':
printf("Transmit size: Decreasing from %d bytes to %d bytes\n\n", MAX_PACKET_SIZE, MIN_PACKET_SIZE);
break;
case 'r':
printf("Transmit size: Random bewteen %d bytes and %d bytes\n\n", MIN_PACKET_SIZE, MAX_PACKET_SIZE);
break;
}
}
// Initialize the NTAPI library and thereby check if NTAPI_VERSION can be used together with this library
if ((status = NT_Init(NTAPI_VERSION)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_Init() failed: %s\n", errorBuffer);
return status;
}
// Open the info stream to get the timestamp method.
if ((status = NT_InfoOpen(&hInfo, "Info")) != 0) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_InfoOpen failed: %s\n", errorBuffer);
return status;
}
// Find the adapter on which the port is placed.
infoRead.u.port_v9.portNo = (uint8_t) txPort;
if ((status = NT_InfoRead(hInfo, &infoRead)) != 0) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_InfoRead failed: %s\n", errorBuffer);
NT_InfoClose(hInfo);
return status;
}
// Do we have 64 bit timestamp support
// Check whether or not TX is supported on the selected port
fprintf(stderr, "Transmit is not possible on the selected port %d\n", txPort);
NT_InfoClose(hInfo);
return 1;
}
// Check which settings that are enabled.
timestampMethod = infoRead.u.port_v9.data.adapterInfo.timestampMethod;
txTiming = infoRead.u.port_v9.data.adapterInfo.txTiming;
// Check whether 4GA TX is supported. If yes, EXT7-desc is not supported
// Check transmission port is up
if (infoRead.u.port_v9.data.state != NT_LINK_STATE_UP) {
fprintf(stderr, "Port %d is down.\n", txPort);
(void)NT_InfoClose(hInfo);
return 1;
}
// Close info stream again
NT_InfoClose(hInfo);
// Check that the adapter has support for 64bit timestamp
if (timestamp64BitSupport == 0 && txRate < 1000000) {
fprintf(stderr, "The adapter does not support %lld bits/second\n", (long long int)txRate);
return 1;
}
// Open the stat stream.
if ((status = NT_StatOpen(&hStatStream, "ExampleStat")) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_StatOpen() failed: %s\n", errorBuffer);
return status;
}
// Read the statistics counters to clear the statistics
hStat.u.query_v3.poll=0; // Wait for a new set
hStat.u.query_v3.clear=1; // Clear statistics
if ((status = NT_StatRead(hStatStream, &hStat)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_StatRead() failed: %s\n", errorBuffer);
(void)NT_StatClose(hStatStream);
return status;
}
// Open a TX hostbuffer from TX host buffer pool on the NUMA node of the adapter with the txPort
if (txType == 0) {
status = NT_NetTxOpen(&hNetTx, "TxStreamPort", (uint64_t)0x1<<txPort, NT_NETTX_NUMA_ADAPTER_HB, min_hb_size_in_mb); /* Prefill segments */
} else {
status = NT_NetTxOpen(&hNetTx, "TxStreamPort", (uint64_t)0x1<<txPort, NT_NETTX_NUMA_ADAPTER_HB, 0);
}
if (status != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetTxOpen() failed: %s\n", errorBuffer);
NT_StatClose(hStatStream);
return status;
}
//
// Set transmission limit
//
uint64_t oldTxRate;
int bTxPortRateLimitSet = 0;
if (is4GATxSupported && txRate > 0) {
status = _TxPortRateLimit(txPort, txRate, &oldTxRate);
bTxPortRateLimitSet = status == NT_SUCCESS;
}
const size_t ntDescrLength =
is4GATxSupported ? NT_DESCR_NT_LENGTH : NT_DESCR_EXT7_LENGTH;
// Create segments and transmit to port txPort
while (status == NT_SUCCESS && appRunning==1) {
uint64_t spaceLeftInSegment = SEGMENT_LENGTH;
// Get a TX segment of size SEGMENT_LENGTH
status = NT_NetTxGet(hNetTx, &hNetBufTx, txPort, SEGMENT_LENGTH, NT_NETTX_SEGMENT_OPTION_RAW, 1000);
if (status == NT_STATUS_TIMEOUT) {
status = NT_SUCCESS;
continue;
}
if (status != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetTxGet() failed: %s\n", errorBuffer);
continue;
}
reuse_idx = segmentsSent % NUM_MB;
// Get a packet buffer pointer from the segment.
_nt_net_build_pkt_netbuf(hNetBufTx, &pktNetBuf);
if (txType != 0 /* pre-fill/reuse not done */ ||
seg_hdr_ptr[reuse_idx] == NULL /* pre-fill segment */) {
// Add packets to the segment.
// IMPORTANT: The segment must be compeletely filled before it can be sent.
// Failing to do this causes the adapter to malfunction. In order to fill
// the segment a 64 byte or larger dummy packet is added to the end of the
// segment.
packetsInSegment[reuse_idx] = 0;
for (;;) {
// Build the packet
NT_NET_SET_PKT_CLEAR_DESCR_NT((&pktNetBuf)); // Clear the packet descriptor. Must be done first.
NT_NET_SET_PKT_DESCR_TYPE_NT((&pktNetBuf)); // Set the packet to use Std0 descriptor
}
else {
NT_NET_SET_PKT_CLEAR_DESCR_EXT7((&pktNetBuf)); // Clear the packet descriptor. Must be done first.
NT_NET_SET_PKT_DESCR_TYPE_EXT7((&pktNetBuf)); // Set the packet to use extended descriptor 7
}
NT_NET_SET_PKT_RECALC_L2_CRC((&pktNetBuf), 1); // Force the FPGA to calculate a new L2 crc.
NT_NET_SET_PKT_TXPORT((&pktNetBuf), txPort); // Only necessary if transmitting on a capture adapter.
NT_NET_UPDATE_PKT_L2_PTR((&pktNetBuf)); // Update the L2 ptr of the packet to match the new descriptor
// We only want <maxPacketsInSegment> in each segment.
// When we reach the limit, then fill the rest of the
// segments with dummy packets
if (packetsInSegment[reuse_idx] < maxPacketsInSegment) {
// Set the capture/stored length of the packet.
// This includes the packet header and the alignment padding
NT_NET_SET_PKT_CAP_LENGTH_NOALIGN((&pktNetBuf), (uint16_t)currentPacketSize);
} else {
NT_NET_SET_PKT_CAP_LENGTH((&pktNetBuf), (uint16_t)currentPacketSize);
}
NT_NET_SET_PKT_WIRE_LENGTH((&pktNetBuf), (uint16_t)currentPacketSize);// Set the wire length of the packet.
if (bTxPortRateLimitSet || txRate == 0) {
// Transmit at the port's configured rate
NT_NET_SET_PKT_TXNOW((&pktNetBuf), 1);
}
else {
// Set the timestamp so the rate can be controlled
// If packets are timestamped when last/first octet is received
// the timestamp must reflect this.
if (timestampMethod==NT_TIMESTAMP_METHOD_EOF) {
ts += (((uint64_t)NT_NET_GET_PKT_WIRE_LENGTH((&pktNetBuf))+20ULL)*8ULL*1000000000ULL)/txRate;
// Round up to nearest 10ns
tsR+=((((((uint64_t)NT_NET_GET_PKT_WIRE_LENGTH((&pktNetBuf))+20ULL)*8ULL*1000000000ULL)/txRate)+9)/10)*10;
// Compensate for the round up error?
if ((tsR-ts)>10) {
tsR-=10;
}
NT_NET_SET_PKT_TIMESTAMP((&pktNetBuf), tsR/10);
} else {
NT_NET_SET_PKT_TIMESTAMP((&pktNetBuf), tsR/10);
ts += (((uint64_t)NT_NET_GET_PKT_WIRE_LENGTH((&pktNetBuf))+20ULL)*8ULL*1000000000ULL)/txRate;
// Round up to nearest 10ns
tsR+=((((((uint64_t)NT_NET_GET_PKT_WIRE_LENGTH((&pktNetBuf))+20ULL)*8ULL*1000000000ULL)/txRate)+9)/10)*10;
// Compensate for the round up error?
if ((tsR-ts)>10) {
tsR-=10;
}
}
if (firstPacket == 0) {
if (txTiming == NT_TX_TIMING_ABSOLUTE) {
// Use absolut tx timing
// Setup first packet to transmit. Only the first packet must be setup
// to Synchronize tx clock. TXNOW must be set to 0 for absolut TX timing
// to work. The default is 0.
NT_NET_SET_PKT_TXNOW((&pktNetBuf), 0); // Wait for tx delay before the packet is sent
NT_NET_SET_PKT_TXSETCLOCK((&pktNetBuf), 1); // Synchronize tx clock to timestamp in first packet.
}
else {
// Use Legacy mode/Relative tx timing.
// First packet must be sent with txnow=1
NT_NET_SET_PKT_TXNOW((&pktNetBuf), 1);
}
firstPacket = 1;
}
}
// Create a packet with an incrementing 32bit pattern.
// Note that the packet does not contain valid MAC address etc.
for (i = 0; i < currentPacketSize/4; i++) {
*((uint32_t*)NT_NET_GET_PKT_L2_PTR((&pktNetBuf))+i) = payload++;
}
// Packets in the segment
packetsInSegment[reuse_idx]++;
// Get the next packet size to transmit.
switch (txType) {
default:
case 0:
// static packet size
currentPacketSize = txSize;
break;
case 'i':
// Increment packet size
currentPacketSize += (uint32_t)sizeof(uint32_t);
if (currentPacketSize > MAX_PACKET_SIZE) {
currentPacketSize = MIN_PACKET_SIZE;
}
break;
case 'd':
// Decrement packet size
currentPacketSize -= (uint32_t)sizeof(uint32_t);
if (currentPacketSize < MIN_PACKET_SIZE) {
currentPacketSize = MAX_PACKET_SIZE;
}
break;
case 'r':
// Random packet size aligned packet size to uint32_t
currentPacketSize=(RANDOM(MIN_PACKET_SIZE, MAX_PACKET_SIZE)+3)&~3;
break;
}
txPacketSize = currentPacketSize;
}
else {
// We don't want to fill more packets into the segment, so create a
// dummy packet to fill up the segment.
// Set size of the dummy packet.
NT_NET_SET_PKT_CAP_LENGTH_NOALIGN((&pktNetBuf), (uint16_t)128);
} else {
NT_NET_SET_PKT_CAP_LENGTH((&pktNetBuf), (uint16_t)128);
}
NT_NET_SET_PKT_WIRE_LENGTH((&pktNetBuf), (uint16_t)128);// Size of the dummy packet.
NT_NET_SET_PKT_TXIGNORE((&pktNetBuf), 1); // Do not transmit the packet
txPacketSize = 128;
}
// Move the pointer to next packet and get the length of the remining space in the segment
spaceLeftInSegment = _nt_net_get_next_packet(hNetBufTx, NT_NET_GET_SEGMENT_LENGTH(hNetBufTx), &pktNetBuf);
// Check if we have space for excately one packet.
if (spaceLeftInSegment == txPacketSize + ntDescrLength) {
// There is exactly space for one packet more.
continue;
}
// Check if there is space for a packet of currentPacketSize + extended 7 header
// and a 64 byte dummy packet + extended header 7. We must be sure that we are able to create a dummy
// packet to be able to close/fill the segment.
if (spaceLeftInSegment < (txPacketSize + ntDescrLength) + (64 + ntDescrLength)) {
if (spaceLeftInSegment > 0) {
// No space for another packet. But we need to deliver a completely filled segment,
// therefore we must create a dummy packet to fill the segment.
NT_NET_SET_PKT_CLEAR_DESCR_NT((&pktNetBuf)); // Clear the packet descriptor. Must be done first.
NT_NET_SET_PKT_DESCR_TYPE_NT((&pktNetBuf)); // Set the packet to use Std0 descriptor
}
else {
NT_NET_SET_PKT_CLEAR_DESCR_EXT7((&pktNetBuf)); // Clear the packet descriptor. Must be done first.
NT_NET_SET_PKT_DESCR_TYPE_EXT7((&pktNetBuf)); // Set the packet to use extended descriptor 7
}
spaceLeftInSegment -= ntDescrLength;
// Make space for packet header
NT_NET_SET_PKT_CAP_LENGTH_NOALIGN((&pktNetBuf), (uint16_t)spaceLeftInSegment);
} else {
NT_NET_SET_PKT_CAP_LENGTH((&pktNetBuf), (uint16_t)spaceLeftInSegment);
}
NT_NET_SET_PKT_WIRE_LENGTH((&pktNetBuf), (uint16_t)spaceLeftInSegment);
// We don't want to send this packet. Set tx ignore to tell the FPGA,
// not to send the packet.
NT_NET_SET_PKT_TXIGNORE((&pktNetBuf), 1);
// As this is a dummy packet. We don't care about the payload.
}
// break to send the segment
break;
}
} // end-for `build packet`
}
if (txType == 0) { /* pre-fill/reuse */
if (seg_hdr_ptr[reuse_idx] == NULL) {
/* Remember that segment is now initialized */
seg_hdr_ptr[reuse_idx] = (void *)hNetBufTx->hHdr;
} else {
/* Check integrity of driver/application */
assert(seg_hdr_ptr[reuse_idx] == hNetBufTx->hHdr);
}
}
// Release the TX buffer and the segment will be transmitted
if ((status = NT_NetTxRelease(hNetTx, hNetBufTx)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetTxRelease() failed: %s\n", errorBuffer);
continue;
}
// Update our statistics counters
segmentsSent++;
packetsSent += packetsInSegment[reuse_idx];
if (((segmentsSent & 0xF) == 0) || (txRate < 10000000)) {
printf("Sent: %8d segments\r", segmentsSent);
fflush(stdout);
}
if (maxPacketsInSegment != 0xFFFFFFFF) {
// If we are using slow transmit, we don't want to fill up the adapter with
// packets. Therefore we only want to have <maxPacketsInAdapter> number of
// packets waiting in the adapter to be transmitted. Taht means that the
// difference between the number of packets sent to the adapter and the
// number of packets TX'ed by the adapter must be less than <maxPacketsInAdapter>
// before we are sending more packets to the adapter.
do {
// Read the statistics counters
hStat.u.query_v3.poll=0; // Wait for a new set
hStat.u.query_v3.clear=0; // Clear statistics
if ((status = NT_StatRead(hStatStream, &hStat)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_StatRead() failed: %s\n", errorBuffer);
}
hStat.u.query_v3.data.port.aPorts[txPort].tx.valid.RMON1 = 0;
} while (hStat.u.query_v3.data.port.aPorts[txPort].tx.valid.RMON1 &&
(maxPacketsInAdapter < (packetsSent - hStat.u.query_v3.data.port.aPorts[txPort].tx.RMON1.pkts))
&& (appRunning==1) && status == NT_SUCCESS);
}
}
if (status == NT_SUCCESS) {
//
// Wait for the adapter to transmit all of the buffered packets.
//
appRunning=1; // Wait for completion or ctrl-c
(void)_NetTxDataCheckHbInfo(&hNetTx, true /* Wait for completion */);
}
if (packetsSent > 0)
printf("\n\nSent: %d segments and %llu packets\n\n", segmentsSent, (long long int)packetsSent);
// Close the TX stream
(void)NT_NetTxClose(hNetTx);
// Close the stat stream
(void)NT_StatClose(hStatStream);
//
// Restore transmission limit
//
if (bTxPortRateLimitSet) {
(void)_TxPortRateLimit(txPort, oldTxRate, NULL);
}
// Close down the NTAPI library
return status;
}