net/replayGS/replayGS_example.c

Reference Documentation

Platform
Intel® PAC
Napatech SmartNIC
Content Type
Reference Information
Capture Software Version
Link™ Capture Software 12.10
Napatech Software Suite: net/replayGS/replayGS_example.c
net/replayGS/replayGS_example.c

Description

This source file is an example of how to replay a capture file onto a port using NTAPI and the global synchronous transmit feature.

With this example is it possible make a synchronous transmit between several accelerators placed in either same server or different serves. In order for the synchronous transmit to work all adapters must be synchronized using timesync.

The following NTAPI functions are used:

Prerequisites

  • A capture file, that has been captured with the net/capture/capture_example.c example.
  • All adapters must support the global sync feature.
  • TxTiming must be set to ABSOLUTE in ntservice.ini
  • Timestamp format must be NATIVE_UNIX
  • ntservice.ini must have at least one HostBuffersTx defined. Below is an example of a minimum ini-file. It will create a 32MB TX hostbuffer from NUMA node 0.
    [System]
    TimestampFormat = NATIVE_UNIX
    [Adapter0]
    AdapterType = NT20E2
    BusId = 00:0a:00.00
    HostBuffersTx = [1,32,0]
    TxTiming = ABSOLUTE

Program flow

The following is required to perform synchronized replay of a captured file:


Code

/*
*
* Copyright 2023 Napatech A/S. All Rights Reserved.
*
* 1. Copying, modification, and distribution of this file, or executable
* versions of this file, is governed by the terms of the Napatech Software
* license agreement under which this file was made available. If you do not
* agree to the terms of the license do not install, copy, access or
* otherwise use this file.
*
* 2. Under the Napatech Software license agreement you are granted a
* limited, non-exclusive, non-assignable, copyright license to copy, modify
* and distribute this file in conjunction with Napatech SmartNIC's and
* similar hardware manufactured or supplied by Napatech A/S.
*
* 3. The full Napatech Software license agreement is included in this
* distribution, please see "NP-0405 Napatech Software license
* agreement.pdf"
*
* 4. Redistributions of source code must retain this copyright notice,
* list of conditions and the following disclaimer.
*
* THIS SOFTWARE IS PROVIDED "AS IS" WITHOUT ANY WARRANTIES, EXPRESS OR
* IMPLIED, AND NAPATECH DISCLAIMS ALL IMPLIED WARRANTIES INCLUDING ANY
* IMPLIED WARRANTY OF TITLE, MERCHANTABILITY, NONINFRINGEMENT, OR OF
* FITNESS FOR A PARTICULAR PURPOSE. TO THE EXTENT NOT PROHIBITED BY
* APPLICABLE LAW, IN NO EVENT SHALL NAPATECH BE LIABLE FOR PERSONAL INJURY,
* OR ANY INCIDENTAL, SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES WHATSOEVER,
* INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, CORRUPTION OR
* LOSS OF DATA, FAILURE TO TRANSMIT OR RECEIVE ANY DATA OR INFORMATION,
* BUSINESS INTERRUPTION OR ANY OTHER COMMERCIAL DAMAGES OR LOSSES, ARISING
* OUT OF OR RELATED TO YOUR USE OR INABILITY TO USE NAPATECH SOFTWARE OR
* SERVICES OR ANY THIRD PARTY SOFTWARE OR APPLICATIONS IN CONJUNCTION WITH
* THE NAPATECH SOFTWARE OR SERVICES, HOWEVER CAUSED, REGARDLESS OF THE THEORY
* OF LIABILITY (CONTRACT, TORT OR OTHERWISE) AND EVEN IF NAPATECH HAS BEEN
* ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. SOME JURISDICTIONS DO NOT ALLOW
* THE EXCLUSION OR LIMITATION OF LIABILITY FOR PERSONAL INJURY, OR OF
* INCIDENTAL OR CONSEQUENTIAL DAMAGES, SO THIS LIMITATION MAY NOT APPLY TO YOU.
*
*
*/
/**
* @example net/replayGS/replayGS_example.c
* @section replayGS_example_description Description
*
* This source file is an example of how to replay a capture file onto
* a port using NTAPI and the global synchronous transmit feature.
*
* With this example is it possible make a synchronous transmit between
* several accelerators placed in either same server or different serves. In order
* for the synchronous transmit to work all adapters must be synchronized using
* timesync.
*
* The following NTAPI functions are used:
* - @ref NT_Init()
* - @ref NT_InfoOpen()
* - @ref NT_InfoRead()
* - @ref NT_InfoClose()
* - @ref NT_ConfigOpen()
* - @ref NT_ConfigRead()
* - @ref NT_ConfigWrite()
* - @ref NT_ConfigClose()
* - @ref NT_NetFileOpen()
* - @ref NT_NetTxOpen()
* - @ref NT_NTPL()
* - @ref NT_NetFileGet()
* - @ref NT_NetTxGet()
* - @ref NT_NET_GET_SEGMENT_PTR()
* - @ref NT_NET_GET_SEGMENT_LENGTH()
* - @ref NT_NetTxRelease()
* - @ref NT_NetFileRelease()
* - @ref NT_NetFileClose()
* - @ref NT_NetTxClose()
* - @ref NT_ExplainError()
*
* @section replayGS_example_prerequisites Prerequisites
* - A capture file, that has been captured with the @ref
* net/capture/capture_example.c "net/capture/capture_example.c" example.
* - All adapters must support the global sync feature.
* - TxTiming must be set to ABSOLUTE in ntservice.ini
* - Timestamp format must be NATIVE_UNIX
* - ntservice.ini must have at least one HostBuffersTx
* defined. Below is an example of a minimum ini-file. It will
* create a 32MB TX hostbuffer from NUMA node 0.
* @code
* [System]
* TimestampFormat = NATIVE_UNIX
*
* [Adapter0]
* AdapterType = NT20E2
* BusId = 00:0a:00.00
* HostBuffersTx = [1,32,0]
* TxTiming = ABSOLUTE
* @endcode
*
* @section replayGS_example_flow Program flow
* @{
* The following is required to perform synchronized replay of a captured file:
* - \#include/nt.h - Applications/Tools only need to include @ref
* nt.h to obtain prototypes, macros etc. from NTAPI.
* - @ref NT_Init(@ref NTAPI_VERSION) - Initialize the NTAPI
* library. @ref NTAPI_VERSION is a define that describes the version
* of the API described in the header files included by @ref
* nt.h. NT_Init() will ask the NTAPI library to convert return data
* to the @ref NTAPI_VERSION if possible. This will ensure that
* applications can run on NTAPI libraries of newer versions.
* - @ref NT_InfoOpen() - Open the information stream.
* - @ref NT_InfoRead() - Read @ref NtInfoAdapter_v6_s "the adapter information" to check whether or not
* we are using absolut TX timing. The @ref NtTxTimingMethod_e "txTiming"
* tells whether relative or absolut TX timing is enabled.
* - @ref NT_InfoClose() - Close the information stream.
* - @ref NT_ConfigOpen() - Open the config stream.
* - @ref NT_ConfigRead() - Read the adapter timestamp from the adapter. Needed
* in order to synchronize the transmit.
* - @ref NT_ConfigWrite() - Write the synchronize offset to the adapter.
* - @ref NT_ConfigClose() - Close the config stream.
* - @ref NT_NetFileOpen() - Open the captured file and assign it to a stream.
* - @ref NT_NetTxOpen() - Open a hostbuffer than can transmit packets to port 0.
* - @ref NT_NetFileGet() - Get a segment from the file. This call will
* return NT_SUCCESS upon return of a segment and
* @ref NT_STATUS_END_OF_FILE when there is no more segments
* avaialble. The latter will cause the example to exit.
* - @ref NT_NetTxGet() - Get an empty tx buffer. This will get a segment of the
* requested length from the hostbuffer that will be sent
* onto port 0 when the buffer is released.
* - @ref NT_NET_GET_SEGMENT_PTR() and @ref NT_NET_GET_SEGMENT_LENGTH() are used
* to tell where the segment data is located and how much there is.
* @ref _nt_net_build_pkt_netbuf() and @ref _nt_net_get_next_packet() are used to traverse
* packets inside a segment. This is usefull if ie. the timing of the packet
* transmit rate needs to be changed.
* - @ref NT_NetTxRelease() - Release the segment buffer. Once a tx
* buffer is released it will be transmitted according to the
* timestamps of the packets in the segment.
* - @ref NT_NetFileRelease() - Release the segment from the file stream.
* - @ref NT_NetFileClose() - Close the file stream when no more segments can be found.
* - @ref NT_NetTxClose() - Close the TX stream.
*
*<hr>
* @section replayGS_example_code Code
* @}
*/
// Include this in order to access the Napatech API
#include <nt.h>
#if defined(__linux__) || defined(__FreeBSD__)
#define _GNU_SOURCE
#define __USE_XOPEN // fix of implicit declaration of function 'strptime'
#include <string.h> // memcpy
#include <stdarg.h>
#include <stdlib.h>
#include <signal.h>
#include <unistd.h>
#include <ctype.h>
#elif defined(WIN32) || defined (WIN64)
#include <winsock2.h>
#include "strptime.h"
#endif
#include <argparse.h>
#include <time.h>
#if defined(WIN32) || defined (WIN64)
#define snprintf(dst, ...) _snprintf_s((dst), _countof(dst), __VA_ARGS__)
#define strtoull _strtoui64
#define timegm _mkgmtime
#endif
static const char *usageText[] = {
"The replay example can replay a data file stored with the \"capture\" example.\n"
"The packets will be transmitted according to the TxPort field in the\n"
"packet descriptors. The TxPort field is relative to port0 on the adapter\n"
"provided the packets and therefore it is needed to select a port0 offset\n"
"via the option \"-p\".\n"
"The replay rate will be identical to the rate at which the packets were received.\n"
"\n"
"Syntax:\n"
"\n"
"replayGS -f <file> [-p <port offset>] [-g <timestamp>] [-s <starttime>] [-t] [--help] \n"
"\nCommands:\n",
NULL};
static char *opt_file = NULL;
static int opt_portOffset = -1;
static char *opt_start = NULL;
static int opt_time = 0;
static int appRunning=1;
static const char *progname; //!< Program name for the program itself
static uint32_t optionMask = 0;
static uint64_t gsyncAdapterTS = 0; //!< The timestamp of the adapter
static uint64_t gsyncFirstTS = 0; //!< The timestamp of the first packet in the file
static uint64_t gsyncSyncTS = 0; //!< The timestamp of the oldset timestamp of all the files that it sent synchronized
static uint64_t gsyncStartTime = 0; //!< The start time for the transmit to begin.
static uint64_t gsyncOffset = 0; //!< The calculated transmit delay
static uint64_t gsyncMaxTS = 0; //!< The timestamp of the last packet in the file. Used to calculate when the transmit is finish.
static int gsyncAdapter = 0; //!< The adapter number for the adapter used to transmit the file.
/*
* Defines used when parsing the commandline.
*/
#define OPTION_FILE (1<<1)
#define OPTION_PORTOFFSET (1<<2)
#define OPTION_GSYNC (1<<3)
#define OPTION_START (1<<4)
#define OPTION_TIMESTAMP (1<<5)
/**
* Table of valid options.
*/
struct argparse_option arg_options[] = {
OPT_STRING( 'f', "file", &opt_file, "Specifies the file to replay.", NULL, 0, 0, "file name"),
OPT_INTEGER('p', "port", &opt_portOffset, "Port offset to port0 on the adapter on which the data\n"
"should be replayed.", NULL, 0, 0, "offset"),
OPT_UINT64('g', NULL, &gsyncSyncTS, "Enable global synchronization mode.\n"
"Earliest 10-ns ticks time stamp to synchronize with", NULL, 0, 0, "timestamp"),
OPT_STRING('s', NULL, &opt_start, "Absolute transmission begin time stamp.\n"
"Format: YYYY/MM/DD-HH:MM:SS, HH:MM:SS or HH:MM", NULL, 0, 0, "starttime"),
OPT_BOOLEAN('t', NULL, &opt_time, "Find the first time stamp (in 10-ns ticks) in the capture file.", NULL, 0, 0, NULL),
};
/*
* The function called when user is pressing CTRL-C
*/
#if defined(WIN32) || defined (WIN64)
static BOOL WINAPI StopApplication(int sig)
#else
static void StopApplication(int sig __attribute__((unused)))
#endif
{
#ifdef WIN32
return TRUE;
#else
if (sig == SIGINT)
#endif
}
/*
* Display a header on the screen
*/
static void DisplayProgramHeader(void)
{
int i;
printf("\nNapatech Example - %s\n", progname);
/* Print separator line */
for (i = 0; i < 78; ++i) {
printf("=");
}
printf("\n");
}
/*
* Display a footer on the screen
*/
static void DisplayProgramFooter (void)
{
printf("\n");
}
/**
* @brief Print a human readable time on the screen.
*
* This function is called to print a human readable
* time on the screen. Mainly used to print the transmit
* start time.
*
* @param[in] ts The timestamp to print
*
* @retval NT_SUCCESS Function succeded
* @retval != NT_SUCCESS Function failed
*/
static void PrintTimeStamp(uint64_t ts)
{
time_t tmp_ts;
struct tm *loc_time;
char ts_buf[256];
tmp_ts = (time_t) (ts / 100000000);
loc_time = localtime(&tmp_ts);
snprintf(ts_buf, sizeof(ts_buf), "%04d/%02d/%02d-%02d:%02d:%02d.%09lu",
loc_time->tm_year+1900, loc_time->tm_mon+1, loc_time->tm_mday,
loc_time->tm_hour, loc_time->tm_min, loc_time->tm_sec,
(unsigned long) (ts % 100000000 * 10));
printf("%s", ts_buf);
}
/**
* @brief Calculate the transmit offset
*
* This function is called to calculate the transmit offset.
* CalcOffset calculates the global offset to set on an adapter in global
* sync mode. The clocks of the adapters used must be synchronized.
* All time stamps supplieds as parameters must be in 10-ns ticks.
* The offset is calculated such that transmission starts on ts_start.
*
* The adapter implements the following logic in GSYNC mode.
* Let Tp denote the time stamp of a packet, t the current (running)
* time, and offset the GSYNC offset in the adapter.
*
* Tp > t + offset : wait, do not send the packet yet
* Tp <= t + offset : send the packet
*
* Replay calculates gsyncOffset a little differently, namely as
*
* t - Tp = gsyncOffset
*
* and hence
*
* Tp = -gsyncOffset + t
*
* If we insert this last equation into the right-hand side of the
* expression used in the adapter (Tp = t + offset), we get
*
* Tp = t + offset = -gsyncOffset + t
*
* and therefore
*
* offset = -gsyncOffset
*
* ReplayGS therefore uses the negated value of gsyncOffset as the
* offset value on the adapter.
*
* @param[in] ts_start The start time of the transmit
* @param[in] ts1 Timestamp 1 from either the local file or from
* a file sent from another server
* @param[in] ts2 Timestamp 2 from either the local file or from
* a file sent from another server
*
* @retval NT_SUCCESS Function succeded
* @retval != NT_SUCCESS Function failed
*/
static uint64_t CalcOffset(uint64_t ts_start, uint64_t ts1, uint64_t ts2)
{
uint64_t offset = 0U; // offset value
uint64_t ts_min = 0U; // minimum time stamp of ts1 and ts2
ts_min = ts1 <= ts2 ? ts1 : ts2; // Chose the oldest of the timestamps
offset = ts_start - ts_min; // Calculate the offset
return offset;
}
/**
* @brief Gets the timestamp from the adapter
*
* This function is called to get the adapter timestamp
*
* @param[in] adapterNo Adapter number to get timestamp from
* @param[out] ts Pointer to the timestamp
*
* @retval NT_SUCCESS Function succeded
* @retval != NT_SUCCESS Function failed
*/
static NtError_t GetAdapterTimestamp(int adapterNo, uint64_t *ts)
{
NtError_t status;
char errorBuffer[NT_ERRBUF_SIZE]; // Error buffer
NtConfigStream_t hStream; // Config stream handle
NtConfig_t configRead; // Config stream data container
// Open the config stream
if ((status = NT_ConfigOpen(&hStream, "Replay")) != NT_SUCCESS) {
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_ConfigOpen() failed: %s\n", errorBuffer);
NT_ConfigClose(hStream);
return status;
}
// Read the adapter timestamp
configRead.parm = NT_CONFIG_PARM_ADAPTER_TIMESTAMP; // Read adapter timestamp command
configRead.u.timestampRead.adapter = (uint8_t) adapterNo; // Adapter number of the adapter
if ((status = NT_ConfigRead(hStream, &configRead)) != NT_SUCCESS) {
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_ConfigRead() failed: %s\n", errorBuffer);
NT_ConfigClose(hStream);
return status;
}
*ts = configRead.u.timestampRead.data.nativeUnixTs; // Store the native unix version of the timestamp.
// Close the config stream
if ((status = NT_ConfigClose(hStream)) != NT_SUCCESS) {
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_ConfigOpen() failed: %s\n", errorBuffer);
return status;
}
return NT_SUCCESS;
}
/**
* @brief Sets Global Sync
*
* This function is called to enable or disable global sync
*
* @param[in] adapterNo Adapter number to get timestamp from
* @param[in] portMask Bitmask for ports to enable
* @param[in] offset Global sync offset
*
* @retval NT_SUCCESS Function succeded
* @retval != NT_SUCCESS Function failed
*/
static NtError_t SetGlobalSync(int adapterNo, uint32_t portMask, uint64_t offset)
{
NtError_t status;
char errorBuffer[NT_ERRBUF_SIZE]; // Error buffer
NtConfigStream_t hStream; // Config stream handle
NtConfig_t configRead; // Config stream data container
// Open the config stream
if ((status = NT_ConfigOpen(&hStream, "Replay")) != NT_SUCCESS) {
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_ConfigOpen() failed: %s\n", errorBuffer);
NT_ConfigClose(hStream);
return status;
}
// Send the global sync offset to the adapteer
configRead.parm = NT_CONFIG_PARM_ADAPTER_GLOBAL_SYNC; // Global sync offset write command
configRead.u.globalSyncWrite.adapterNo = (uint8_t) adapterNo; // Adapter number
configRead.u.globalSyncWrite.data.portEnableMask = portMask; // Port enable mask for the
//ports where global sync should be enabled
configRead.u.globalSyncWrite.data.timeSynchronizedTxOffset = (~offset) + 1; // The calculated global sync offset.
// Note: The value must be written to
// the as two's complement.
configRead.u.globalSyncWrite.data.timeSynchronizedTxOffsetUpdate = portMask!=0?1:0; // Set to 1 when updating the offset.
if ((status = NT_ConfigWrite(hStream, &configRead)) != NT_SUCCESS) {
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_ConfigWrite() failed: %s\n", errorBuffer);
NT_ConfigClose(hStream);
return status;
}
// Close the config stream
if ((status = NT_ConfigClose(hStream)) != NT_SUCCESS) {
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_ConfigOpen() failed: %s\n", errorBuffer);
return status;
}
return NT_SUCCESS;
}
/**
* @brief Get the timestamp of the first packet in the file
*
* This function is called to get the timestamp of the first packet in the file
*
* @param[out] ts Pointer to variable holding the timestamp
*
* @retval 0 Function succeded
* @retval 1 Function failed
*/
static int GetTimeStamp(uint64_t *ts)
{
int status; // Status variable
char errorBuffer[NT_ERRBUF_SIZE]; // Error buffer
NtNetStreamFile_t hNetFile; // Handle to the File stream
NtNetBuf_t hNetBufFile; // Net buffer container. Used to return packets from the file stream
struct NtNetBuf_s pktNetBuf;
// Open the capture file to to read the timestamp)
if ((status = NT_NetFileOpen(&hNetFile, "replay", NT_NET_INTERFACE_SEGMENT, opt_file)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetFileOpen() failed: %s\n", errorBuffer);
return 1;
}
// Read the first segment of data from the file
if ((status = NT_NetFileGet(hNetFile, &hNetBufFile)) != NT_SUCCESS) {
if (status == NT_STATUS_END_OF_FILE) {
fprintf(stderr, "The file %s has no data\n", opt_file);
return 1;
}
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetFileGet() failed: %s\n", errorBuffer);
return 1;
}
// Get a packet buffer pointer from the segment.
_nt_net_build_pkt_netbuf(hNetBufFile, &pktNetBuf);
// Store the timestamp
*ts = NT_NET_GET_PKT_TIMESTAMP((&pktNetBuf));
// Close the file stream
NT_NetFileClose(hNetFile);
return 0;
}
/**
* @brief Parse the start time
*
* This function is called to parse the start time entered as input option
*
* @param[in] dts Pointer to string containing the start time
* @param[out] ts Pointer to variable to hold the converted start time
*
* @retval 0 Function succeded
* @retval -1 Function failed
*/
static int ParseStartTime(const char *dts, uint64_t *ts)
{
char *rem_d;
struct tm tm;
time_t t;
struct tm *loc_time;
time_t now;
/* Reset tm structure because strptime is not required to initialize
all members variables. */
memset(&tm, 0, sizeof(tm));
/* Did user specify date+time in ISO format? */
rem_d = strptime(dts, "%Y/%m/%d-%H:%M:%S", &tm);
if (!rem_d) {
/* Did user specify date and time in locale's representation? */
rem_d = strptime(dts, "%x %X", &tm);
if (!rem_d) {
memset(&tm, 0, sizeof(tm));
/* Did user specify (just) time in locale's representation? */
rem_d = strptime(dts, "%X", &tm);
if (!rem_d) {
memset(&tm, 0, sizeof(tm));
/* Did user specify only hour and minute? */
rem_d = strptime(dts, "%H:%M:%S", &tm);
if (!rem_d) {
memset(&tm, 0, sizeof(tm));
/* Did user specify only hour and minute? */
rem_d = strptime(dts, "%H:%M", &tm);
if (!rem_d)
/* Failed to parse date/time */
return -1;
}
}
/* User has only specified time, get yy/mm/dd from host, not adapter;
assume date on host and adapter are the same. */
now = time(NULL);
loc_time = localtime(&now);
tm.tm_year = loc_time->tm_year;
tm.tm_mon = loc_time->tm_mon;
tm.tm_mday = loc_time->tm_mday;
}
}
t = timegm(&tm);
if (t == (time_t) -1)
return -1;
/* convert to 10-nsec ticks */
*ts = t * 100000000;
return 0;
}
/**
* @brief Print the transmission start time on the screen.
*
* This function is called to print the transmission start time on the screen.
*
* @param[in] adapter Adapter number
* @param[in] firstTS The time stamp of the first packet in the file
* @param[in] offset The calculated transmit offset
*
* @retval NT_SUCCESS Function succeded
* @retval != NT_SUCCESS Function failed
*/
static int printTransmissionStart(int adapter, uint64_t firstTS, uint64_t offset)
{
uint64_t ts;
int64_t diff;
int status;
if ((status = GetAdapterTimestamp(adapter, &ts)) != NT_SUCCESS) {
fprintf(stderr, "Reading adapter timestamp failed\n");
return status;
}
diff = (((int64_t)((firstTS + offset) - ts) / 100000000) + 1);
if (diff >= 0) {
printf("Transmission begins on ");
PrintTimeStamp(firstTS + offset);
printf(" (%lld sec) \r", (unsigned long long int)diff);
fflush(stdout);
}
return NT_SUCCESS;
}
int main(int argc, const char *argv[])
{
int i;
const char *p;
uint64_t numPackets=0; // The number of packets replayed
uint64_t numBytes=0; // The number of bytes replayed
int firstPacket=1; // First packet to transmit
char errorBuffer[NT_ERRBUF_SIZE]; // Error buffer
int status; // Status variable
NtNetStreamFile_t hNetFile; // Handle to the File stream
NtNetBuf_t hNetBufFile; // Net buffer container. Used to return segments from the file stream
NtNetStreamTx_t hNetTx; // Handle to the TX stream
NtNetBuf_t hNetBufTx; // Net buffer container. Used when getting a transmit buffer
NtInfoStream_t hInfo; // Handle to a info stream
NtInfo_t infoRead; // Buffer to hold data from infostream
struct NtNetBuf_s pktNetBuf; // Packet netbuf structure.
struct argparse argparse;
// Strip path
i=(int)strlen(&argv[0][0]);
p=argv[0];
for ( ; i != 0; i--) {
#ifdef WIN32
if ((p[i-1]=='\\') || (p[i-1]=='/')) {
#else
if ((p[i-1]=='.') || (p[i-1]=='\\') || (p[i-1]=='/')) {
#endif
break;
}
}
progname = &argv[0][i]; // Store the daemon name so it can be used in Usage
// ***************************************************************
// Parse the commandline to find the necessary global sync options
// ***************************************************************
argparse_init(&argparse, arg_options, usageText, 0);
argparse_parse(&argparse, argc, argv);
if (opt_file != NULL) {
}
if (opt_portOffset != -1) {
}
if (gsyncSyncTS != 0) {
}
if (opt_start != NULL) {
fprintf(stderr, ">>> Error: \"-s %s\" time format is invalid\n", opt_start);
return 1;
}
}
if (opt_time != 0) {
}
// ***********************************************
// Validate input parameters
// ***********************************************
// We must always have a file.
if (!(optionMask & OPTION_FILE)) {
fprintf(stderr, "No file selected.\n");
return 1;
}
// If -t only -f is allowed
if (optionMask & ~(OPTION_FILE | OPTION_TIMESTAMP)) {
fprintf(stderr, "Only option -f is allowed with option -t.\n");
return 1;
}
}
else {
// Must always have a port.
fprintf(stderr, "Missing portoffset -p.\n");
return 1;
}
// Global sync start time must be defined
fprintf(stderr, "Missing start time -s.\n");
return 1;
}
}
// ***********************************************
// Initialize the NTAPI library and thereby check
// if NTAPI_VERSION can be used together with this
// library
// ***********************************************
if ((status = NT_Init(NTAPI_VERSION)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_Init() failed: %s\n", errorBuffer);
return -1;
}
// *************************************
// Get the timestamp of the first packet
// *************************************
if ((status = GetTimeStamp(&gsyncFirstTS)) != 0) {
return status;
}
if (optionMask & OPTION_TIMESTAMP) {
// We just want to get the first timestamp in the file.
// Print the timestamp and return
printf("Timestamp = %llu\n", (long long unsigned int)gsyncFirstTS);
return 0;
}
// Set up ctrl+c handler
#if defined(WIN32) || defined (WIN64)
SetConsoleCtrlHandler((PHANDLER_ROUTINE)StopApplication, TRUE);
#else
struct sigaction newaction; // Ctrl+c handle
memset(&newaction, 0, sizeof(newaction));
newaction.sa_handler = StopApplication;
if (sigaction(SIGINT, &newaction, NULL) < 0) {
fprintf(stderr, "Failed to register SIGINT sigaction.\n");
exit(1);
}
#endif
// **********************************
// Read the port and adapter settings
// **********************************
// Open the infostream.
if ((status = NT_InfoOpen(&hInfo, "replay")) != NT_SUCCESS) {
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_InfoOpen() failed: %s\n", errorBuffer);
return -1;
}
// Read the port and adapter settings
infoRead.u.port_v9.portNo = (uint8_t) opt_portOffset;
if ((status = NT_InfoRead(hInfo, &infoRead)) != NT_SUCCESS) {
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_InfoOpen() failed: %s\n", errorBuffer);
NT_InfoClose(hInfo);
return -1;
}
NT_InfoClose(hInfo);
// The selected port offset must be equal to the first port on the adapter.
if (infoRead.u.port_v9.data.adapterInfo.portOffset != (uint32_t)opt_portOffset) {
fprintf(stderr, "\"-p %d\" is not a offset to a port0 on a physical adapter.\n", opt_portOffset);
return -1;
}
// Get the adapter number
// ***********************************************
// Check that the setup of the adapter is correct.
// ***********************************************
// Is sync supported?
fprintf(stderr, "Global sync is not supported by this adapter no. %d.\n", gsyncAdapter);
return -1;
}
// Only absolut TX timing mode can be used
fprintf(stderr, "Global sync cannot be used when running relative TX timing.\n");
return -1;
}
// Only native unix timestamp can be used
fprintf(stderr, "Global sync can only be used when using \"native unix\" timestamps.\n");
return -1;
}
// Adapter must be running the capture/replay profile for this to work.
fprintf(stderr, "Please use the \"CaptureReplay\" profile.\n");
return -1;
}
// **********************************
// Setup Global sync for the adapter.
// **********************************
// Get the adapter time stamp
fprintf(stderr, "Reading adapter timestamp failed\n");
return -1;
}
// Validate start time is not in the past
fprintf(stderr, "Error: Start time is in the past.\n");
return -1;
}
// Sync timestamp -g is not defined. Use first timestamp in the
// packet as sync timestamp. This means the transmission will start
// with respect to the local file and not with respect to the other
// files in this synchronized transmit.
}
// Calculate the global sync offset
// Write the offset to the adapter
if (SetGlobalSync(gsyncAdapter, 0xFF, gsyncOffset) != NT_SUCCESS) {
fprintf(stderr, "Setting Global Sync failed\n");
return 1;
}
// Print the transmission start time on the screen.
fprintf(stderr, "Printing start time failed\n");
return 1;
}
// ****************************
// Start transmitting the file.
// ****************************
// Open the capture file to replay (captured with the capture example)
if ((status = NT_NetFileOpen(&hNetFile, "FileStream", NT_NET_INTERFACE_SEGMENT, opt_file)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetFileOpen() failed: %s\n", errorBuffer);
return -1;
}
// Open a TX hostbuffer on NUMA node 0 that can transmit to port 0
if ((status = NT_NetTxOpen(&hNetTx, "TxStreamPort0", (uint64_t)1<<opt_portOffset, NT_NETTX_NUMA_ADAPTER_HB, 0)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetTxOpen() failed: %s\n", errorBuffer);
return -1;
}
// Get segments from the file and transmit them to port 0
while (appRunning) {
// Get the packet
if ((status = NT_NetFileGet(hNetFile, &hNetBufFile)) != NT_SUCCESS) {
if (status == NT_STATUS_END_OF_FILE) {
// The file has no more data
break;
}
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetFileGet() failed: %s\n", errorBuffer);
return -1;
}
// Store the last timestamp in the read segment. We need this
// timestamp in order to know when the transmit is finished.
if ((status = NT_NetFileRead(hNetFile, &data)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NtNetFileRead_t() failed: %s\n", errorBuffer);
return -1;
}
// Get a TX buffer for this segment
while ((status = NT_NetTxGet(hNetTx, &hNetBufTx, opt_portOffset, NT_NET_GET_SEGMENT_LENGTH(hNetBufFile), NT_NETTX_SEGMENT_OPTION_RAW, 1000)) != NT_SUCCESS) {
if (status != NT_STATUS_TIMEOUT) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetFileGet() failed: %s\n", errorBuffer);
return -1;
}
// Print the start time on the screen and count down to start.
fprintf(stderr, "Printing start time failed\n");
return 1;
}
if (appRunning == 0) {
goto CTRLEXIT;
}
}
// Copy the segment into the TX buffer
memcpy(NT_NET_GET_SEGMENT_PTR(hNetBufTx), NT_NET_GET_SEGMENT_PTR(hNetBufFile), NT_NET_GET_SEGMENT_LENGTH(hNetBufFile));
// Build a packet netbuf structure
_nt_net_build_pkt_netbuf(hNetBufTx, &pktNetBuf);
if (firstPacket == 1) {
// The first packet must have txnow=0 and txsetclock=0
NT_NET_SET_PKT_TXNOW((&pktNetBuf), 0); // Gsync mode: txnow = 0
NT_NET_SET_PKT_TXSETCLOCK((&pktNetBuf), 0); // Gsync mode: txsetclock = 0
firstPacket = 0;
}
// Optional step. Here all packets are inspected before sent
if (NT_NET_GET_SEGMENT_LENGTH(hNetBufTx)) {
do {
// Just count the amount of packets and wire length
numPackets++;
numBytes+=NT_NET_GET_PKT_WIRE_LENGTH((&pktNetBuf));
} while (_nt_net_get_next_packet(hNetBufTx, NT_NET_GET_SEGMENT_LENGTH(hNetBufTx), &pktNetBuf)>0);
}
// Release the TX buffer and the packets within the segment will be transmitted
if ((status = NT_NetTxRelease(hNetTx, hNetBufTx)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetTxRelease() failed: %s\n", errorBuffer);
return -1;
}
// Release the file packet
if ((status = NT_NetFileRelease(hNetFile, hNetBufFile)) != NT_SUCCESS) {
// Get the status code as text
NT_ExplainError(status, errorBuffer, sizeof(errorBuffer));
fprintf(stderr, "NT_NetFileRelease() failed: %s\n", errorBuffer);
return -1;
}
// Print the number of packets transmitted to the screen
if (!(numPackets % 1000)) {
printf("Packets transmitted so far: %llu \r",
(unsigned long long) numPackets);
fflush(stdout);
}
}
CTRLEXIT:
// Close the file stream
NT_NetFileClose(hNetFile);
// ************************************
// Wait for the file to be transmitted.
// ************************************
// We need to wait for the file to be transmitted before we can disable
// gsync again. Disabling the gsync before the file has been trasnitted,
// we cause the file to be transmitted immediately.
// To wait we are comparing the adapter timestamp with the last packet timstamp
// used in the file. When the adapter timestamp is greater than the last timestamp
// + the gsync offset. Then the file has been transmitted.
// Set app running so the user has a chance to cancel the wait.
while (appRunning) {
uint64_t ts;
int64_t diff;
if (GetAdapterTimestamp(gsyncAdapter, &ts) != NT_SUCCESS) {
fprintf(stderr, "Reading adapter timestamp failed\n");
return 1;
}
if (ts > (gsyncMaxTS + gsyncOffset)) {
break;
}
diff = (((int64_t)((gsyncMaxTS + gsyncOffset) - ts) / 100000000) + 1);
if (diff < 0) {
// The file is transmitted.
break;
}
printf("Awaiting completion in %lld seconds \r", (long long int)diff); fflush(stdout);
// Sleep 1 sec
#if defined(__linux__) || defined(__FreeBSD__)
sleep(1);
#elif defined(WIN32) || defined (WIN64)
Sleep(1000); // sleep 1000 milliseconds = 1 second
#endif
}
// Close the TX stream. If cancel on close is enabled. The remaning data will be
// canceled when the stream is closed. If cancel on close is not enabled the
// remaining packets will be transmitted according to the timestamp, when
// global sync is disabled. According to the timestamp means that the adapter
// will look at the timestamp before transmitting the packets. This will
// either cause the packets to be transmitted immediately or cause the to
// hang until the timestamp matches the adapter timestamp. In onther way
// it is unpredictable to say what will happen.
NT_NetTxClose(hNetTx);
// We need to disable Global Sync before we stop.
if (SetGlobalSync(gsyncAdapter, 0, 0) != NT_SUCCESS) {
fprintf(stderr, "Disabling Global Sync failed\n");
return 1;
}
printf("Done: %llu packets %llu bytes has been replayed \n", (unsigned long long int)numPackets, (unsigned long long int)numBytes);
return 0;
}