net/replayGS/replayGS_example.c

Reference Documentation

product_line_custom
Napatech SmartNIC
category
Reference Information

Description

This source file is an example of how to replay a capture file onto a port using NTAPI and the global synchronous transmit feature.

With this example is it possible make a synchronous transmit between several accelerators placed in either same server or different serves. In order for the synchronous transmit to work all adapters must be synchronized using timesync.

Prerequisites

  • A capture file, that has been captured with the net/capture/capture_example.c example.

  • All adapters must support the global sync feature.

  • TxTiming must be set to ABSOLUTE in ntservice.ini

  • Timestamp format must be NATIVE_UNIX

  • ntservice.ini must have at least one HostBuffersTx defined. Below is an example of a minimum ini-file. It will create a 32MB TX hostbuffer from NUMA node 0. [System] TimestampFormat = NATIVE_UNIX [Adapter0] AdapterType = NT20E2 BusId = 00:0a:00.00 HostBuffersTx = [1,32,0] TxTiming = ABSOLUTE

Program flow

The following is required to perform synchronized replay of a captured file:

Code

/* * %NT_SOFTWARE_LICENSE% */ /** * @example net/replayGS/replayGS_example.c * @section replayGS_example_description Description * * This source file is an example of how to replay a capture file onto * a port using NTAPI and the global synchronous transmit feature. * * With this example is it possible make a synchronous transmit between * several accelerators placed in either same server or different serves. In order * for the synchronous transmit to work all adapters must be synchronized using * timesync. * * The following NTAPI functions are used: * - @ref NT_Init() * - @ref NT_InfoOpen() * - @ref NT_InfoRead() * - @ref NT_InfoClose() * - @ref NT_ConfigOpen() * - @ref NT_ConfigRead() * - @ref NT_ConfigWrite() * - @ref NT_ConfigClose() * - @ref NT_NetFileOpen() * - @ref NT_NetTxOpen() * - @ref NT_NTPL() * - @ref NT_NetFileGet() * - @ref NT_NetTxGet() * - @ref NT_NET_GET_SEGMENT_PTR() * - @ref NT_NET_GET_SEGMENT_LENGTH() * - @ref NT_NetTxRelease() * - @ref NT_NetFileRelease() * - @ref NT_NetFileClose() * - @ref NT_NetTxClose() * - @ref NT_ExplainError() * * @section replayGS_example_prerequisites Prerequisites * - A capture file, that has been captured with the @ref * net/capture/capture_example.c "net/capture/capture_example.c" example. * - All adapters must support the global sync feature. * - TxTiming must be set to ABSOLUTE in ntservice.ini * - Timestamp format must be NATIVE_UNIX * - ntservice.ini must have at least one HostBuffersTx * defined. Below is an example of a minimum ini-file. It will * create a 32MB TX hostbuffer from NUMA node 0. * @code * [System] * TimestampFormat = NATIVE_UNIX * * [Adapter0] * AdapterType = NT20E2 * BusId = 00:0a:00.00 * HostBuffersTx = [1,32,0] * TxTiming = ABSOLUTE * @endcode * * @section replayGS_example_flow Program flow * @{ * The following is required to perform synchronized replay of a captured file: * - \#include/nt.h - Applications/Tools only need to include @ref * nt.h to obtain prototypes, macros etc. from NTAPI. * - @ref NT_Init(@ref NTAPI_VERSION) - Initialize the NTAPI * library. @ref NTAPI_VERSION is a define that describes the version * of the API described in the header files included by @ref * nt.h. NT_Init() will ask the NTAPI library to convert return data * to the @ref NTAPI_VERSION if possible. This will ensure that * applications can run on NTAPI libraries of newer versions. * - @ref NT_InfoOpen() - Open the information stream. * - @ref NT_InfoRead() - Read @ref NtInfoAdapter_v6_s "the adapter information" to check whether or not * we are using absolut TX timing. The @ref NtTxTimingMethod_e "txTiming" * tells whether relative or absolut TX timing is enabled. * - @ref NT_InfoClose() - Close the information stream. * - @ref NT_ConfigOpen() - Open the config stream. * - @ref NT_ConfigRead() - Read the adapter timestamp from the adapter. Needed * in order to synchronize the transmit. * - @ref NT_ConfigWrite() - Write the synchronize offset to the adapter. * - @ref NT_ConfigClose() - Close the config stream. * - @ref NT_NetFileOpen() - Open the captured file and assign it to a stream. * - @ref NT_NetTxOpen() - Open a hostbuffer than can transmit packets to port 0. * - @ref NT_NetFileGet() - Get a segment from the file. This call will * return NT_SUCCESS upon return of a segment and * @ref NT_STATUS_END_OF_FILE when there is no more segments * avaialble. The latter will cause the example to exit. * - @ref NT_NetTxGet() - Get an empty tx buffer. This will get a segment of the * requested length from the hostbuffer that will be sent * onto port 0 when the buffer is released. * - @ref NT_NET_GET_SEGMENT_PTR() and @ref NT_NET_GET_SEGMENT_LENGTH() are used * to tell where the segment data is located and how much there is. * @ref _nt_net_build_pkt_netbuf() and @ref _nt_net_get_next_packet() are used to traverse * packets inside a segment. This is usefull if ie. the timing of the packet * transmit rate needs to be changed. * - @ref NT_NetTxRelease() - Release the segment buffer. Once a tx * buffer is released it will be transmitted according to the * timestamps of the packets in the segment. * - @ref NT_NetFileRelease() - Release the segment from the file stream. * - @ref NT_NetFileClose() - Close the file stream when no more segments can be found. * - @ref NT_NetTxClose() - Close the TX stream. * *<hr> * @section replayGS_example_code Code * @} */ #if defined(__linux__) || defined(__FreeBSD__) #define _GNU_SOURCE #include <string.h> // memcpy #include <stdarg.h> #include <stdlib.h> #include <signal.h> #include <unistd.h> #include <ctype.h> #elif defined(WIN32) || defined (WIN64) #include <winsock2.h> #include "strptime.h" #define timegm _mkgmtime #endif #include <argparse.h> #include <time.h> #include <nt.h> #if defined(WIN32) || defined (WIN64) #define strtoull _strtoui64 #endif static const char *usageText[] = { "The replay example can replay a data file stored with the \"capture\" example.\n" "The packets will be transmitted according to the TxPort field in the\n" "packet descriptors. The TxPort field is relative to port0 on the adapter\n" "provided the packets and therefore it is needed to select a port0 offset\n" "via the option \"-p\".\n" "The replay rate will be identical to the rate at which the packets were received.\n" "\n" "Syntax:\n" "\n" "replayGS -f <file> [-p <port offset>] [-g <timestamp>] [-s <starttime>] [-t] [--help] \n" "\nCommands:\n", NULL}; static char *opt_file = NULL; static int opt_portOffset = -1; static char *opt_start = NULL; static int opt_time = 0; static int appRunning=1; static const char *progname; //!< Program name for the program itself static uint32_t optionMask = 0; static uint64_t gsyncAdapterTS = 0; //!< The timestamp of the adapter static uint64_t gsyncFirstTS = 0; //!< The timestamp of the first packet in the file static uint64_t gsyncSyncTS = 0; //!< The timestamp of the oldset timestamp of all the files that it sent synchronized static uint64_t gsyncStartTime = 0; //!< The start time for the transmit to begin. static uint64_t gsyncOffset = 0; //!< The calculated transmit delay static uint64_t gsyncMaxTS = 0; //!< The timestamp of the last packet in the file. Used to calculate when the transmit is finish. static int gsyncAdapter = 0; //!< The adapter number for the adapter used to transmit the file. /* * Defines used when parsing the commandline. */ #define OPTION_FILE (1<<1) #define OPTION_PORTOFFSET (1<<2) #define OPTION_GSYNC (1<<3) #define OPTION_START (1<<4) #define OPTION_TIMESTAMP (1<<5) /** * Table of valid options. */ struct argparse_option arg_options[] = { OPT_HELP(), OPT_STRING( 'f', "file", &opt_file, "Specifies the file to replay.", NULL, 0, 0, "file name"), OPT_INTEGER('p', "port", &opt_portOffset, "Port offset to port0 on the adapter on which the data\n" "should be replayed.", NULL, 0, 0, "offset"), OPT_UINT64('g', NULL, &gsyncSyncTS, "Enable global synchronization mode.\n" "Earliest 10-ns ticks time stamp to synchronize with", NULL, 0, 0, "timestamp"), OPT_STRING('s', NULL, &opt_start, "Absolute transmission begin time stamp.\n" "Format: YYYY/MM/DD-HH:MM:SS, HH:MM:SS or HH:MM", NULL, 0, 0, "starttime"), OPT_BOOLEAN('t', NULL, &opt_time, "Find the first time stamp (in 10-ns ticks) in the capture file.", NULL, 0, 0, NULL), OPT_END(), }; /* * The function called when user is pressing CTRL-C */ #if defined(WIN32) || defined (WIN64) static BOOL WINAPI StopApplication(int sig) #else static void StopApplication(int sig __attribute__((unused))) #endif { #ifdef WIN32 appRunning=0; return TRUE; #else if (sig == SIGINT) appRunning=0; #endif } /* * Display a header on the screen */ static void DisplayProgramHeader(void) { int i; printf("\nNapatech Example - %s\n", progname); /* Print separator line */ for(i=0; i<78; ++i) { printf("="); } printf("\n"); } /* * Display a footer on the screen */ static void DisplayProgramFooter (void) { printf("\n"); } /** * @brief Print a human readable time on the screen. * * This function is called to print a human readable * time on the screen. Mainly used to print the transmit * start time. * * @param[in] ts The timestamp to print * * @retval NT_SUCCESS Function succeded * @retval != NT_SUCCESS Function failed */ static void PrintTimeStamp(uint64_t ts) { time_t tmp_ts; struct tm *loc_time; char ts_buf[256]; tmp_ts = (time_t) (ts / 100000000); loc_time = localtime(&tmp_ts); sprintf(ts_buf, "%04d/%02d/%02d-%02d:%02d:%02d.%09lu", loc_time->tm_year+1900, loc_time->tm_mon+1, loc_time->tm_mday, loc_time->tm_hour, loc_time->tm_min, loc_time->tm_sec, (unsigned long) (ts % 100000000 * 10)); printf("%s", ts_buf); } /** * @brief Calculate the transmit offset * * This function is called to calculate the transmit offset. * CalcOffset calculates the global offset to set on an adapter in global * sync mode. The clocks of the adapters used must be synchronized. * All time stamps supplieds as parameters must be in 10-ns ticks. * The offset is calculated such that transmission starts on ts_start. * * The adapter implements the following logic in GSYNC mode. * Let Tp denote the time stamp of a packet, t the current (running) * time, and offset the GSYNC offset in the adapter. * * Tp > t + offset : wait, do not send the packet yet * Tp <= t + offset : send the packet * * Replay calculates gsyncOffset a little differently, namely as * * t - Tp = gsyncOffset * * and hence * * Tp = -gsyncOffset + t * * If we insert this last equation into the right-hand side of the * expression used in the adapter (Tp = t + offset), we get * * Tp = t + offset = -gsyncOffset + t * * and therefore * * offset = -gsyncOffset * * ReplayGS therefore uses the negated value of gsyncOffset as the * offset value on the adapter. * * @param[in] ts_start The start time of the transmit * @param[in] ts1 Timestamp 1 from either the local file or from * a file sent from another server * @param[in] ts2 Timestamp 2 from either the local file or from * a file sent from another server * * @retval NT_SUCCESS Function succeded * @retval != NT_SUCCESS Function failed */ static uint64_t CalcOffset(uint64_t ts_start, uint64_t ts1, uint64_t ts2) { uint64_t offset = 0U; // offset value uint64_t ts_min = 0U; // minimum time stamp of ts1 and ts2 ts_min = ts1 <= ts2 ? ts1 : ts2; // Chose the oldest of the timestamps offset = ts_start - ts_min; // Calculate the offset return offset; } /** * @brief Gets the timestamp from the adapter * * This function is called to get the adapter timestamp * * @param[in] adapterNo Adapter number to get timestamp from * @param[out] ts Pointer to the timestamp * * @retval NT_SUCCESS Function succeded * @retval != NT_SUCCESS Function failed */ static uint32_t GetAdapterTimestamp(int adapterNo, uint64_t *ts) { uint32_t status; char errorBuffer[NT_ERRBUF_SIZE]; // Error buffer NtConfigStream_t hStream; // Config stream handle NtConfig_t configRead; // Config stream data container // Open the config stream if((status = NT_ConfigOpen(&hStream, "Replay")) != NT_SUCCESS) { NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_ConfigOpen() failed: %s\n", errorBuffer); NT_ConfigClose(hStream); return status; } // Read the adapter timestamp configRead.parm = NT_CONFIG_PARM_ADAPTER_TIMESTAMP; // Read adapter timestamp command configRead.u.timestampRead.adapter = (uint8_t) adapterNo; // Adapter number of the adapter if((status = NT_ConfigRead(hStream, &configRead)) != NT_SUCCESS) { NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_ConfigRead() failed: %s\n", errorBuffer); NT_ConfigClose(hStream); return status; } *ts = configRead.u.timestampRead.data.nativeUnixTs; // Store the native unix version of the timestamp. // Close the config stream if((status = NT_ConfigClose(hStream)) != NT_SUCCESS) { NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_ConfigOpen() failed: %s\n", errorBuffer); return status; } return NT_SUCCESS; } /** * @brief Sets Global Sync * * This function is called to enable or disable global sync * * @param[in] adapterNo Adapter number to get timestamp from * @param[in] portMask Bitmask for ports to enable * @param[in] offset Global sync offset * * @retval NT_SUCCESS Function succeded * @retval != NT_SUCCESS Function failed */ static uint32_t SetGlobalSync(int adapterNo, int portMask, uint64_t offset) { uint32_t status; char errorBuffer[NT_ERRBUF_SIZE]; // Error buffer NtConfigStream_t hStream; // Config stream handle NtConfig_t configRead; // Config stream data container // Open the config stream if((status = NT_ConfigOpen(&hStream, "Replay")) != NT_SUCCESS) { NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_ConfigOpen() failed: %s\n", errorBuffer); NT_ConfigClose(hStream); return status; } // Send the global sync offset to the adapteer configRead.parm = NT_CONFIG_PARM_ADAPTER_GLOBAL_SYNC; // Global sync offset write command configRead.u.globalSyncWrite.adapterNo = (uint8_t) adapterNo; // Adapter number configRead.u.globalSyncWrite.data.portEnableMask = portMask; // Port enable mask for the //ports where global sync should be enabled configRead.u.globalSyncWrite.data.timeSynchronizedTxOffset = (~offset) + 1; // The calculated global sync offset. // Note: The value must be written to // the as two's complement. configRead.u.globalSyncWrite.data.timeSynchronizedTxOffsetUpdate = portMask!=0?1:0; // Set to 1 when updating the offset. if((status = NT_ConfigWrite(hStream, &configRead)) != NT_SUCCESS) { NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_ConfigWrite() failed: %s\n", errorBuffer); NT_ConfigClose(hStream); return status; } // Close the config stream if((status = NT_ConfigClose(hStream)) != NT_SUCCESS) { NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_ConfigOpen() failed: %s\n", errorBuffer); return status; } return NT_SUCCESS; } /** * @brief Get the timestamp of the first packet in the file * * This function is called to get the timestamp of the first packet in the file * * @param[out] ts Pointer to variable holding the timestamp * * @retval 0 Function succeded * @retval 1 Function failed */ static int GetTimeStamp(uint64_t *ts) { int status; // Status variable char errorBuffer[NT_ERRBUF_SIZE]; // Error buffer NtNetStreamFile_t hNetFile; // Handle to the File stream NtNetBuf_t hNetBufFile; // Net buffer container. Used to return packets from the file stream struct NtNetBuf_s pktNetBuf; // Open the capture file to to read the timestamp) if((status = NT_NetFileOpen(&hNetFile, "replay", NT_NET_INTERFACE_SEGMENT, opt_file)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NetFileOpen() failed: %s\n", errorBuffer); return 1; } // Read the first segment of data from the file if((status = NT_NetFileGet(hNetFile, &hNetBufFile)) != NT_SUCCESS) { if(status == NT_STATUS_END_OF_FILE) { fprintf(stderr, "The file %s has no data\n", opt_file); return 1; } // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NetFileGet() failed: %s\n", errorBuffer); return 1; } // Get a packet buffer pointer from the segment. _nt_net_build_pkt_netbuf(hNetBufFile, &pktNetBuf); // Store the timestamp *ts = NT_NET_GET_PKT_TIMESTAMP((&pktNetBuf)); // Close the file stream NT_NetFileClose(hNetFile); return 0; } /** * @brief Parse the start time * * This function is called to parse the start time entered as input option * * @param[in] dts Pointer to string containing the start time * @param[out] ts Pointer to variable to hold the converted start time * * @retval 0 Function succeded * @retval -1 Function failed */ static int ParseStartTime(const char *dts, uint64_t *ts) { char *rem_d; struct tm tm; time_t t; struct tm *loc_time; time_t now; /* Reset tm structure because strptime is not required to initialize all members variables. */ memset(&tm, 0, sizeof(tm)); /* Did user specify date+time in ISO format? */ rem_d = strptime(dts, "%Y/%m/%d-%H:%M:%S", &tm); if (!rem_d) { /* Did user specify date and time in locale's representation? */ rem_d = strptime(dts, "%x %X", &tm); if (!rem_d) { memset(&tm, 0, sizeof(tm)); /* Did user specify (just) time in locale's representation? */ rem_d = strptime(dts, "%X", &tm); if (!rem_d) { memset(&tm, 0, sizeof(tm)); /* Did user specify only hour and minute? */ rem_d = strptime(dts, "%H:%M:%S", &tm); if (!rem_d) { memset(&tm, 0, sizeof(tm)); /* Did user specify only hour and minute? */ rem_d = strptime(dts, "%H:%M", &tm); if (!rem_d) /* Failed to parse date/time */ return -1; } } /* User has only specified time, get yy/mm/dd from host, not adapter; assume date on host and adapter are the same. */ now = time(NULL); loc_time = localtime(&now); tm.tm_year = loc_time->tm_year; tm.tm_mon = loc_time->tm_mon; tm.tm_mday = loc_time->tm_mday; } } t = timegm(&tm); if (t == (time_t) -1) return -1; /* convert to 10-nsec ticks */ *ts = t * 100000000;; return 0; } /** * @brief Print the transmission start time on the screen. * * This function is called to print the transmission start time on the screen. * * @param[in] adapter Adapter number * @param[in] firstTS The time stamp of the first packet in the file * @param[in] offset The calculated transmit offset * * @retval NT_SUCCESS Function succeded * @retval != NT_SUCCESS Function failed */ static int printTransmissionStart(int adapter, uint64_t firstTS, uint64_t offset) { uint64_t ts; int64_t diff; int status; if ((status = GetAdapterTimestamp(adapter, &ts)) != NT_SUCCESS) { fprintf(stderr, "Reading adapter timestamp failed\n"); return status; } diff = (((int64_t)((firstTS + offset) - ts) / 100000000) + 1); if(diff >= 0) { printf("Transmission begins on "); PrintTimeStamp(firstTS + offset); printf(" (%lld sec) \r", (unsigned long long int)diff); fflush(stdout); } return NT_SUCCESS; } int main(int argc, const char *argv[]) { int i; const char *p; uint64_t numPackets=0; // The number of packets replayed uint64_t numBytes=0; // The number of bytes replayed int firstPacket=1; // First packet to transmit char errorBuffer[NT_ERRBUF_SIZE]; // Error buffer int status; // Status variable NtNetStreamFile_t hNetFile; // Handle to the File stream NtNetBuf_t hNetBufFile; // Net buffer container. Used to return segments from the file stream NtNetStreamTx_t hNetTx; // Handle to the TX stream NtNetBuf_t hNetBufTx; // Net buffer container. Used when getting a transmit buffer NtInfoStream_t hInfo; // Handle to a info stream NtInfo_t infoRead; // Buffer to hold data from infostream struct NtNetBuf_s pktNetBuf; // Packet netbuf structure. struct argparse argparse; // Strip path i=(int)strlen(&argv[0][0]); p=argv[0]; for(;i!=0;i--) { #ifdef WIN32 if((p[i-1]=='\\') || (p[i-1]=='/')) { #else if((p[i-1]=='.') || (p[i-1]=='\\') || (p[i-1]=='/')) { #endif break; } } progname = &argv[0][i]; // Store the daemon name so it can be used in Usage DisplayProgramHeader(); // *************************************************************** // Parse the commandline to find the necessary global sync options // *************************************************************** argparse_init(&argparse, arg_options, usageText, 0); argparse_parse(&argparse, argc, argv); if (opt_file != NULL) { optionMask |= OPTION_FILE; } if (opt_portOffset != -1) { optionMask |= OPTION_PORTOFFSET; } if (gsyncSyncTS != 0) { optionMask |= OPTION_GSYNC; } if (opt_start != NULL) { if (ParseStartTime(opt_start, &gsyncStartTime) != 0) { fprintf(stderr, ">>> Error: \"-s %s\" time format is invalid\n", opt_start); return 1; } optionMask |= OPTION_START; } if (opt_time != 0) { optionMask |= OPTION_TIMESTAMP; } // *********************************************** // Validate input parameters // *********************************************** // We must always have a file. if(!(optionMask & OPTION_FILE)) { fprintf(stderr, "No file selected.\n"); return 1; } // If -t only -f is allowed if(optionMask & OPTION_TIMESTAMP) { if (optionMask & ~(OPTION_FILE | OPTION_TIMESTAMP)) { fprintf(stderr, "Only option -f is allowed with option -t.\n"); return 1; } } else { // Must always have a port. if(!(optionMask & OPTION_PORTOFFSET)) { fprintf(stderr, "Missing portoffset -p.\n"); return 1; } // Global sync start time must be defined if(!(optionMask & OPTION_START)) { fprintf(stderr, "Missing start time -s.\n"); return 1; } } // *********************************************** // Initialize the NTAPI library and thereby check // if NTAPI_VERSION can be used together with this // library // *********************************************** if((status = NT_Init(NTAPI_VERSION)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_Init() failed: %s\n", errorBuffer); return -1; } // ************************************* // Get the timestamp of the first packet // ************************************* if((status = GetTimeStamp(&gsyncFirstTS)) != 0) { return status; } if(optionMask & OPTION_TIMESTAMP) { // We just want to get the first timestamp in the file. // Print the timestamp and return printf("Timestamp = %llu\n", (long long unsigned int)gsyncFirstTS); return 0; } // Set up ctrl+c handler #if defined(WIN32) || defined (WIN64) SetConsoleCtrlHandler((PHANDLER_ROUTINE)StopApplication, TRUE); #else struct sigaction newaction; // Ctrl+c handle memset(&newaction, 0, sizeof(newaction)); newaction.sa_handler = StopApplication; if (sigaction(SIGINT, &newaction, NULL) < 0) { fprintf(stderr, "Failed to register SIGINT sigaction.\n"); exit(1); } #endif // ********************************** // Read the port and adapter settings // ********************************** // Open the infostream. if((status = NT_InfoOpen(&hInfo, "replay")) != NT_SUCCESS) { NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_InfoOpen() failed: %s\n", errorBuffer); return -1; } // Read the port and adapter settings infoRead.cmd = NT_INFO_CMD_READ_PORT_V9; infoRead.u.port_v9.portNo = (uint8_t) opt_portOffset; if((status = NT_InfoRead(hInfo, &infoRead)) != NT_SUCCESS) { NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_InfoOpen() failed: %s\n", errorBuffer); NT_InfoClose(hInfo); return -1; } NT_InfoClose(hInfo); // The selected port offset must be equal to the first port on the adapter. if(infoRead.u.port_v9.data.adapterInfo.portOffset != (uint32_t)opt_portOffset) { fprintf(stderr, "\"-p %d\" is not a offset to a port0 on a physical adapter.\n", opt_portOffset); return -1; } // Get the adapter number gsyncAdapter = infoRead.u.port_v9.data.adapterNo; // *********************************************** // Check that the setup of the adapter is correct. // *********************************************** // Is sync supported? if(infoRead.u.port_v9.data.adapterInfo.globalSync.supported == 0) { fprintf(stderr, "Global sync is not supported by this adapter no. %d.\n", gsyncAdapter); return -1; } // Only absolut TX timing mode can be used if(infoRead.u.port_v9.data.adapterInfo.txTiming == NT_TX_TIMING_RELATIVE) { fprintf(stderr, "Global sync cannot be used when running relative TX timing.\n"); return -1; } // Only native unix timestamp can be used if(infoRead.u.port_v9.data.adapterInfo.timestampType != NT_TIMESTAMP_TYPE_NATIVE_UNIX) { fprintf(stderr, "Global sync can only be used when using \"native unix\" timestamps.\n"); return -1; } // Adapter must be running the capture/replay profile for this to work. if(infoRead.u.port_v9.data.adapterInfo.profile != NT_PROFILE_TYPE_CAPTURE_REPLAY) { fprintf(stderr, "Please use the \"CaptureReplay\" profile.\n"); return -1; } // ********************************** // Setup Global sync for the adapter. // ********************************** // Get the adapter time stamp if (GetAdapterTimestamp(gsyncAdapter, &gsyncAdapterTS) != NT_SUCCESS) { fprintf(stderr, "Reading adapter timestamp failed\n"); return -1; } // Validate start time is not in the past if (gsyncStartTime <= gsyncAdapterTS) { fprintf(stderr, "Error: Start time is in the past.\n"); return -1; } // Sync timestamp -g is not defined. Use first timestamp in the // packet as sync timestamp. This means the transmission will start // with respect to the local file and not with respect to the other // files in this synchronized transmit. if(!(optionMask & OPTION_GSYNC)) { gsyncSyncTS = gsyncFirstTS; } // Calculate the global sync offset gsyncOffset = CalcOffset(gsyncStartTime, gsyncFirstTS, gsyncSyncTS); // Write the offset to the adapter if (SetGlobalSync(gsyncAdapter, 0xFF, gsyncOffset) != NT_SUCCESS) { fprintf(stderr, "Setting Global Sync failed\n"); return 1; } // Print the transmission start time on the screen. if (printTransmissionStart(gsyncAdapter, gsyncFirstTS, gsyncOffset) != NT_SUCCESS) { fprintf(stderr, "Printing start time failed\n"); return 1; } // **************************** // Start transmitting the file. // **************************** // Open the capture file to replay (captured with the capture example) if((status = NT_NetFileOpen(&hNetFile, "FileStream", NT_NET_INTERFACE_SEGMENT, opt_file)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NetFileOpen() failed: %s\n", errorBuffer); return -1; } // Open a TX hostbuffer on NUMA node 0 that can transmit to port 0 if((status = NT_NetTxOpen(&hNetTx, "TxStreamPort0", (uint64_t)1<<opt_portOffset, NT_NETTX_NUMA_ADAPTER_HB, 0)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NetTxOpen() failed: %s\n", errorBuffer); return -1; } // Get segments from the file and transmit them to port 0 while(appRunning) { NtNetFileRead_t data; // Get the packet if((status = NT_NetFileGet(hNetFile, &hNetBufFile)) != NT_SUCCESS) { if(status == NT_STATUS_END_OF_FILE) { // The file has no more data break; } // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NetFileGet() failed: %s\n", errorBuffer); return -1; } // Store the last timestamp in the read segment. We need this // timestamp in order to know when the transmit is finished. data.cmd = NT_NETFILE_READ_INFO_CMD_V1; if((status = NT_NetFileRead(hNetFile, &data)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NtNetFileRead_t() failed: %s\n", errorBuffer); return -1; } gsyncMaxTS = data.u.info_v1.lastTimestamp; // Get a TX buffer for this segment while ((status = NT_NetTxGet(hNetTx, &hNetBufTx, opt_portOffset, NT_NET_GET_SEGMENT_LENGTH(hNetBufFile), NT_NETTX_SEGMENT_OPTION_RAW, 1000)) != NT_SUCCESS) { if(status != NT_STATUS_TIMEOUT) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NetFileGet() failed: %s\n", errorBuffer); return -1; } // Print the start time on the screen and count down to start. if (printTransmissionStart(gsyncAdapter, gsyncFirstTS, gsyncOffset) != NT_SUCCESS) { fprintf(stderr, "Printing start time failed\n"); return 1; } if (appRunning == 0) { goto CTRLEXIT; } } // Copy the segment into the TX buffer memcpy(NT_NET_GET_SEGMENT_PTR(hNetBufTx), NT_NET_GET_SEGMENT_PTR(hNetBufFile), NT_NET_GET_SEGMENT_LENGTH(hNetBufFile)); // Build a packet netbuf structure _nt_net_build_pkt_netbuf(hNetBufTx, &pktNetBuf); if(firstPacket == 1) { // The first packet must have txnow=0 and txsetclock=0 NT_NET_SET_PKT_TXNOW((&pktNetBuf), 0); // Gsync mode: txnow = 0 NT_NET_SET_PKT_TXSETCLOCK((&pktNetBuf), 0); // Gsync mode: txsetclock = 0 firstPacket = 0; } // Optional step. Here all packets are inspected before sent if(NT_NET_GET_SEGMENT_LENGTH(hNetBufTx)) { do { // Just count the amount of packets and wire length numPackets++; numBytes+=NT_NET_GET_PKT_WIRE_LENGTH((&pktNetBuf)); } while(_nt_net_get_next_packet(hNetBufTx, NT_NET_GET_SEGMENT_LENGTH(hNetBufTx), &pktNetBuf)>0); } // Release the TX buffer and the packets within the segment will be transmitted if((status = NT_NetTxRelease(hNetTx, hNetBufTx)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NetTxRelease() failed: %s\n", errorBuffer); return -1; } // Release the file packet if((status = NT_NetFileRelease(hNetFile, hNetBufFile)) != NT_SUCCESS) { // Get the status code as text NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); fprintf(stderr, "NT_NetFileRelease() failed: %s\n", errorBuffer); return -1; } // Print the number of packets transmitted to the screen if(!(numPackets % 1000)) { printf("Packets transmitted so far: %llu \r", (unsigned long long) numPackets); fflush(stdout); } } CTRLEXIT: // Close the file stream NT_NetFileClose(hNetFile); // ************************************ // Wait for the file to be transmitted. // ************************************ // We need to wait for the file to be transmitted before we can disable // gsync again. Disabling the gsync before the file has been trasnitted, // we cause the file to be transmitted immediately. // To wait we are comparing the adapter timestamp with the last packet timstamp // used in the file. When the adapter timestamp is greater than the last timestamp // + the gsync offset. Then the file has been transmitted. // Set app running so the user has a chance to cancel the wait. appRunning=1; while (appRunning) { uint64_t ts; int64_t diff; if (GetAdapterTimestamp(gsyncAdapter, &ts) != NT_SUCCESS) { fprintf(stderr, "Reading adapter timestamp failed\n"); return 1; } if (ts > (gsyncMaxTS + gsyncOffset)) { break; } diff = (((int64_t)((gsyncMaxTS + gsyncOffset) - ts) / 100000000) + 1); if(diff < 0) { // The file is transmitted. break; } printf("Awaiting completion in %lld seconds \r", (long long int)diff); fflush(stdout); // Sleep 1 sec #if defined(__linux__) || defined(__FreeBSD__) sleep(1); #elif defined(WIN32) || defined (WIN64) Sleep(1000); // sleep 1000 milliseconds = 1 second #endif } // Close the TX stream. If cancel on close is enabled. The remaning data will be // canceled when the stream is closed. If cancel on close is not enabled the // remaining packets will be transmitted according to the timestamp, when // global sync is disabled. According to the timestamp means that the adapter // will look at the timestamp before transmitting the packets. This will // either cause the packets to be transmitted immediately or cause the to // hang until the timestamp matches the adapter timestamp. In onther way // it is unpredictable to say what will happen. NT_NetTxClose(hNetTx); // We need to disable Global Sync before we stop. if (SetGlobalSync(gsyncAdapter, 0, 0) != NT_SUCCESS) { appRunning = 0; fprintf(stderr, "Disabling Global Sync failed\n"); return 1; } printf("Done: %llu packets %llu bytes has been replayed \n", (unsigned long long int)numPackets, (unsigned long long int)numBytes); DisplayProgramFooter(); return 0; }