Coordinated Time-Synchronized Transmit

Reference Documentation

product_line_custom
Intel® PAC
Napatech SmartNIC
category
Reference Information
Napatech Software Suite: Coordinated Time-Synchronized Transmit
Coordinated Time-Synchronized Transmit

Introduction

This feature is used to synchronize transmit across several adapters and servers. Synchronizing transmits means that the start of a file transmit can be synchronized over a number of adapters and servers so the transmission is started at the same time.
All adapters must be time synchronized for the synchronized transmit to work. This can be done either by using NT timesync or PPS timesync.

Prerequisites

In order for coordinated time-synchronized transmit to work, the following prerequisites must be fulfilled.
  1. The capture file must be in 3GD format
  2. All adapters must support the coordinated time-synchronized transmit feature
  3. TxTiming must be set to ABSOLUTE in the ntservice.ini file
  4. The timestamp format TimestampFormat must be set to NATIVE_UNIX in the ntservice.ini file
  5. All adapters must be synchronized using time sync
[System]
TimestampFormat = NATIVE_UNIX
[Adapter0]
TxTiming = ABSOLUTE

How it Works

To synchronize the transmit an offset is programmed to the adapter telling the adapter when to start the transmit. To calculate the transmit, offset 3 values are needed.
  1. The adapter timestamp
  2. The timestamp of the first packet in the capture file
  3. A time transmit for starting the transmit
The transmit delay is calculated in the following way where:
  • Tp = The time stamp of the first packet
  • Ta = The current (running) timestamp of the adapter
  • Ts = The time for starting the transmit
  • Of = The transmit offset for the adapter
  • Offset = The calculated transmit offset
The adapter uses the offset in the following way:
Ts - Tp > Ta + Of : wait, do not send the packet yet
Ts - Tp <= Ta + Of : send the packet
The APP calculates the transmit offset differently, namely as:
Ta - Ts + Tp = Offset
and hence:
Ts - Tp = -Offset + Ta
If this last equation is inserted into the right-hand side of the expression used in the adapter (Ts - Tp = Ta + Of), this becomes:
Ts - Tp = Ta + Of = -Offset + Ta
and therefore:
Of = -Offset
The APP therefore uses the negated value of Of as the offset value on the adapter.

Set Up the Coordinated Time-Synchronized Transmit

The coordinated time-synchronized transmit is enabled per port using the config stream NT_ConfigWrite to program the transmit offset offset and a port enable mask to the adapter. The transmit offset is common for all port on the adapter.
To set up coordinated time-synchronized transmit, the globalSyncWrite structure in the NtConfig_t structure must be used. See the setup example below:
// Send the global sync offset to the adapter
configWrite.parm = NT_CONFIG_PARM_ADAPTER_GLOBAL_SYNC; // Global sync offset write command
configWrite.u.globalSyncWrite.adapterNo = adapterNo; // Adapter number
configWrite.u.globalSyncWrite.data.portEnableMask = portMask; // Port enable mask for the
// ports where global sync should be enabled
configWrite.u.globalSyncWrite.data.timeSynchronizedTxOffset = (~offset) + 1; // The calculated global sync offset.
// Note: The value must be written to
// the as two's complement.
configWrite.u.globalSyncWrite.data.timeSynchronizedTxOffsetUpdate = portMask!=0?1:0; // Set to 1 when updating the offset.
if((status = NT_ConfigWrite(hStream, &configWrite)) != NT_SUCCESS) {
fprintf(stderr, "NT_ConfigWrite() failed: %s\n", errorBuffer);
NT_ConfigClose(hStream);
return status;
}
Note: The offset programmed to the adapter must be the two's compliment of the calculated offset.

Coordinated Transmit with all Transmissions Starting at the Same Time

A coordinated transmit can be done with all transmissions starting at the same time no matter what the timestamps are.
3 capture files need to be transmitted from 3 different devices:
Capture File First Timestamp
Test1.cap 0001
Test2.cap 0307
Test3.cap 1246
The timestamp of the first package in each file needs to be found. This is done using the replay tool or the replayGS example.
Tool:
  • replay -f Test1.cap -t
  • replay -f Test2.cap -t
  • replay -f Test3.cap -t
Example:
  • replayGS -f Test1.cap -t
  • replayGS -f Test2.cap -t
  • replayGS -f Test3.cap -t
Both programs print the timestamp to the screen.
The result is a timestamp for the first packet in each file : 0001, 0307 and 1246.
The timestamp is needed to calculate the transmit offset and must be used as input to the programs.
Tool:
  • replay -f Test1.cap -g 0001 -s 02:34
  • replay -f Test2.cap -g 0307 -s 02:34
  • replay -f Test3.cap -g 1246 -s 02:34
Example:
  • replayGS -f Test1.cap -g 0001 -s 02:34
  • replayGS -f Test2.cap -g 0307 -s 02:34
  • replayGS -f Test3.cap -g 1246 -s 02:34
This causes the first packet of all files to be transmitted at the time 02:34. The next packets in the files are transmitted according to their timestamps.
Note: The start time used is the adapter time, not the OS time.

Coordinated Transmit with All Transmissions Starting According to the Timestamp

A coordinated transmit can be done with all transmissions starting according to the timestamp in the first packet. In the above example this means that the first packet in Test1.cap will be sent at 02:34 and the first packet in Test2.cap and Test3.cap is sent respectively 306 and 1245 ticks later.
For this scenario the oldest timestamp is used as parameter.
Tool:
  • replay -f Test1.cap -g 0001 -s 02:34
  • replay -f Test2.cap -g 0001 -s 02:34
  • replay -f Test3.cap -g 0001 -s 02:34
Example:
  • replayGS -f Test1.cap -g 0001 -s 02:34
  • replayGS -f Test2.cap -g 0001 -s 02:34
  • replayGS -f Test3.cap -g 0001 -s 02:34

Example Using Coordinated Time-Synchronized Transmit

How to use coordinated time-synchronized transmit can be seen in the replayGS example.

Transmit Clock and Ports

When Coordinated Time-Synchronized Transmit is enabled, all the enabled ports share the same TX clock. The TX clock is used when transmitting packets. A packet is transmitted when the (TX clock - TxDelayOffset) is greater than or equal to the timestamp in the packet. The TX clock is equal to the adapter time.
When Coordinated Time-Synchronized Transmit is enabled the TxDelayOffset must be calculated in a way so the TX will be started at the time wanted.
The TxDelayOffset can be calculated as:
TxDelayOffset = TimeOfStart - AdapterTime - TimeStampOfFirstPacket.
Using this calculation, the first packet will be transmitted at TimeOfStart. As it can be seen above, the calculation is a bit advanced.
Note
The Coordinated Time-Synchronized Transmit can be enabled/disabled per port, but disabling some of the ports can result in unpredictable transmits on the disabled ports.