Introduction
Prefilled transmission segments enable an application to fill a part, or all, of a transmission segment once and then use the segment contents to send the same packets multiple times. The feature is best suited for applications that replay the same packets over and over again, however applications can also modify the contents of the segments along the way. An advanced application may even be able to detect at run-time that reuse is feasible.
The prefill of transmission (TX) segments feature ensures that the way the system allocates TX segments to the application is predictable; when the application is able to predict the allocation sequence, it may be able to reuse the contents of a TX segment and in that way save CPU resources. The prefill feature does not affect the contents of the packets in the TX segments used by the application; it is only the application that modifies the contents of the segment, which also applies when packet descriptors in the segments direct the adapter to modify the packets when they are transmitted like setting the ignore packet bit in the descriptor.
The feature may help an application reduce its CPU usage, but the exact performance depends on the application. For example, (dependent on the options used) the pktgen
tool uses the feature to reduce its CPU usage by recycling transmission segments that it creates once.
Configuration
It is not necessary to configure or initialize the feature; it is built into the driver and accessed through the API.
API
An application can specify the NT_NETTX_SEGMENT_OPTION_RAW option when it allocates a transmission segment. This option instructs the driver to retain the contents of the segment while the driver has control of the segment and to allocate segments to the application in a predictable way. In other words, the option ensures that the contents of the segment passed on to the application is identical to the contents when the application last released that segment, and the application can predict segment allocation. When using the option, it is only the application that modifies the contents of the segment.
When using the prefill feature, there are some properties worth highlighting as follow:
When calling NT_NetTxOpen the fifth argument named
minHostBufferSize
specifies the number of one megabyte segments (also called slabs) to be reused.For example, if a program needs twelve megabytes, it should specify the value twelve as the
minHostBufferSize
argument in the call to NT_NetTxOpen, which directs the driver to set up twelve segments of one megabyte each.The driver arranges the twelve segments as pearls on a (closed) circular string.
- Use NT_NETTX_SEGMENT_OPTION_RAW as
packetOption
in the call to NT_NetTxGet. - The maximum segment size (the fourth argument in the call to NT_NetTxGet) is one megabyte.
- If there are N segments (pearls) on the circular string, the first N calls to NT_NetTxGet return a new segment, while call N+1 returns the (same) segment from call 1. If the function g denotes getting a segment, then the circular property implies g(x) = g(x mod N), where mod denotes the integer modulus operator.
It is legal to release allocated segments out of order, but the contents (the packets) of released segments are transmitted in the same order as the allocations, i.e., as returned by NT_NetTxGet. If the function si=g(i) denotes getting segment with number i, and r(si) denotes releasing segment number i, then the following three call sequences yield the same result:
g(1), r(s1), g(2), r(s2)
g(1), g(2), r(s2), r(s1)
g(1), g(2), r(s1), r(s2)
- It is legal to specify a different value for the fourth argument named
packetSize
in the call to NT_NetTxGet for the same segment. The system never modifies the contents of the entire segment (up to 1 megabyte in size), only the application does, but the application must only read and write the number of bytes specified by thepacketSize
argument. - When calling NT_NetTxGet, it is recommended that the arguments of NT_NetTxGet be used consistently to ensure proper reuse.
Limitations
The option NT_NETTX_SEGMENT_OPTION_RAW pertains to a segment, not the entire host buffer. It is essential that an application uses the option consistently. Once an application uses the option for an adapter, it should use that option all of the time (for that same adapter).
Tips
- Use the prefill feature for an application that tends to transmit the packets using a fixed pattern. When an application is able to reuse the contents of segments, it generally reduces the CPU usage of the application.
- An application should use the smallest possible
minHostBufferSize
when calling the function NT_NetTxOpen. Specifying a value that is too large, considering that an application does not actually use the superfluous segments, wastes PCI Express bandwidth; an application should always try to pack as many packets as possible into a segment to optimize performance. - An application can use NT_NET_SET_PKT_TXIGNORE to set the
txIgnore
bit in the packet descriptor for frames (packets) that it does not want to send. The application can use this approach to fill the segment once, and then use thetxIgnore
bit to control which of the frames to transmit.