 |
|
Examining Option 1a in More Detail
This section examines option 1a in more detail than was discussed earlier in the "RSS Implementation Options" section. This section shows the sequence of packet receive processing from the perspective of a network adapter.
The sequence of events shown here is intended to be a high-level example, and implementation issues may force different implementation options. All options presented in this paper are examples that can be used to help IHVs understand various RSS implementation issues. Additional implementation options are also possible.
1. At the network adapter, as packets arrive off the wire, the hash fields are selected according to the hash type, the RSS hash is calculated, the hash mask is applied to the result, the indirection table result is added to the BaseCPUNumber, and the result is used to find the CPU which should process the packet.
2. The packet contents are transferred into host memory using Direct Memory Access (DMA) writes (the packet contents could be transferred to a processor-specific pool of buffers), and a receive descriptor is transferred into the receive descriptor queue for the specific processor by means of a DMA write.
3. Depending on vendor innovation, a CPU vector might be transferred by means of a DMA into host memory when the network adapter is about to cause an interrupt.
4. An interrupt will eventually be posted to the host to indicate presence of new data. Exactly when the interrupt fires depends on the vendor’s interrupt moderation scheme. Depending on the system architecture supported by the system vendor, the interrupt will either be distributed to any host processor (based on a vendor-specific heuristic), or it will be routed to the processor that the system vendor has designated as the processor to handle the adapter’s interrupts. Normally, the network adapter will no longer interrupt the host until it is specifically enabled by one of the DPCs.
5. If additional packets arrive at the network adapter, data and descriptors are transferred by means of DMA to host memory. The network adapter must start tracking the CPU vector from the time the interrupt was sent, not from the time when the interrupt was reenabled. Otherwise, a race condition exists where a particular receive descriptor queue might not be processed for an indeterminate amount of time. The exact race condition exists if a packet arrives on a different receive descriptor queue, say queue 2, than that specified in the CPU vector (say, queues 0 and 1). The miniport driver DPCs for queues 0 and 1 finish processing and reenable the interrupt. The network adapter starts tracking receive descriptors again, but the received packets do not contain any additional packets that hash to receive descriptor queue 2. If the CPU vector updates began immediately after sending the interrupt, then the race condition is eliminated.
6. The interrupt service routine runs on the host processor that the interrupt was routed to. The interrupt service routine processes the CPU vector, initializes an atomic counter to the number of DPCs about to be scheduled, and then schedules a vector of DPCs to execute.
7. When the DPCs run, they process the receive descriptors on their receive descriptor queue. They each hand packets (preferably an array of packets) up to the NDIS interface, preserving the packet order in the receive descriptor queue. However, because multiple DPCs are running in parallel, the packet order between CPUs may not match the order in which the network adapter received the packets from the network.
8. Depending on vendor innovation, the device interrupt is enabled either just before all DPCs have exited or at some other time.