Category Archives: Datasheet

3.7. Optional Diagnostics and Network Statistics

3.7. Optional Diagnostics and Network Statistics

In particular at faster line rates, diagnosing network protocol issues can be costly, time-consuming and challenging. Therefore, MLE NPAP comes with specialized diagnostics blocks, providing a powerful suite of tools for in-depth network analysis and efficient troubleshooting of network behavior. These diagnostics offer TCP/UDP/IPv4 layer-specific visibility of counters and status information similar to those in Linux or Windows, for example:

• TCP Core Activity: Track performance and network usage of individual cores.
• TCP Peer Balance: Monitor data flow and identify peer misconfigurations.
• Buffer Optimization: Fine-tune buffer settings using diagnostic data and RTT.
• General Troubleshooting: Track packets, errors, and pinpoint issue sources.

MLE NPAP’s diagnostics blocks are accessed using AXI4-Lite and enable efficient diagnosis and a faster path towards resolving network problems when using hardware acceleration.

3.7.1. Diagnostics Data Acquisition Modes

The MLE NPAP diagnostics system supports multiple data acquisition modes to cater to different debugging and analysis needs. These modes define how the underlying counter and event state registers are updated. The selected mode applies universally to all readable counter and event state registers within the respective diagnostics block:

3.7.2. Network Diagnostics Resource Costs

For better design visibility and for gathering network statistics and diagnostics MLE NPAP has the option to spend additional FPGA resources. 

Read More

3.8. Optional Benchmarking using Netperf

3.8. Optional Benchmarking using Netperf

MLE NPAP can instantiate a fully accelerated version of Netperf and many of our Evaluation Reference Designs do so. This Netperf/Netserver block is compatible with open source Netperf 2.6 and can be used for system-wide tuning, for functionality analysis and for performance benchmarking.

The charts below show the results of a 25 GbE NPAP-to-NPAP benchmark, underlining NPAP’s superior performance:

Read More

3.9. Network Impairment and Bit Error Insertion

3.9. Network Impairment and Bit Error Insertion

MLE NPAP has an optional integrated Network Impairment Generator (Network Impairment is also known as Bit Error Insertion). This helps to rigorously test and validate the robustness of your network by emulating real-world network impairments, such as Bit Error Rates (BER), at line speed.

The Ethernet standard requires a BER no higher than 1×10⁻¹². NPAP’s Network Impairment Generator allows you to test your system far beyond this ideal requirement, simulating real-world conditions like those found in high-EMI environments or when using slip rings, for example.

The Network Impairment Generator IP Core sits on the data path between the TCP/UDP/IPv4 layer and the Ethernet MAC layer, allowing for precise, controlled injection of errors. This enables you to perform stress testing and gain valuable insights into system performance under non-ideal conditions, ensuring your implementation remains resilient against link quality degradation.

The graph below illustrates how different levels of Bit Error Rate (BER) over a 10 GBit/s link, emulated using NPAP’s Network Impairment Generator, affect the TCP throughput of a TCP connection between two MLE NPAP instances. Obviously, the TCP re-transmissions “eat” into the net data throughput:

Read More

4. System-Level Architecture Using TCP/UDP/IPv4

4. System-Level Architecture Using TCP/UDP/IPv4 Team MLE has gained long and deep experiences with integrating NPAP into networked and distributed systems and will support you in identifying, implementing and testing the right architecture choice. Key aspects are outlined below. Please contact us for more details. Contact Us 🌐 www.missinglinkelectronics.com MLE (Missing Link Electronics) is offering technologies and solutions for Domain-Specific Architectures, which focus on heterogeneous computing using FPGAs. MLE is headquartered in Silicon Valley with offices in Neu-Ulm and Berlin, Germany.

Read More

4.1. Picking the “Right” TCP Peer

4.1. Picking the “Right” TCP Peer MLE NPAP has a clear focus on highest performance in a very resource efficient FPGA implementation which is reliable (the key reason to use TCP) and interoperable with (almost) any other TCP/UDP/IPv4 stack.  When you optimize data transports towards low latency and/or high bandwidth, keep in mind how both, TCP flow control and TCP congestion control, function and do parameterize both peers accordingly.  NPAP runs the entire protocol stack as a digital circuit. So, when NPAP is on the receiving side, TCP packets will be checked, and acknowledged (ACK’ed), in a very short time and at a very high rate (close to line rate). That may challenge a “slow sender”. Similarly, when NPAP is on the sender side, TCP packets will be generated and sent in a very short time and at a very high rate (close to line rate). That may challenge a “slow receiver”. Experimenting, and tweaking parameters on either side, is key to delivering good performance. Predictable high bandwidth and low latency is typically delivered by a “balanced” TCP connection, for example by putting NPAP on both sides. When implementing so-called High-Speed Data Acquisition systems we do recommend investigating NPAP not only for sensor-side TCP/IPv4 acceleration but also for PC-side: 🌐 www.missinglinkelectronics.com MLE

Read More

4.2. Picking the Right TCP Rx/Tx Buffer Sizes

4.2. Picking the Right TCP Rx/Tx Buffer Sizes

Obviously, larger Tx and Rx buffers deliver higher bandwidths, but at the cost of transport latency. Worse, NPAP Tx and Rx buffers require expensive FPGA BRAM resources. To help you pick a good tradeoff, here is the metric to determine TCP buffer sizes for TCP (keep in mind, TCP buffers are placed on both ends: Tx side and Rx side):

Read More

4.3. Picking the Right Number of TCP Cores

4.3. Picking the Right Number of TCP Cores

In typical software systems, the cost of opening a TCP connection is quite CPU expensive, and may take a long time because of RTT and processing times in the operating system. Therefore, most software driven systems keep a TCP connection alive “forever” rather than closing it. In software, the low costs of system RAM for storing each TCP connection’s state are not worth the CPU processing costs.

For NPAP, each TCP connection which is open at the same time requires a dedicated TCP Core – which costs FPGA / ASIC resources. However, if RTT is low such as in a LAN, and with the very low costs of opening and closing a TCP connection in NPAP (a few hundred FPGA clock cycles), “time sharing” TCP Cores can save a lot of FPGA resources without any negatives. Please refer to the MLE Technical Brief TB20201203 “Deterministic Networking with TSN-10/25/50/100G” for more information on “time sharing” TCP Cores.

Deterministic Networking with TSN-10/25/50/100G

Typically, FPGA routing is a key limiting factor: The more TCP Cores you instantiate, the more on-chip AXI-Stream FPGA routing congestion you will see, the harder it is to  achieve timing closure. As a rule of thumb, more than 20 TCP Cores may cause FPGA routing issues (obviously depending on your FPGA device and toolchain).

Likely On-Chip Routing Issue:

Read More

4.4. Optimizing NPAP for Linerate Performance

4.4. Optimizing NPAP for Linerate Performance

MLE is constantly working with FPGA vendors to improve NPAP clock frequency. While NPAP originally was designed for ASIC implementation, MLE has adopted NPAP for efficient implementation using modern FPGA fabric. Unlike other TCP stacks for FPGA, NPAP features a 128 bit wide bi-directional datapath which puts NPAP into a unique position for realizing high-bandwidth FPGA-based SmartNICs without “FPGA bloat”. 

Larger bit widths, 512 bits or more, cause “FPGA bloat” which is wasting FPGA resources. Smaller bit widths, 64 bits or less, do require unrealistic high clock frequencies to deliver high line rates as the following table shows:

Read More

4.5. Hybrid Acceleration by Combining NPAP With FPGA Network Interface Cards (NIC)

4.5. Hybrid Acceleration by Combining NPAP With FPGA Network Interface Cards (NIC)

Hybrid acceleration refers to the concept of combining MLE NPAP for full acceleration with an FPGA Network Interface Card (NIC). The NIC does some network packet handling in FPGA logic and then DMAs the data into a host CPU which then runs the Linux TCP/UDP/IPv4 stack. Because TCP does so-called fair bandwidth sharing *refer to RFC2914, RFC5166 and RFC8312) the motivation is to allow trade-offs between performance (as in throughput and latency) over FPGA resource usage for many TCP/IPv4 connections. For example, if you run many TCP connections over one single 100 GigE link, you may want full acceleration for, say, up to 20 TCP connections: When 3 TCP connections are open, then each one gets 33 Gbps bandwidth which clearly needs a full accelerator. But for moments, when 200 TCP connections are open, then each TCP connection gets 500 Mbps bandwidth, so you can run most of them un-accelerated, in software. Obviously, this needs an architectural decision where Team MLE can help!

At MLE we have been using (and contributing to) the Corundum project: http://corundum.io 

Corundum is an open-source, high-performance FPGA-based NIC and platform for In-Network Compute. Features include a high performance datapath, 10G/25G/100G Ethernet, PCIe connectivity to the host, a custom, high performance, tightly-integrated PCIe DMA engine, many (1000+) transmit, receive, completion, and event queues, scatter/gather DMA, MSI, multiple interfaces, multiple ports per interface, per-port transmit scheduling including high precision TDMA, flow hashing, RSS, checksum offloading, and native IEEE 1588 PTP timestamping. A Linux driver is included that integrates with the Linux networking stack. Development and debugging is facilitated by an extensive simulation framework that covers the entire system from a simulation model of the driver and PCI express interface on one side to the Ethernet interfaces on the other side (https://docs.corundum.io/en/latest/contents.html). 

One of the key advantages of the Corundum architecture is support for In-Network Processing inside the FPGA logic, shown as “App” in the block diagram:

Read More

4.6. Implementing Time-Sensitive Networking (TSN)

4.6. Implementing Time-Sensitive Networking (TSN) TSN has become a set of emerging, open IEEE standards with momentum in industrial markets (for 10/100/1000 Mbps speed) and in next-generation Automotive Zone architectures (for 10/25/50 Gbps speeds). Aspects such as Time-Aware Traffic Shaping also find application in telecommunication, Provider Back-Bone (PBB) Switching or Software-Define Wide Area Networks (SD-WAN), for example. TSN and TCP can be combined according to the OSI Layers.  The outcome is a deterministic and reliable network protocol, which makes TCP/IPv4 over TSN a very good candidate for all networking where IT (Information Technology) and OT (Operations Technology) converge, or in Systems-of-Systems backbones. TSN itself is quickly evolving so please contact us for more information! Contact Us 🌐 www.missinglinkelectronics.com MLE (Missing Link Electronics) is offering technologies and solutions for Domain-Specific Architectures, which focus on heterogeneous computing using FPGAs. MLE is headquartered in Silicon Valley with offices in Neu-Ulm and Berlin, Germany.

Read More