P4 and INC Architecture: Implementing In-Packet Computing & State Retention
As GPU clusters utilizing RoCEv2 scale up in AI data centers, there is growing interest in high-performance network infrastructures that leverage SmartNICs, DPUs, and programmable switches.
Among these technologies, In-Network Computing (INC)—which performs computational processing within network devices—holds promise for reducing communication latency and alleviating host CPU load. P4 is widely used as a key programming language for implementing such INC capabilities.
This article explains the mechanisms of INC implementation using P4, the in-network computing architecture, state retention design, and the memory requirements that support these elements.
Fundamentals of P4 and In-Network Computing (INC)
P4 is a language that enables the programming of data plane processing in network devices. In-Network Computing (INC) leverages this capability to perform data processing directly within the switch; this technology contributes to accelerating large-scale distributed systems and reducing host workloads, attracting growing interest in the network equipment design community.
What Is P4?
P4 (Programming Protocol-independent Packet Processors) is a domain-specific language designed to define packet processing logic for network devices. Its defining characteristic is the ability to rewrite packet processing logic—which was previously fixed in traditional ASICs—much like software. The specifications were established by the P4 Language Consortium, and the language is designed to be compiled for diverse execution environments, including general-purpose processors, FPGAs, network processors, and ASICs.
In recent years, P4-based programmable switches have seen increasing adoption in Ethernet fabrics for large-scale AI data centers and cloud networks.
Why INC Is Gaining Attention
In traditional network processing, computations were performed only after packets traveled through switches to reach the host server. However, with the proliferation of cloud services and the explosion of data volumes, CPU processing loads on the host and latency caused by round-trip communication have become major challenges. INC addresses these issues by performing operations—such as filtering, aggregation, and caching—inside the programmable switch simultaneously with packet forwarding; this approach reduces the volume of data transferred to the host and minimizes overall processing latency.
In AI training clusters, specifically, the increase in inter-GPU communication means that processing involving the host CPU can become a system bottleneck. This has led to growing interest in offloading portions of this processing to the network using technologies such as SmartNICs, DPUs, and P4-enabled switches.
Differences Between Host-Based Processing and INC
In host-based processing, packets reach a server equipped with a CPU and memory before operations are performed, allowing for flexibility in handling complex calculations and large-scale data lookups. In contrast, INC performs operations during the brief interval while a packet passes through the switch; a major advantage is obtaining results without waiting for round-trip communication. However, the operations INC can perform are heavily constrained by hardware limitations—such as the number of pipeline stages and memory capacity—so the two methods should be selected based on the specific processing requirements and latency needs.
Design Elements of In-Network Computing (INC) with P4
INC using P4 is realized by combining match-action tables with a pipeline architecture. In actual INC implementations, factors such as the placement of these elements and the location of state storage significantly influence processing performance, scalability, and implementation complexity; therefore, careful consideration from the initial design stage is essential.
Leveraging Match-Action Tables
Match-action tables form the core of a P4 program. By defining match conditions—which check packet header field values—and the corresponding actions to be executed, developers can specify operations such as header rewriting, counter updates, and filtering. In INC implementations, this table structure is used to perform aggregation and conditional branching, thereby constructing the computational logic applied to individual packets. Accurate estimation during the design phase is crucial, because table size and the number of entries are constrained by the switch ASIC’s memory resources.
Pipeline Design for Programmable Switches
Programmable switches process packets through a staged pipeline consisting of a parser, multiple match-action stages, and a deparser. P4’s ability to define the pipeline structure programmatically means that design decisions regarding which stage hosts specific computational logic directly impact throughput and latency. Designers must be aware that computational resources and memory capacity within each stage are limited; improper resource allocation across stages can prevent the system from achieving its target performance.
In AI data centers, where real-time communication processing at the scale of hundreds of Gbps is required, pipeline design optimization has a major impact on system performance. Designing for effective integration with SmartNICs and DPUs is particularly important.
Implementation Examples: P4COM and P4CEP
Several concrete implementation frameworks using P4 have been proposed within the field of INC research. P4COM is a framework that enables MapReduce-style distributed processing directly on programmable switches, while P4CEP handles complex event processing to detect correlations between multiple events within the switch. Both share the characteristic of performing computations while maintaining state using the switch’s memory resources; consequently, achieving practical deployment requires a careful evaluation of the trade-off between memory capacity and processing performance.
These technologies hold promise for applications such as network telemetry, AI cluster monitoring, real-time anomaly detection, and aggregation processing for distributed systems.
The Importance of State Retention in INC Implementations
Many INC-based computations require not only the processing of individual packets but also aggregation or decision-making that spans multiple packets. The mechanism for maintaining state within the switch is therefore a critical factor determining the performance, accuracy, and long-term operational stability of an INC implementation—making it an issue that demands careful consideration from the initial design stages.
Constraints Imposed by Stateful Processing
Stateful processing in P4 is achieved by storing values—such as counters and flags—in areas known as registers. However, complex state read/write operations increase the pipeline’s total clock cycle count and implementation difficulty, because programmable switch pipelines operate under strict constraints regarding processing time per stage. Designs relying heavily on conditional branching also lead to higher management costs; consequently, a design philosophy that limits state management to the absolute minimum is essential from the earliest stages of development.
State Management Using Registers and Metadata
P4 utilizes persistent storage areas called registers to hold values shared across packets. By combining these with “metadata”—temporary variables valid only during packet processing—developers can manage the state required for updating aggregate values and performing conditional checks. There is a physical limit to the scale of processing that can be implemented, since these mechanisms rely on the on-chip memory capacity of the switch ASIC; as the number of states grows, it is often necessary to consider switching to a higher-end switch ASIC model with greater memory capacity.
In recent years, SmartNICs and DPUs have increasingly offloaded network processing to reduce the load on host CPUs, making efficient state management a critical design challenge in these environments as well.
The Role of Memory Technology in State Retention
The memory performance required to support state retention varies depending on the implementation layer. For states referenced within the pipeline, keeping pace with processing speeds is the top priority; conversely, in the layer where the control plane stores aggregation results, non-volatility (retaining state during power loss) and write endurance (withstanding long-term operation) become paramount. Making the right choices regarding memory usage based on these differing characteristics is a crucial design decision that determines the reliability and scalability of the entire INC implementation, while the growing need for telemetry infrastructures in AI data centers to continuously record statistical data and anomaly detection results further increases the importance of non-volatile memory.
Memory Requirements for INC and the Use of FeRAM
State retention in In-Network Computing (INC) imposes different requirements on the high-speed data path within the pipeline versus the persistence layer on the control plane. FeRAM (Ferroelectric RAM) stands out as a compelling choice for the latter, offering a combination of non-volatility and high endurance. Understanding these characteristics is essential to selecting memory accurately.
Three Key Memory Requirements for State Retention
The memory requirements for INC state retention vary depending on the implementation layer. In the high-speed path, where updates occur on a per-packet basis, processing speed is the top priority; consequently, on-chip SRAM embedded directly into the silicon is the standard choice for pipeline stage registers. In contrast, the layer responsible for persisting aggregation results demands non-volatility (to retain data after power loss) and high endurance (to withstand frequent write operations); selecting a non-volatile memory that meets these criteria is critical to the success of the implementation.
In particular, recording telemetry data and anomaly event histories requires a memory solution capable of handling both high-frequency writes and data retention following power loss.
Comparison of Non-Volatile Memory Candidates
In the control plane layer responsible for persisting aggregated values and flags, the use of non-volatile memory is a prerequisite. While candidate technologies like EEPROM and Flash offer high capacity and low cost, they require an erase operation prior to writing; consequently, write speeds are limited to the range of tens of microseconds to several milliseconds, and write endurance is capped at approximately 100,000 to 1 million cycles. In contrast, FeRAM (Ferroelectric RAM) eliminates the need for an erase operation, achieves write times of around 150 nanoseconds, and offers write endurance reaching 10^13 cycles, making it superior for applications involving frequent updates.
Use Cases for FeRAM
FeRAM excels in scenarios requiring high write frequency where data loss is unacceptable. For instance, when writing counters or anomaly detection flags that are updated at intervals of a few milliseconds via the control plane, EEPROM could quickly reach its maximum write cycle limit.
FeRAM’s combination of high-speed writes and high endurance makes it well-suited for telecommunications infrastructure equipment requiring frequent state updates, such as:
- P4-based INC systems
- SmartNICs
- DPUs
- FPGA accelerators
- Network telemetry devices
Similar requirements also apply to 800G and 1.6T optical modules. Within these modules, data such as CMIS management information, diagnostic logs, FEC error counters, temperature history, and DSP configuration settings are continuously updated, creating a need for highly durable non-volatile memory.
Summary
As AI data centers scale up, there is growing interest in In-Network Computing (INC), a technology that equips network devices themselves with computational capabilities. Programmable switches, SmartNICs, and DPUs leveraging P4 are key technologies that contribute to enhancing the efficiency of GPU clusters and large-scale Ethernet fabrics.
Such systems require the continuous storage of data such as aggregated information, telemetry data, and anomaly event logs. FeRAM, combining high endurance, high-speed writes, and non-volatility, is a strong memory technology option—not only for P4/INC systems but also for next-generation communication infrastructure and optical modules.
RAMXEED FeRAM Product Lineup
