PER-LANE BIT RATE SATURATION AND PARALLELISM

Data Processing Chip I/O. Advances in CMOS manufacturing and chip packaging technologies have allowed an unabated 40% per year exponential growth of the digital processing capability of single-die and multi-die (chiplet) packages. This is visualized in Fig. 1 [1], showing the multi-decade capacity scaling of Ethernet switch chips (green). However, per-lane electrical chip I/O rates have only scaled at 20% per year (red), enabled by a ${2\times}$ boost from multi-level modulation (PAM-4) and leveraging significant digital signal processing (DSP) and forward error correction (FEC) afforded by advanced CMOS nodes. Analog bandwidths and electrical symbol rates have only scaled at 13% per year (purple), as have optical Ethernet rates per lane (blue). Sampling rates of data processing chip-integrated analog-to-digital and digital-to-analog converters (ADCs, DACs, orange) have yet been scaling slower and are showing a strong saturation in the 100 GS/s range [2]. The highest-speed electrical I/Os and the highest-speed optical Ethernet interfaces commercially available today are at 112 Gbps PAM-4 (56 GBaud) per lane, and the next Ethernet standard to double this rate is expected for 2026 [3]. The latest Ethernet switches and ML/AI training chips interface just above 50 Tbps per die, and individual tiles of co-packaged machine learning/artificial intelligence (ML/AI) dies sport as much as 144 Tbps of I/O [4]. At such capacities, real estate and power consumption of on-chip I/O blocks (“SerDes”) becomes a major issue, with as much as 40% of the chip area spent on SerDes (at ${\sim}{0.2}\;{{\rm Tbps/mm}^2}$ and ${\sim}{4} {-} {5}\;{\rm pJ/bit}$), leaving only 60% of the chip area for revenue-generating data processing [5]. The widening gap between chip I/O rates and data processing capabilities results in an exponentially increasing number of I/O lanes (cf. Fig. 1, double arrow). The latest Ethernet switches at 51.2 Tbps require 512 full-duplex 100-Gbps electrical I/O lanes, corresponding to ${\gt} {4000}$ electrical contacts for high-speed differential signals, not counting power supplies and low-speed pins. Doubling the number of I/O lanes to double chip capacity is a packaging challenge, but further parallelization is unavoidable given the saturating per-lane I/O rates.

 

Fig. 1. CMOS data processing and chip I/O scaling trends (solid: historic data; dotted: announcements/expectations).

Download Full Size | PDF

The saturation in symbol rates and converter speeds is consistent with basic ${{f}_T}/{{f}_{{\rm MAX}}}$ properties of advanced CMOS [6], with ${{f}_T}$ scaling only at 4% per year; cf. Fig. 1 (black) [7]. Other semiconductor materials (such as InP) are inherently faster but do not allow for the sophisticated DSP that is necessary to compensate for RF losses and other transmission impairments plaguing high-speed multi-level signals: the dB loss of high-speed substrate traces increases approximately linearly with frequency. Hence, doubling the symbol rate implies doubling the dB roll-off of the channel, to be compensated for by significantly more complex DSP. In short, high-speed signals are punished by the channel. Note in this context that scaling bit rates through multi-level modulation only yields logarithmic gains: to keep doubling bit rates starting at two levels (PAM-2 or NRZ), one needs 4, 16, and 256 amplitude levels (PAM-4, PAM-16, PAM-256). Such a scaling is fundamentally limited by noise, which on short-reach links is set by the digitizing electronics [8]. A typical link with high-speed ADCs/DACs yields maximum signal-to-noise ratios (SNRs) between 25 and 30 dB [8–10], which comfortably enables PAM-4, lets PAM-8 be borderline, and induces a substantial error floor for PAM-16 that would require complex DSP and FEC (with significant latency) to operate reliably; the practical feasibility of 400-Gbps per-lane chip-integrated SerDes I/Os is thus highly questionable.

As a result of saturating interface rates and the substantial chip areas and powers needed to generate and receive such high-speed signals [5], the industry is considering chipletized architectures using real-estate friendly, highly parallel, and low-speed electrical interconnects that become possible on advanced fine-pitched (and more expensive) package substrates, such as UCIe [11] with up to 32-Gbps NRZ per lane, a target of up to 5 Tbps/mm of (full-duplex) chip-edge I/O density, and a clock-forwarded die-to-die reach of 2 mm. To escape chip packages, converter chiplets that translate UCIe to high-speed electrical signals are needed. To avoid fan-out, these converters need to be “pitch-matched” to the UCIe edge I/O density. The edge I/O density of today’s SerDes is typically below 500 Gbps/mm [5]. In addition, emerging co-packaged optics (CPO) allow for fiber-to-the-package designs, reduce lossy RF trace lengths, and enable higher bit rates. Pitch-matched CPOs achieve ${\sim} {200}\;{\rm Gbps/mm}$ with the help of poorly scaling wavelength-division multiplexing (WDM) [12,13]; scaling to multi-Tbps/mm requires 2D fiber arrays with a high degree of spatial parallelism [14].

Optics DSP Chips. In contrast to data processing chips, where I/O blocks are a “necessary evil” that enables revenue-generating compute functions, optics DSP chips are built for the sole purpose of I/O, to convert a set of high-speed signals interfacing with a host ASIC to a set of high-speed signals interfacing with optical hardware. Optics DSP chips exist for direct-detection (DD) and coherent-detection (COH) optics [15] and can afford to spend 100% of their area and power on I/O functions. Leading optics DSP chips interface 1.6 Tbps (DD) to 2.4 Tbps (COH), full-duplex host plus line side, at ${\lesssim} {0.1}\;{{\rm Tbps/mm}^2}$ (DD) and ${\sim}{0.02}\;{{\rm Tbps/mm}^2}$ (COH), the latter an order of magnitude below their data-processor-integrated counterparts. Figure 2 [1] shows per-lane optical interface rates in research (red) and products (blue). Post-2010, commercial interfaces were only able to sustain their 20% per-year growth rate due to an ${\sim}{10\times}$ boost by coherent detection, which multiplexes four tributaries (two quadratures, two polarizations), each carrying four- to eight-level PAM. Underlying analog bandwidths (symbol rates) have only scaled at 13% per year, in analogy to the symbol rates of CMOS chip I/Os; cf. Figure 1. Similar to the saturation of ADC/DAC sampling rates at ${\sim}{100}\;{\rm GS/s}$, a saturation of symbol rates at ${\sim}{200}\;{\rm GBaud}$ is visible in optics research records. The first ${\sim}{200{\text-}\rm GBaud}$ results were reported in 2016 [16] and no substantial progress has been made since. The fastest DD and COH research is at 538 Gbps [17] and 2 Tbps [18] to date, both relying on sophisticated DSP, modulation, and coding. The highest-speed commercial coherent optics use probabilistically shaped 140-GBaud QAM at 1.2 Tbps per-wavelength [19]; 200-GBaud optics DSP chips yielding 1.6 Tbps per-wavelength interface rates have been announced [20]. Note, though, that increasing symbol rates comes at a steep cost in terms of DSP complexity, as here, too, high-speed signals are punished by the channel. For example, the digital filter compensating for fiber chromatic dispersion scales with (up to) the square of the symbol rate [21,22].

 

Fig. 2. Wide-area optical interface scaling trends (solid: historic data; dotted: announcements/expectations).

Download Full Size | PDF

Table 1. Approximate Cost per Additionally Deployed Fiber Path in Multiples of Transceiver Cost per 100G

View Table

FIBER CAPACITY SATURATION AND PARALLELISM

Table 1 [23–25] shows the approximate cost per additionally deployed fiber strand (including line equipment), measured in units of 100-Gbps transceiver cost. For example, the cost per additionally deployed fiber path on a terrestrial 1000-km long-haul (LH) link roughly equals 4 Tbps of transponder cost. (Numbers are approximate and may have substantial variations.) The very high relative fiber costs in LH systems mandate multiplexing as much capacity onto a single fiber as possible to amortize line costs. This is achieved through WDM and coherent detection, with the goal to maximize a system’s spectral efficiency (SE), i.e., the system capacity per system bandwidth. For datacenter interconnects (DCIs), the cost per fiber is comparable to the cost per 100G of transceiver hardware, which requires much less multiplexing and instead asks for a careful balance between transponder technology (DD versus COH) and the number of wavelengths per fiber [23,26]. In these systems, achieving high SE is consequently much less important. Within the data center, and particularly for ML/AI clusters and memory disaggregation, there is little need for multiplexing traffic onto a single fiber from a techno-economic point of view. In some cases, secondary considerations such as reusing already deployed fiber infrastructure or optical switching may favor multiplexing, while the need for flexible fan-out in ML/AI clusters and lowest complexity/cost favor highly parallel single-wavelength spatial multiplexing (SDM) [23]. In support of increasingly massive SDM systems, multi-fiber connectors with 72 multi-mode or 32 single-mode fibers and high-fiber-count cables with 6912 fibers are commercially available today [27,28]. Larger array connector/fiber bundle technologies (${\gt}{1000}\;{\rm fibers}$ [29]) will rapidly advance, driven by exponentially increasing spatial parallelism across all optical networking applications.

Figure 3 shows the WDM transmission capacity per single-mode fiber (SMF), in both research (red) and products (blue) [1]. Research results are saturating just above 200 Tbps, while commercial products remain well below 100 Tbps. Note that these “maximally achievable WDM capacities” can be misleading due to the fundamental trade-off between transmission capacity and distance. WDM research capacities exceeding 100 Tbps have been limited to distances of ${\sim}{100}\;{\rm km}$ and below, which makes their practical significance questionable in view of Table 1 and confirms the strong saturation of practically meaningful per-fiber WDM capacities. Figure 4 illustrates the rate-reach trade-off in more detail, showing record-SE research results (red) together with Shannon capacity estimates of an SMF with transceiver noise floors of 25 and 30 dB, and reflecting the above discussion on high-speed electrical DAC/ADC noise limits [25,30]. The blue curve in Fig. 4 roughly indicates the state of the art in commercially available coherent transceivers.

 

Fig. 3. Wide-area optical fiber capacity scaling trends (solid: historic data; dotted: announcements/expectations).

Download Full Size | PDF

 

Fig. 4. SE-reach trade-off in fiber communications [25,30].

Download Full Size | PDF

Figure 3 also shows per-wavelength interface rates, redrawn from Fig. 2. The distance between interface rates and WDM capacities represents the number of WDM channels per system (double arrow). Over the past 20 years, leading commercial systems have supported a roughly constant ${\sim}{100}$ channels spanning the system’s optical amplification bandwidth. In terms of scalability, note that modern WDM interfaces spectrally shape signals to occupy a sharply defined frequency band whose width is given by the symbol rate. For a fixed system bandwidth, doubling the per-channel symbol rate halves the number of supported WDM channels, leaving the system capacity constant (cf. channels ${1,\ldots, 8}$ versus ${\rm A,\ldots, D}$ in Fig. 3). This differs from electrical or few-channel (DCI and intra-DC) WDM systems, where per-fiber capacities increase with increasing pre-channel symbol rates. To increase long-haul WDM system capacities, one must take one of the following approaches.

Increase SE by increasing the complexity of the underlying modulation format. Over the past decade, SEs of commercial systems have only improved by ${\sim}{4\times}$ (${\sim}{11}\%$ per year) and should not be expected to increase much further. Long-haul systems face Shannon limits of the underlying fiber channel [30], while shorter-reach systems of up to several hundred kilometers are impacted by transceiver noise floors [25]; cf. Fig. 4.

Reduce system margins by dynamically adapting system bit rates to the respective channel SNR. This promises up to a factor ${\sim}{2\times}$ in additional capacity, with practical implications for network control [31].

Increase system bandwidth beyond the conventional (C) band. Today’s commercial systems already incorporate the longer-wavelength (L) band for a ${2\times}$ capacity gain. Research systems include the shorter-wavelength (S) band. Further expanding system bandwidths promises little returns, even when using advanced hollow-core fiber or quantum techniques at higher carrier frequencies, as detailed in [25,32].

Increase spatial multiplicity through spatially parallel transmission paths (SDM). This is the only long-term sustainable way to grow optical fiber system capacities [33], achieved through massively parallel fiber cables [28], multi-core fibers (MCFs) [34], and few-mode fibers [35,36]. Although many of the Pbps capacities reported on MCFs in Fig. 3 (green) [25] transmit over significantly less than 100 km, multi-thousand kilometer transmission of several 100 Tbps has been demonstrated, clearly extending per-fiber capacities over SMF for a meaningful system reach. However, the techno-economic benefits of MCFs relative to a core equivalent bundle of SMFs have not yet been proven, nor are all operational aspects of a massively increased number of spatial paths fully resolved.

SYSTEM ENERGY LIMITS AND PARALLELISM

While the energy per floating-point operation of general-purpose compute systems shrinks by about ${\sim}{35}\%$ per year [37], high-end switch ASICs with large I/O capacities (such as Ethernet switch chips) are on an ${\sim}{20}\%$ per year energy-per-bit reduction trajectory [38], as are short-reach optical pluggables [39] and long-haul optical communication systems [40]. However, complete switch boxes only show ${\sim}{10}\%$ in energy-per-bit reduction per year [38]. At a switch chip capacity growth rate of ${\sim}{40}\% {\rm /year}$ (cf. Fig. 1), the power density of switch boxes is increasing by ${\sim}{25}\%$ per year. This has resulted in cooling and power feed limitations for data center racks and has led to less densely populated racks and larger switch boxes. The latter is also forced by faceplate densities, which do not scale at 40% per year either. The net result is a larger data center footprint: the “size of the cloud” (as measured in the real estate area occupied by data centers) is growing at ${\sim}{40}\%$ per year, with an estimated size of the U.S. cloud of ${\sim}{10}\;{{\rm km}^2}$ today [1]. This significant increase in data center area is yet another reflection of massive spatial parallelism making up for technology scaling deficiencies. New architectural approaches to improve the energy efficiency (and density) of switch boxes are being investigated, including direct-drive linear pluggable optics (LPO) that halve the optics power by eliminating the optics DSP from the optical pluggable module [41], vertical line card (VLC) architectures that substantially reduce RF trace losses between ASICs and pluggable optics [42], and near-package and co-packaged optics (NPO/CPO) that minimize RF losses between optics and ASICs at the expense of system modularity, flexibility, and serviceability [12–14].

Chip-integrated SerDes show a mere ${\sim}{10}\%$ energy-per-bit reduction per year [5,38] due to the less favorable energy scaling of high-speed analog blocks relative to digital blocks as well as the need for more DSP to enable higher per-lane bit rates. As a result, chip I/O blocks keep consuming an increasingly larger fraction not only of the total chip area but also of the total chip power [5], which drives chipletized architectures with slow-and-parallel die-to-die interfaces and high-speed converter chiplets. This also opens a longer-term possibility to co-optimize SerDes and chipletized CPOs, which is not economical for current chip-integrated SerDes due to the high costs of advanced CMOS tape-outs.

Terrestrial WDM systems are not as energy constrained as other systems, but must be designed with energy efficiency in mind. For example, based on the SE-reach trade-off of Fig. 4, one may design a 2000-km system at an aggregate SE of 16 b/s/Hz either by serially concatenating twenty 100-km systems at 16 b/s/Hz, or by using just two parallel 2000-km systems, each at ${\rm SE}\sim{10}\;{\rm b/s/Hz}$ (for an aggregate 20 b/s/Hz), the latter being much more energy efficient [43]. “Parallel” may mean the use of multiple optical amplification bands (as a short-term stop-gap solution), or it may mean spatially parallel for long-term scalability.

A class of systems that is particularly energy constrained is subsea optical cables spanning reaches of ${\sim}{6000}\;{\rm km}$ (trans-Atlantic) and ${\sim}{11},{\!000}\;{\rm km}$ (trans-Pacific) [24,44]. For such systems, the electrical energy to power the optical in-line amplifiers placed every ${\sim}{50} {-} {100}\;{\rm km}$ within the cable must be fed from the shore ends of the cable, with a maximum feed voltage of ${\sim}{15}\;{\rm kV}$. Given these limitations, it can be shown that capacity is maximized by “diluting” the available electrical power across as many parallel optical paths as economically feasible. [24,25,44]. This design approach has been adopted over the past few years, increasing the number of fiber pairs per cable from typically 2 to 4, to 16 for the most recent cable installations. Cables with 32 fiber pairs as well as cables with MCFs have been announced. Using such massively parallel subsea cables, the per-cable capacity (cf. Fig. 3, orange) is expected to continue to scale and to break the Pbps mark.

CONCLUSION

Communication systems from die-to-die interconnects to ultra-long-haul optical subsea fiber cables are facing severe scalability limitations arising from saturating per-lane bit rates, fiber Shannon capacity limits, energy constraints, and chip and system packaging limitations. The only common solution ensuring scalability is massive spatial parallelism, which is being increasingly embraced across all systems applications.