StarWind, Intel, Mellanox, and Supermicro present the industry's fastest hyperconverged cluster built to demonstrate outstanding hardware utilization efficiency.

Published at LXer: The NanoKVM-PCIe is a recent solution from Sipeed, designed to simplify remote management of ATX PC cases and 2U servers. Built on the RISC-V architecture, it offers low power...

" The mainstream PCIe Gen 3x4 P34A60 mainstream SSD from Silicon Power does offer good all-around performance and does not drop to very slow speeds when the cache fills up completely, as we could notice from the HD Tune Pro write test. When copying large files continuously to the drive, we haven�t seen drops of under 90MB/s, which is great for a TL... [PCSTATS]

When you think of bulk storage, you probably still think about hard drives, and that's fair enough; spinning rust still offers superior value in terms of price per gigabyte. When storage density is more important than value, though, solid-state drives win again. That's because there are SSDs like Micron's new ION 6550, which can store up to

Comments

Read the in depth Review of MSI SPATIUM M480 2TB PCIe 4.0 NVMe M.2 SSD Storage. Know detailed info about MSI SPATIUM M480 2TB PCIe 4.0 NVMe M.2 SSD configuration, design and performance quality along with pros & cons, Digit rating, verdict based on user opinions/feedback.

Read the in depth Review of Lexar NM760 M.2 2280 PCIe Gen4 NVMe SSD Storage. Know detailed info about Lexar NM760 M.2 2280 PCIe Gen4 NVMe SSD configuration, design and performance quality along with pros & cons, Digit rating, verdict based on user opinions/feedback.

PCI-SIG DevCon 2024 – 32^nd Anniversary

For more than a decade, Cadence has been well-known in the industry for its strong commitment and support for PCIe technology. We recognize the importance of ensuring a robust PCIe ecosystem and appreciate the leadership PCI-SIG provides. To honor the 32^nd anniversary of the PCI-SIG Developer’s Conference, Cadence is announcing a complete PCIe 7.0 IP solution for HPC/AI markets.

Why Are Standards Like PCIe So Important?

From the simplest building blocks like GPIOs to the most advanced high-speed interfaces, IP subsystems are the lifeblood of the chipmaking ecosystem. A key enabler for IP has been the collaboration between industry and academia in the creation of standards and protocols for interfaces. PCI-SIG drives some of the key definitions and compliance specifications and ensures the interoperability of interface IP.

HPC/AI markets continue to demand high throughput, low latency, and power efficiency. This is fueling technology advancements, ensuring the sustainability of PCIe technology for generations to come. As a close PCI-SIG member, we gain valuable early insights into the evolving specs and the latest compliance standards. PCIe 7.0 specifications and beyond will enable the market to scale, and we look forward to helping our customers build best-in-class cutting-edge SoCs using Cadence IP solutions.

Figure 1. Evolution of PCIe Data Rates (source PCI-SIG)

What’s New This Year at DevCon?

At DevCon ’24, the PCIe 7.0 standard will take center stage, and Cadence is showing off a full suite of IP subsystem solutions for PCIe 7.0 this year.

What Sets Cadence Apart?

At Cadence, we believe in building a full subsystem for our testchips with eight lanes of PHY along with a full 8-lane controller. Adding a controller to our testchip significantly increases the efficiency and granularity in characterization and stress testing and enables us to demonstrate interoperability with real-world systems. We are also able to test the entire protocol stack as an 8-lane solution that encompasses many of the applications our customers use in practice. This approach significantly reduces the risks in our customers’ SoC designs.

Figure 2: Piper - Cadence PHY IP for PCIe 7.0

Figure 3: Industry’s first IP subsystem for PCIe 7.0

Which Market Is This For?

At a time when accelerated computing has gone mainstream, PCIe links are going to take on a role of higher importance in systems. Direct GPU-to-GPU communication is crucial for scaling out complex computational tasks across multiple graphics processing units (GPUs) or accelerators within servers or computing pods. There is a growing recognition within the industry of a need for scalable, open architecture in high-performance computing. As AI and data-intensive applications evolve, the demand for such technologies will likely increase, positioning PCIe 7.0 as a critical component in the next generation of interface IP.

Here's a recent article describing a potential use case for PCIe 7.0.

Figure 4: Example use case for PCIe 7.0

Why Are Optical Links Important?

It takes multiple buildings of data centers to train AI/ML models today. These buildings are increasingly being distributed across geographies, requiring optical fiber networks that are great at handling the increased bandwidth over long distances. However, these optical modules soon hit a power wall where all the budgeted power is used to drive the signal from point A to point B, and there is not enough power left to run the actual CPUs and GPUs. Such scenarios create a need for non-retimed, linear topologies. Linear Pluggable Optics (LPO) links can significantly reduce module power consumption and latency when compared to traditional Digital Signal Processing (DSP) based retimed optical solutions, which is critical for accelerating AI performance. Swapping from DSP-based solutions to LPO results in significant cost savings that help drive down expenditure due to lower power and cooling requirements, but this requires a robust high-performance ASIC to drive the optics rather than retimers/DSP.

To showcase the robustness of Cadence IP, we have demonstrated that our subsystem testchip board for PCIe 7.0 can successfully transmit and receive 128GT/s signals through a non-retimed opto-electrical link configured in an external loopback mode with multiple orders of margin to spare.

Figure 5: Example of ASIC driving linear optics

Compliance Is Key

For PCIe 6.0, the official compliance program has not started yet; this is typical for the SIG where the official compliance follows a few years after the spec is ratified to give enough time for the ecosystem to have initial products ready, and for test and equipment vendors to get their hardware/software up and running. At this time, PCIe Gen6 implementations can only be officially certified up to PCIe 5.0 level (the highest official compliance test suite that the SIG supports). We have taken our PCIe 6.0 IP subsystem solution to the SIG for multiple process nodes, and they are all listed as compliant. You can run this query on the pcisig.com website under the Developers->Integrators list by making the following selections:

Due to space limitations, not all combinations could be tested at the May workshop (e.g., N3 root port) – this will be tested in the next workshop.

Also, the SIG just held an “FYI” compliance event this week to bring together the ecosystem for confidential testing (no results were reported, and data cannot be shared outside without violating the PCI-SIG NDA). We participated in the event with multiple systems and can report that our systems have done quite well. The test ecosystem is not mature yet, and a few more FYI workshops will be conducted before the official compliance for 6.0 is launched. We have collaborated with all the key test vendors for electrical and protocol testing throughout the year. As early as the middle of last year, we were able to provide test cards to all these vendors to demo PCIe 6.0 capabilities in their booths at various events. Many of them recorded these videos, and they can be found online.

• Cadence Subsystem IP for PCIe 6.0: Protocol and Electrical Testing
• Cadence Subsystem IP for CXL Protocol Test Demo
• Cadence Subsystem IP for CXL2.0/3.0 Protocol Test Demo
• Cadence Subsystem IP for PCIe 6.0: Protocol Stack Demo

More at the PCI-SIG Developers Conference

Check us out at the PCI-SIG Developer’s conference on June 12 and 13 to see the following demonstrations:

• Robust performance of Cadence IP for PCIe 7.0 transmitting and receiving 128GT/s signals over non-retimed optics
• Capabilities of Cadence IP for PCIe 7.0 measured using oscilloscope instrumentation detailing its stable electrical performance and margin
• The reliability of Cadence IP for PCIe 6.0 interface using Test Equipment to characterize the PHY receiver quality
• A PCI-SIG-compliant Cadence IP subsystem for PCIe 6.0 optimized for both power and performance

As a leader in PCI Express, Anish Mathew of Cadence will share his valuable insights on an important topic: “Impact of UIO ECN on PCIe Controller Design and Performance,” highlighting the strides made by the Cadence design team in achieving this implementation.

Figure 6: Cadence UIO Implementation Summary

Summary

Cadence showcased PCIe 7.0-ready IP at PCI-SIG Developers Conference 2023 and continues to lead in PCIe IP development, offering complete solutions in advanced nodes for PCIe 7.0 that will be generally available early next year. With a full suite of solutions encompassing PHYs, Controllers, Software, and Verification IP, Cadence is proud to be a member of the PCI-SIG community and is heavily invested in PCIe. Cadence was the first IP provider to bring complete subsystem solutions for PCIe 3.0, 4.0, 5.0, and 6.0 with industry-leading PPA and we are proud to continue this trend with our latest IP subsystem solution for PCIe 7.0, which sets new benchmarks for power, performance, area, and time to market.

PCI-SIG DevCon 2024 was a great success for Cadence. We posted the blog, Cadence Demonstrates Complete PCIe 7.0 Solution at PCI-SIG DevCon ‘24 a day before the event to advertise our IP solutions for PCIe 7.0, which resulted in a lot of extra traffic at our booth. All of the attendees were excited to see Cadence demonstrate the robustness of 128GT/s PCIe 7.0 IP's TX and RX capabilities over a real-world, low-latency, non-retimed, linear optics connector. We achieved and maintained a consistent, impressive pre-FEC BER of ~3E-8 (PCIe spec requires 1E-6) for the entire duration of the event, spanning over two full days with no breaks. This provides an ample margin for RS FEC. As seen in the picture below, the receiver Eye PAM4 histograms have good linearity and margin. This is the world’s first stable demonstration of 128 GT/s TX and RX over off-the-shelf optical connectors—by far the main attraction of DevCon this year.

Cadence 128 GT/s TX and RX capability over optics

Block diagram of Cadence PHY for PCIe 7.0 128 GT/s demo setup with linear pluggable optics

As a leader in PCIe, our PCIe controller architect Anish Mathew shared his valuable insights on an important topic: “Impact of UIO ECN on PCIe Controller Design and Performance,” highlighting the strides made by the Cadence design team in achieving this implementation.

Anish Mathew presenting “Impact of UIO ECN on PCIe Controller Design and Performance”

In summary, Cadence had a dominating presence on the demo floor with a record number of PCIe demos:

• PCIe 7.0 over optics
• PCIe 7.0 electrical
• PCIe 6.0 RP/EP interop back-to back
• PCIe 6.0 protocol in FLIT mode with Lecroy Exerciser (at Cadence booth)
• PCIe 6.0 protocol in FLIT mode (at the Lecroy booth)
• PCIe 6.0 JTOL with Anritsu and Tektronix equipment (at Tektronix booth)
• PCIe 6.0 protocol with Viavi Protocol Analyzer (at Viavi booth)
• PCIe 6.0 System Level Interop Demo with Gen5 platform (at SerialTek booth)

The Cadence team and its partners did a great job in coordinating and setting up the demos that worked flawlessly. This was the culmination of many weeks of hard work and dedication. Four different vendors featured our IP for PCIe 6.0. They attracted a lot of attention and drove traffic back to us.

Highlights of Cadence demos for PCIe 7.0 and 6.0

Cadence team at the PCI-SIG Developers Conference 2024

Thanks to everyone who attended the 32nd PCI-SIG DevCon. We really appreciate your interest in Cadence IP, and a big thanks to our partners and customers for all the positive feedback and for creating so much buzz for the Cadence brand.

As technology continues to evolve at a rapid pace, the importance of robust and efficient interconnect standards cannot be overstated. Peripheral Component Interconnect Express (PCIe) has been a cornerstone in high-speed data transfer, enabling seamless communication between various hardware components.   

With the advent of PCIe 6.1 ECN, a significant advancement in speed and efficiency, ensuring the accuracy and reliability of its operations is paramount. One critical aspect of this is the verification of shared credit updates. For detailed understanding on Shared Credit, please refer Understanding PCIe 6.0 Shared Flow Control. 

In this blog, we will discuss why this verification is essential and what it entails.  

Introduction 

PCIe 6.1 ECN brings numerous advancements over earlier versions, such as increased bandwidth and faster data transfer speeds.   

A crucial mechanism for efficient data transmission in PCIe 6.0 is the credit-based flow control system. In this system, devices monitor credits, representing the buffer capacity available for incoming data.   

When a device transmits data, it uses credits, which are replenished or adjusted once the data is received and processed. This system ensures that the sender does not overload the receiver.  

Given the critical role of shared credit updates in maintaining the integrity and efficiency of data transfers, verification of these updates is crucial.  Proper management of credit updates is essential to ensure data integrity, as any discrepancies can lead to data loss, corruption, or system crashes.   

Verification also guarantees efficient resource allocation, preventing scenarios where some components are starved of credit while others have an excess, thus avoiding inefficiencies.  Credit inefficiencies pose issues in low power negotiations by preventing devices from entering low power states. Additionally, verification involves checking for proper error handling mechanisms, ensuring that the system can recover gracefully from errors in credit updates and maintain overall stability.   

PCIe 6.1 ECN Flow Control Optimizations Over PCIe 6.0

PCIe 6.1 ECN builds on the FLIT-based architecture introduced in PCIe 6.0, further optimizing flow control mechanisms to handle increased data rates and improved efficiency.  PCIe 6.1 ECN introduced refinements in credit management, making the allocation and advertisement of credits more precise, which helps in reducing bottlenecks and improving data flow efficiency.  Enhancements in flow control protocols ensure better management of buffer spaces and more efficient credit allocation. These enhancements are designed to handle the increased data rates and throughput demands of next-generation applications, ensuring robust and efficient data flow across PCIe devices.  

Below are some major updates: 

1. There have been improvements in error detection and correction mechanisms in PCIe 6.1 ECN to enhance flow control reliability by ensuring that corrupted data packets are detected and handled appropriately without disrupting the flow of valid packets.  
2. The merged credit system, which was a key feature introduced int PCIe 6.0 to simplify and optimize credit management, was further enhanced in PCIe 6.1 ECN to improve performance and efficiency.  
3. PCIe 6.1 ECN introduced better algorithms for allocating and reclaiming merged credits to handle high data rates, introduced more robust error detection and correction mechanism reducing the degradation or system instability. 
4. PCIe 6.1 ECN provided clear guidelines on how to implement the merged credit system correctly, helping developers to implement more reliable systems. For more details, please refer to Specifications section 2.6.1 Flow Control (FC) Rules.

Summary 

In summary, PCIe 6.0 is a complex protocol with many verification challenges. You must understand many new Spec changes and think about the robust verification plan for the new features and backward compatible tests impacted by new features. Cadence’s PCIe 6.0 Verification IP is fully compliant with the latest PCIe Express 6.0 specifications and provides an effective and efficient way to verify the components interfacing with the PCIe 6.0 interface. Cadence VIP for PCIe 6.0 provides exhaustive verification of PCIe-based IP and SoCs, and we are working with early adopter customers to speed up every verification stage.   

More Information

For more info on how Cadence PCIe Verification IP and Triple Check VIP enable users to confidently verify PCIe 6.0, see VIP for PCI Express, VIP for Compute Express Link  and TripleCheck for PCI Express    

See the PCI-SIG website for more details on PCIe in general and the different PCI standards.  

For more information on PCIe 6.0 new features, please visit PCIeLaneMargin, PCIe6.0LaneMargin, and Demonstrating PCIe 6.0 Equalization Procedure.

Cadence PCIe/CXL VIP support for Partial Header Encryption in Integrity and Data Encryption.(read more)

Peripheral Component Interconnect Express (PCIe) is a high-speed interface standard widely used for connecting processors, memory, and peripherals. With the increasing reliance on PCIe to handle sensitive data and critical high-speed data transfer, ensuring data integrity and encryption during verification is the most essential goal. As we know, in the field of verification, randomization is a key technique that drives robust PCIe verification. It introduces unpredictability to simulate real-world conditions and uncover hidden bugs from the design. This blog examines the significance of randomization in PCIe IDE verification, focusing on how it ensures data integrity and encryption reliability, while also highlighting the unique challenges it presents. For more relevant details and understanding on PCIe IDE you can refer to Introducing PCIe's Integrity and Data Encryption Feature . The Importance of Data Integrity and Data Encryption in PCIe Devices Data Integrity : Ensures that the transmitted data arrives unchanged from source to destination. Even minor corruption in data packets can compromise system reliability, making integrity a critical aspect of PCIe verification. Data Encryption : Protects sensitive data from unauthorized access during transmission. Encryption in PCIe follows a standard to secure information while operating at high speeds. Maintaining both data integrity and data encryption at PCIe’s high-speed data transfer rate of 64GT/s in PCIe 6.0 and 128GT/s in PCIe 7.0 is essential for all end point devices. However, validating these mechanisms requires comprehensive testing and verification methodologies, which is where randomization plays a very crucial role. You can refer to Why IDE Security Technology for PCIe and CXL? for more details on this. Randomization in PCIe Verification Randomization refers to the generation of test scenarios with unpredictable inputs and conditions to expose corner cases. In PCIe verification, this technique helps us to ensure that all possible behaviors are tested, including rare or unexpected situations that could cause data corruption or encryption failures that may cause serious hindrances later. So, for PCIe IDE verification, we are considering the randomization that helps us verify behavior more efficiently. Randomization for Data Integrity Verification Here are some approaches of randomized verifications that mimic real-world traffic conditions, uncovering subtle integrity issues that might not surface in normal verification methods. 1. Randomized Packet Injection: This technique randomized data packets and injected into the communication stream between devices. Here we Inject random, malformed, or out-of-sequence packets into the PCIe link and mix valid and invalid IDE-encrypted packets to check the system’s ability to detect and reject unauthorized or invalid packets. Checking if encryption/decryption occurs correctly across packets. On verifying, we check if the system logs proper errors or alerts when encountering invalid packets. It ensures coverage of different data paths and robust protocol check. This technique helps assess the resilience of the IDE feature in PCIe in below terms: (i) Data corruption: Detecting if the system can maintain data integrity. (ii) Encryption failures: Testing the robustness of the encryption under random data injection. (iii) Packet ordering errors: Ensuring reordering does not affect data delivery. 2. Random Errors and Fault Injection: It involves simulating random bit flips, PCRC errors, or protocol violations to help validate the robustness of error detection and correction mechanisms of PCIe. These techniques help assess how well the PCIe IDE implementation: (i) Detects and responds to unexpected errors. (ii) Maintains secure communication under stress. (iii) Follows the PCIe error recovery and reporting mechanisms (AER – Advanced Error Reporting). (iv) Ensures encryption and decryption states stay synchronized across endpoints. 3. Traffic Pattern Randomization: Randomizing the sequence, size, and timing of data packets helps test how the device maintains data integrity under heavy, unpredictable traffic loads. Randomization for Data Encryption Verification Encryption adds complexity to verification, as encrypted data streams are not readable for traditional checks. Randomization becomes essential to test how encryption behaves under different scenarios. Randomization in data encryption verification ensures that vulnerabilities, such as key reuse or predictable patterns, are identified and mitigated. 1. Random Encryption Keys and Payloads: Randomly varying keys and payloads help validate the correctness of encryption without hardcoding assumptions. This ensures that encryption logic behaves correctly across all possible inputs. 2. Randomized Initialization Vectors (IVs): Many encryption protocols require a unique IV for each transaction. Randomized IVs ensure that encryption does not repeat patterns. To understand the IDE Key management flow, we can follow the below diagram that illustrates a detailed example key programming flow using the IDE_KM protocol. Figure 1: IDE_KM Example As Figure 1 shows, the functionality of the IDE_KM protocol involves Start of IDE_KM Session, Device Capability Discovery, Key Request from the Host, Key Programming to PCIe Device, and Key Acknowledgment. First, the Host starts the IDE_KM session by detecting the presence of the PCIe devices; if the device supports the IDE protocol, the system continues with the key programming process. Then a query occurs to discover the device’s encryption capabilities; it ensures whether the device supports dynamic key updates or static keys. Then the host sends a request to the Key Management Entity to obtain a key suitable for the devices. Once the key is obtained, the host programs the key into the IDE Controller on the PCIe endpoint. Both the host and the device now share the same key to encrypt and authenticate traffic. The device acknowledges that it has received and successfully installed the encryption key and the acknowledgment message is sent back to the host. Once both the host and the PCIe endpoint are configured with the key, a secure communication channel is established. From this point, all data transmitted over the PCIe link is encrypted to maintain confidentiality and integrity. IDE_KM plays a crucial role in distributing keys in a secure manner and maintaining encryption and integrity for PCIe transactions. This key programming flow ensures that a secure communication channel is established between the host and the PCIe device. Hence, the Randomized key approach ensures that the encryption does not repeat patterns. 3. Randomization PHE: Partial Header Encryption (PHE) is an additional mechanism added to Integrity and Data Encryption (IDE) in PCIe 6.0. PHE validation using a variety of traffic; incorporating randomization in APIs provided for validating PHE feature can add more robust Encryption to the data. Partial Header Encryption in Integrity and Data Encryption for PCIe has more detailed information on this. Figure 2: High-Level Flow for Partial Header Encryption 4. Randomization on IDE Address Association Register values: IDE Address Association Register 1/2/3 are supposed to be configured considering the memory address range of IDE partner ports. The fields of IDE address registers are split multiple values such as Memory Base Lower, Memory Limit Lower, Memory Base Upper, and Memory Limit Upper. IDE implementation can have multiple register blocks considering addresses with 32 or 64, different registers sizes, 0-255 selective streams, 0-15 address blocks, etc. This Randomization verification can help verify all the corner cases. Please refer to Figure 2. Figure 3: IDE Address Association Register 5. Random Faults During Encryption: Injecting random faults (e.g., dropped packets or timing mismatches) ensures the system can handle disruptions and prevent data leakage. Challenges of IDE Randomization and its Solution Randomization introduces a vast number of scenarios, making it computationally intensive to simulate every possibility. Constrained randomization limits random inputs to valid ranges while still covering edge cases. Again, using coverage-driven verification to ensure critical scenarios are tested without excessive redundancy. Verifying encrypted data with random inputs increases complexity. Encryption masks data, making it hard to verify outputs without compromising security. Here we can implement various IDE checks on the IDE callback to analyze encrypted traffic without decrypting it. Randomization can trigger unexpected failures, which are often difficult to reproduce. By using seed-based randomization, a specific seed generates a repeatable random sequence. This helps in reproducing and analyzing the behavior more precisely. Conclusion Randomization is a powerful technique in PCIe verification, ensuring robust validation of both data integrity and data encryption. It helps us to uncover subtle bugs and edge cases that a non-randomized testing might miss. In Cadence PCIe VIP, we support full-fledged IDE Verification with rigorous randomized verification that ensures data integrity. Robust and reliable encryption mechanisms ensure secure and efficient data communication. However, randomization also brings various challenges, and to overcome them we adopt a combination of constrained randomization, seed-based testing, and coverage-driven verification. As PCIe continues to evolve with higher speeds and focuses on high security demands, our Cadence PCIe VIP ensures it is in line with industry demand and verify high-performance systems that safeguard data in real-world environments with excellence. For more information, you can refer to Verification of Integrity and Data Encryption(IDE) for PCIe Devices and Industry's First Adopted VIP for PCIe 7.0 . More Information: For more info on how Cadence PCIe Verification IP and TripleCheck VIP enables users to confidently verify IDE, see our VIP for PCI Express , VIP for Compute Express Link for and TripleCheck for PCI Express For more information on PCIe in general, and on the various PCI standards, see the PCI-SIG website .

Each PCIe device may include a media access control (MAC) interface and a physical (PHY) interface that support a plurality of different lane configurations. These interfaces may include hardware modules that support 1×32, 2×16, 4×8, 8×4, 16×2, and 32×1 communication. Instead of physically connecting each of the hardware modules in the MAC interface to respective hardware modules in the PHY interface using dedicated traces, the device may include two bus controllers that arbitrate which hardware modules are connected to a internal bus coupling the two interfaces. When a different lane configuration is desired, the bus controller couples the corresponding hardware module to the internal bus. In this manner, the different lane configurations share the same lanes (and wires) of the bus as the other lane configurations. Accordingly, the shared bus only needs to include enough lanes (and wires) necessary to accommodate the widest lane configuration.

PCIe has been widely adopted in the electronics industry since its first debut in 2003 (PCIe 1.0 standard release) for wide breach of applications, from Data Center Server, Networking, to Mobile, AI/ML, Automotive, IoT, and many others…. It’s a versatile, high-performance, robust, mature interconnect standard with full “backward compatibility” (e.g., a PCIe 3.0 device can still function well in a PCIe 4.0 system) which enables a solid and strong PCIe eco-system in the industry.  While the market, so as the users,  are enjoying the systems, e.g., desktop/laptop, powered (or to be more specific: “bridged”) by PCIe 3.0 since 2010, the industry is pushing hard for the PCIe 4.0 eco-system enablement. Earlier this year, AMD announced it X570 chipset would support the PCIe 4.0 interface and Phison also introduced the world’s first PCIe 4.0 SSD.

On the standard evolution front, the official PCIe 5.0 came out in May 2019, doubling the data rate to 32GT/s from 16GT/s in PCIe 4.0. The PCIe 6.0 standard will be released in 2021 based on the announcement made by PCI-SIG in June’19 with the goal to further double the data rate to 64GT/s with incorporating the PAM4 coding.

PCIe Protocol Evolution

Having said that, is the latest generation of PCIe always desired?  

My answer would be positive. Just like car maker/enthusiast has kept pursuing faster car in the history, there is no doubt that these speed enhancements/upgrades in the electronic world certainly provide a tremendous benefit for especially those applications craving the most throughput, such as Data center, HPC, Networking, Cloud and AI applications.   

But, does every application have to opt for the fastest speed (bandwidth)? My view would be leaning toward “Not really”. Just like we don’t need a 3-second sport car (meaning 0-60mph acceleration < 3s) for daily commute though it would certainly spice some driving fun on the road, but it may not be "the best fit" for most of commuters.

There are applications still well satisfied with PCIe 3.0 (or even older PCIe 2.0) for its best performance and cost balance.  Those applications include, but not limit to, IoT/consumer, Edge AI, SSD (non-enterprise),…etc. They typically need to make trade-off in between the cost, power consumption (especially battery powered), flexibility on changing product features, and time-to-market (TTM). To address such type of market needs, Cadence also offers an PPA (Performance, Power, Area) optimized PCIe 3.0 solution in addition to its high-performance PCIe 4.0 product line.

Cadence PCIe 3.0 PHY Solution (with Multi-Protocol Multi-Link feature)

With leveraging the multi-protocol SerDes implementation, the same Cadence PHY IP support multi-protocol and multi-link operation. Such a multi-protocol enabled PHY gives the SoC developers the optimum flexibility to integrate multiple commonly used interface protocols (e.g., PCIe 3.0 + USB 3.0) with using only a single PHY design.  This would largely save the product development time (faster TTM), reduce the risk of using multiple different PHY instances (for different protocol needs), and with the configurability to enable different product features/protocols.

Some people might say PCIe 3.0 era has gone. I was not quite yet being convinced as I still see its potential to shine a lot of market use cases. What do you think?

More Information

For more information on Cadence's PCIe IP offerings, see our PCI Express page.

For more information on PCIe in general, and on the various PCI standards, see the PCI-SIG website.

Related Posts

• Blog: PCIe Gen4: It’s Official, We’re Compliant
• Blog: The PCIe 4.0 Era Continues at PCI-SIG Developers Conference 2016
• Blog: Cadence PCIe Solutions: Configurable, Compliant, and Low Power

The PCI-SIG finalized the PCIe 4.0 specification with doubling the data to 16GT/s from 8GT/s in PCIe 3.0 in 2017. Products implementing this technology have begun to hit the market in 2019. Earlier this year, AMD announced it X570 chipset would support the PCIe 4.0 interface and Phison also introduced the world’s first PCIe 4.0 SSD.  With the increasing companies are working on PCIe 4.0 related product development, Cadence, as the key and leading PCIe IP solution vendor in the market, has strived for continuous enhancement of its PCIe 4.0 to be the best in the class IP solution. From our initial PCIe 4.0 solution in 4 years ago (revealed in 2015), we have made many advancements and improvements adding features such as Multi-link with any lane assignment, U.2/U.3 connector, and Automotive support. The variety of applications that PCIe4 finds a home in require extensive robustness and reliability testing over and above the compliance tests mandated by the standard body, i.e., PCI-SIG.

PCIe 4.0 TX Eye-Diagram, Loop-back Test (Long-reach) and RX JTOL Margin Test

Cadence IP team has also implemented additional "stress tests" in conjunction to its already comprehensive IP characterization plan, covering electrical, functional, ESD, Latch-up, HTOL, and yield sorting. Take the Receiver Jitter Tolerance Test (JTOL) for instance. JTOL is a key index to test the quality of the receiver of a system. This test use data generator/analyzer to send data to a SerDes receiver which is then looped back through the transmitter back to the instrument. The data received is compared to the data generated and the errors are counted. The data generator introduce jitter into the transmit data pattern to see how well the receiver functions under non-ideal conditions. While PCI-SIG compliance can be obtained on a single lane implementation, real world scenarios require wider implementations under atypical operating conditions. Cadence’s PCIe 4.0 IP was tested against to additional stressed conditions, such as different combination of multi-lanes operations, “temperature drift” conditions, e.g., bring up the chip at room temperature and check the JTOL at high temperature. 

PCIe 4.0 Sub-system Stress Test Setup

Besides complying with electrical parameters and protocol requirements, real world systems have idiosyncrasies of their own. Cadence IP team also built a versatile “System test” setup in house to perform a system level stress test of its PCIe 4.0 sub-system. The Cadence PCIe 4.0 sub-system is connected to a large number of server and desktop motherboards. This set up is tested with 1000s of cycles of repeated stress under varying operating conditions. Stress tests include speed change from 2.5G all the way to 16G and down, link enable/disable, cold boot, warm boot, entering and exiting low power states, and BER test sweeping presets across different channels. Performing this level of stress test verifies that our IP will operate to spec robustly and reliably when presented with the occasional corner cases in the real world.

More Information

For the demonstration of Cadence PCIe4 PHY Receiver Test and Sub-system Stress Test, see the video:

• PCIe 4.0 Sub-system Stress Test
• PCIe 4.0 PHY Receiver JTOL Test

For more information on Cadence's PCIe IP offerings, see our PCI Express page.

For more information on PCIe in general, and on the various PCI standards, see the PCI-SIG website.

Related Posts

• Blog: PCIe Gen4: It’s Official, We’re Compliant
• Blog: PCIe 3.0 Still Shines While PCIe Keeps Evolving
• Blog: The PCIe 4.0 Era Continues at PCI-SIG Developers Conference 2016

USB4 can simultaneously tunnel USB3, PCIe and DisplayPort native protocol traffic through a hierarchy of USB4 routers. The key to tunneling of these protocols is routing table programmed at each ingress adapter. An entry of a routing table maps an incoming HopID, called Input/Ingress HopID to a corresponding pair of Output/Egress Adapter and Egress/Output HopID.

The responsibility of programming routing tables lies with the Connection Manager. Connection Manager, having the complete view of the hierarchy of the routers, programs the routing tables at all relevant adapter ports. Accordingly, the USB3, PCIe and DisplayPort protocol tunneled packets are routed, and reach their respective intended destinations.

The diagrammatic representation below is an example of tunneling of USB3 protocol traffic from USB4 Host Router to USB4 Peripheral Device Router through a USB4 Hub Router. The path from USB3 Host to USB3 Device is depicted by routing tables indicated at A -> B -> C -> D, and the one from USB3 Device to USB3 Host by routing tables indicated at E -> F -> G -> H . Note that the Input HopID from and Output HopID to all three protocol adapters for USB3, PCIe and DisplayPort Aux traffic, are fixed as 8, and for DisplayPort Main Link traffic are fixed as 9.

Once the native protocol traffic come into the transport layer of a USB4 router, the transport layer of it does not know to which native protocol a tunneled packet belongs to. The only way a transport layer tunneled packet is routed through the hierarchy of the routers is using the HopID values and the information programmed in the routing tables.

The figure below shows an example of tunneling of all the three USB3, PCIe and DisplayPort protocol traffic together. The transport layer tunneled packets of each of these native protocols are transported simultaneously through the routers hierarchy.

 Cadence has a mature Verification IP solution for the verification of USB3, PCIe and DisplayPort tunneling. This solution also employs the industry proven VIPs of each of these native protocols for native USB3, PCIe and DisplayPort traffic.

Learn about the challenges and solutions for integrating and verification PCIe(r) Gen4 into an Arm-Based Server SoC. Listen to this relatively short webinar by Arm and Cadence, as they describe the collaboration and results, including methodology and...(read more)