DAC Cable Differences in InfiniBand and Ethernet

      In the specification of DAC, it is usually labeled that it can be used in InfiniBand or Ethernet, and it is impossible to distinguish the difference from the appearance of DAC, because the package and material are the same. Therefore, usually wonder whether the two can be mixed, here is a brief introduction to the differences.
1. Application environment
      InfiniBand and Ethernet are two different sets of network system protocols, which also determines the application of DAC in different hardware and software environments, the differences in these environments, is the root of the different performance requirements of the DAC.
      Ethernet related standards is a reference to what we know as IEEE 802.3, is developed by the Institute of Electrical and Electronics Engineers (IEEE: Institute of Electrical and Electronics Engineers), InfiniBand related standards need to refer to the InfiniBand Architecture Specification, developed by the InfiniBand Trade Association (IBTA: The InfiniBand Trade Association).
2. Speed
      Taking a typical QSFP DAC 4-channel product as an example, the current rates of InfiniBand are:
       • SDR (Single Data Rate): Single data rate, i.e. 8Gb/s
       • DDR (Double Data Rate): Double data rate, i.e. 16Gb/s
       • QDR (Quad Data Rate): Quad Data Rate, i.e. 32Gb/s
       • FDR (Fourteen Data Rate): Fourteen Data Rate, 56Gb/s
       • EDR (Enhanced Data Rate): 100 Gb/s
       • HDR (High Data Rate): 200 Gb/s
      Ethernet is currently available at the following major rates:
       • SFP: 1G Ethernet, i.e. 1Gb/s
       • SFP+: 10G Ethernet, i.e. 10Gb/s
       • SFP28: 25G Ethernet, i.e. 25Gb/s
       • QSFP: 40G Ethernet, which is 40Gb/s
       • QSFP28: 100G Ethernet, 100 Gb/s
       • QSFP28-DD: 200G Ethernet, 200 Gb/s
3. Insertion Loss
      Taking 100G Ethernet as an example, its insertion loss reference standard is subsection 92.10.2 of Chapter 92 in IEEE 802.3-2022, which requires the insertion loss to be less than 22.48 dB at the 12.89 GHz frequency point, as shown in Figure 1 below.

Figure 1 100G Ethernet insertion loss standard

      The InfiniBand EDR rate is also 100G, and its insertion loss reference standard is subsection 6.8.8 in the InfiniBand Architecture Specification, which requires an insertion loss of less than 16.74dB at the 12.89 GHz frequency point, as shown in Figure 2 below.

Figure 2 InfiniBand EDR insertion loss standard

      From the above comparison, it can be seen that the test method and core frequency point of InfiniBand EDR insertion loss are the same as that of 100G Ethernet, but the difference is that the insertion loss requirement of InfiniBand EDR is much smaller than that of 100G Ethernet, i.e., the performance requirement of DAC is more stringent.
4. CDR (Clock and Data Retiming) Default State
      CDR is bypassed or disabled in InfiniBand host devices, which ensures rate backward compatibility, for example, EDR devices can also be used as FDR.
      CDR is on by default for Ethernet 100G and must be disabled first if you want to use it for lower rates.

5. BER
      IEEE defines a BER of 1E-5 for Ethernet, which is corrected to 1E-12 by the FEC in the host, i.e., the Ethernet BER requirement is 1E-12.
      InfiniBand, on the other hand, assumes that no FEC is used in the host and requires a 1E-15 BER. The requirements are much higher than Ethernet.
6. Summary
      From the above comparison, it can be seen that InfiniBand has higher requirements for DACs than Ethernet, so the performance of DACs for InfiniBand is higher than that of Ethernet products with the same rate.
7. Moduletek Models
      Moduletek Limited can provide 56G FDR and 100G EDR products for Infiniband standard with the following models.

Table 3 Infiniband DAC Products

      If you have any questions about the above content, you can contact us by Email : web@moduletek.com