* following connectes are gathered from JEDEC spec and Internet for the personal usage only
1. Scope
reference specification for system and device management capabilities
3. Terms and Definitions
| AIC | Add-in card. Generally accepted as a PCIe CEM adapter. |
| BIOS | Basic Input/Out System, modern usage synonymous with ‘UEFI’ |
| BMC | Baseboard Management Controller |
| CAMM | Compression Attached Memory Module |
| CCI | Component Command Interface (CXL) |
| CMA | Component Measurement and Authentication (PCI-SIG) |
| CMC | JEDEC Memory Controller for Compute Express Link® |
| CMF | JEDEC Memory FRU for Compute Express Link®. Generally accepted as a field replaceable memory device supporting the CXL protocol. Encompasses CMMs, AICs, and modularized trays form factors. See Figure 1. |
| CMM | JEDEC Memory Module for Compute Express Link®. Generally accepted as a modularized memory device supporting the CXL protocol in an enclosed form factor (such as U.2, EDSFF, etc.) with embedded memory media components. See Figure 1. |
| Config | Configuration |
| CXL | Compute Express Link® |
| CXL Memory Device (CMD) | Any CXL device advertising itself to the host as a ‘CXL Memory Device’ on the PCIe bus (as described in the ‘Memory Device Configuration Space Layout’ section of the CXL Specification). Generally accepted as any CXL Type 3 device (independent of form factor) and may include the controller, media, and other components. See Figure 1. |
| DDR | Double Data Rate |
| DIMM | Dual Inline Memory Module |
| DMTF | Distributed Management Task Force |
| DRAM | Dynamic Random Access Memory |
| DOE | Data Object Exchange (PCI-SIG) |
| E1 | A family of slender form factors within EDSFF (SNIA) |
| E3 | A family of form factors within EDSFF (SNIA) |
| ECDSA | Elliptic Curve Digital Signature Algorithm |
| ECN | Engineering Change Notice |
| EDSFF | Enterprise and Datacenter Standard Form Factor (SNIA) |
| FRU | Field Replaceable Unit |
| FRU Information Device | Device which stores vital product data |
| FW | Firmware |
| IDE | Integrity and Data Encryption |
| IPMI | Intelligent Platform Management Interface |
| MCTP | Management Component Transport Protocol (DMTF) |
| Memory Media FRU | Field replaceable memory module without a CXL memory controller (e.g., DIMM or CAMM) |
| MMIO | Memory-mapped Input/Output |
| OOB | Out of Band |
| OCP | Open Compute Project |
| OS | Operating System |
| PCIe | PCI Express (PCI-SIG) |
| PLDM | Platform Level Data Model (DMTF) |
| RSA | Rivest–Shamir–Adleman public-key cryptosystem |
| SHA | Security Hash Algorithm |
| SNIA | Storage Networking Industry Association |
| SPD | Serial Presence Detect, modern usage referring the device on DIMM which stores DIMM vital product data and may perform other functions such as temperature sensing (JEDEC) |
| Temp | Temperature |
| TS | Temperature Sensor |
| SPDM | Security Protocol and Data Model (DMTF) |
| UEFI | Unified Extensible Firmware Interface, modern usage synonymous with ‘BIOS’ (UEFI Forum) |
| VDM | Vendor Defined Message (PCI-SIG) |
| VPD | Vital Product Data |
🔸 E1 폼팩터
- E1.S: M.2보다 향상된 열 분산과 전력 전달을 제공하며, 1U 섀시에서 고밀도 스토리지를 지원합니다. 키오시아코리아 주식회사+1Velog+1
- E1.L: E1.S보다 긴 형태로, 더 많은 용량을 제공하며 블레이드 서버에 적합합니다.
🔸 E3 폼팩터
- E3.S: 2.5인치 드라이브와 유사한 크기로, 최대 16개의 PCIe 레인을 지원하며 고성능 서버에 적합합니다.
- E3.L: E3.S보다 길며, 더 높은 용량과 전력을 제공하여 대용량 저장이 필요한 응용 프로그램에 사용됩니다.
🔸 EXL DIMM Expansion AIC (Add-In Card)
표준 PCIe 슬롯에 장착되는 카드 형태로, 여러 개의 DDR5 DIMM을 탑재하여 메모리 용량을 확장합니다. SMART Modular's 8-DIMM Add-In Card - CXA-8F2W
🔸 CXL DIMM Expansion E5
EDSFF E3.S 폼팩터를 기반으로 한 CXL 메모리 모듈로, 고성능 서버에 적합합니다.
🔸 CXL Fixed In-System
서버 내부에 고정 설치되는 형태의 CXL 메모리 모듈로, 시스템의 메모리 용량을 확장합니다.
🔸 CXL Memory Appliance
외부 확장 박스 형태로, 다수의 DIMM 슬롯을 통해 대규모 메모리 확장을 지원합니다.
4. Management Interface Requirements
4.1 Management Physical Layer Requirements
4.1.1 Two-wire serial bus Interface Requirements
sections of SNIA SFF-TA-1009 Enterprise and Datacenter Standard Pin and Signal Specification Rev 3.1
SMBus/I2C
Section 5.3.2 – SMBus Interface
Section 6.2 – Timings
Section 6.3 – 3.3 V Logical Signal Requirements
Optionally support I3C Basic
Section 5.3.3 – I3C Basic Interface
Section 6.2 – Timings
Section 6.4 – I3C Basic Signal Requirements
Section 10 – I3C Basic Implementation
FRU Information Endpoint in the PCI-SIG Architectural Out of Band Management ECN (section 12.6.1), VPD in section 8
I3C를 지원하는 CXL memory devices는 dynamic address를 할당 받기 전까지는 위에 정의된 고정 I2C address를 사용함
4.1.2 Host bus interface (PCIe)
out-of-band mechanism : a management endpoint via PCIe Vendor Defined MEssages(VDMs) and MCTP over PCIe VDM by MCTP PCIe VDM Transport Binding Specification
In-band mechanism, CXL mailboxes via PCIe configuration space and MMIO by the CXL Specification. Optionally implement a Data Object Exchange (DOE) amilbox via PCIe configuration space and extended capability registers
4.2 Management Message Transport Requirements
MCTP and the CXL mailbox
optional DOE for SPDM over Component Measurement and Authentication(CMA)
4.2.1 Management Component Transport Protocol (MCTP)
DMTP specifications
- Base MCTP - MCTP Base specification , Revision 1.3.1. (DSP0236)
- MCTP over PCIe VDM - MCTP PCIe VDM Transport Binding Specification, Revision 1.2.0 (DSP0238)
- MCTP over SMBus/I2C - DMTF MCTP SMBus/I2C VDM Transport Binding Specification, Revision 1.3.0 (DSP0237)
- MCTP over I3C (optional) - MCTP I3C Transport Binding Specification, Revision 1.0.0 (DSP0233)
4.2.2 CXL Mailboxes
primay and secondary CXL mailboxes
CXL specification only requires a primary mailbox
4.2.3 Data Object Exchange (DOE) Mailboxes
optional DOE for SPDM over Component Measurement an Authentication (CMA) as described in the PCI-SIG Data Boject Exchange(DOE) ENC, march 26, 2020)
4.3 Management Protocol Layer Requirements
PLDM, CCI, and SPDM
4.3.1 Platform Level Data Model (PLDM)
- PLDM Base Specification, Revision 1.1.0 (DSP0240)
- PLDM over MCTP Binding Specification, Revision 1.0.0 (DSP0241)
supporting PLDM message types
- type 2 - PLDM for Platform Monitoring and Control
- type 5 - PLDM for FIrmware Update in DMTF PLDM for Firmware Update Specification, Revision 1.2 (DSP0267)
- type 6 (optional) - PLDM for Redfish Device Enablement in DMTF Platform Level Data Model (PLDM) for Redfish Device Enablement Specification, Revision 1.1.2 (DSP0218)
4.3.2 CXL Component Command Interface (CCI)
Primary and Secondary CXL Mailboxes in CXL Spec
MCTP in DMTF CXL Type 3 Device Component Command Interface over MCTP Binding Specification, Revision 1.0.0 (DSP0281)
4.3.3. Security Protocol and Data Model (SPDM)
section 5.5.
4.4. Firmware Update Requirements
4.4.1 Firmware Update Interface Requirements
- CCI Firmware Update over Primary CXL Mailbox
- CCI Firmware Update over MCTP over PCIe VDM
- CCI Firmware Update over MCTP over 2-Wire Interfaces
- PLDM Type 5 Firmware Update over MCTP over PCIe VDM
- PLDM Type 5 Firmware Update over MCTP over 2-Wire Interfaces
4.4.2 Firmware Update Security Requirements
section 5.3
5. Security
5.1. Signature and Hashing Algorithms
for Secure Boot, Secure FW Update, and SPDM implementations
ECDSA P384 or other asymmetric key signature algorithms (at least 192-bits of security strength in NIST SP 800-57 Part 1 Revision 5(Table 2). ma also spuport RSA 3072
Supporting multiple signature algorithms is not required.
SHA2-384, SHA3-384, or at least 192 bits security strength in NIST SP 800-57 Part 1 Revision 5.
5.2 Secure Boot
by the OCP “Hardware Secure Boot” specification reivsion 1.0, 2018
does not require implementations of the separate OCP “Recovery” and “Attenstation of System Components”
5.3 Secure Firmware Update
security guidelines of NIST SP800-193
5.4 CXL Integrity and Data Encryption (IDE)
section 11. of the CXL 2.0 specification
5.5 Security Protocol and Data Moel (SPDM)
SPDM messages and object exchanges (MCTP message type 5 and optional type 6)
- Security Protocol and Data Model (SPDM) Specification, Revision 1.2.0 (DSP0274)
- Security Protocol and Data Model (SPDM) over MCTP Binding Specification, Revision 1.0.1 (DSP0275)
- Secured Messages using SPDM over MCTP Binding Specification, Revision 1.1.0 (DSP0276)
- Secured Messages using SPDM Specification, Revision 1.1.0 (DSP0277)
5.5.1 SPDM Interface Requirements
4.1-4.3
SPDM/CMA over MCTP over 2-wire interfaces (I2C/SMBus or optionally I3C)
SPDM/CMA over MCTP over PCIe VDM
SPDM via DOE/CMA over PCIe (optional), if the device supports CXL IDE. In-band SPDM sessions are required
5.5.2 SPDM Features
5.5.2.1 Secure Sessions
do not required
5.5.2.2 Mutual Authentication
do not required
5.5.2.3 Certificate Provisioning Support
do not required
5.5.3 SPDM Device Response Messages
5.5.4 SPDM Device Critifcate Slots
DeviceCert Model support : required
AliasCert Model support : optional
5.5.4.1 Certificate Slots for CXL Memory FRUs
CXL memory FRUs must have at least one certifiate slot provisioned.
Two certificate slot configuration options are supported :
- option 1
- slot 0 - Contains the certificate chain attensting the authenticity of the FRU
- slot 1-7 - Not specified
- option 2
- slot 0 - Contains the certificate chain attensting the authenticity of the component executing the SPDM protocol
- slot 1 - Contains the certificate chain attensting the authenticity of the FRU
- slot 2-7 - Not specified
5.5.5 SPDM Certificate Requirements
DMTF SPDM specifications noted in section 10.4
나머지는 모두 optional
5.5.6 SPDM Challenge-Response
5.5.7 SPDM Firmware Measurements
CXL memory devices generate signed measurements and support the following measurement block types:
CXL memory devices do not require support for recomuting all measurements without requiring a Reset. It’s acceptable that is capability flag MEAS_FRESH_CAP = 0.
6. Management Sensors and Effectors
support and expose sensors and effectors for device management using PLDM Type 2 messages and optional Redfish Device Enablement(RDE, PLDM Type 6)
The minium set of sensors and effectors in this sections
- PLDM State Set IDs as defined in DMTF Platform Level Data Model (PLDM) State Set Specification, Revision 1.1.0 (DSP0249)
- PLDM Sensor and Effecter Units as defined in section 27.4 ('Sensor and effector units') of DMTF Platform Level Data Model (PLDM) for Platform Monitoring and Control Specification, Revision 1.2.2 (DSP 0248)
Guidance on how to model a CXL memory device in PLDM Type 2 semantics :
- PLDM Type2 CXL Memory Device Modeling , Revision 1.0 (DSP2067)
6.1. CXL Memory Device Sensors and Effectors
7. Thermal Management
CXL Component Command Interface (CCI) and DMTF Platform Level Data Model (PLDM)
how compliant CXL memory devices implement CCI based thermal management
details on the standard implementation of thermal management in PLDM semantics
7.1. Temperature Sensors
This section directly addresses the implementation of physical or component-integrated temperature sensors on a CXL memory device.
CXL memory FRUs are deviced into two common topologies
aseembled module with local memory media (e.g. JEDEC CMM)
memory media FRU installation module (e.g. AIC with DIMM socket)
DD5 temperature sensor on DIMM (TSOD) is defined by JEDEC with 3 temperature sensors on each DIMM
DRAM ambient temperature sensor calibration should be owned by CMM or DIMM vendor to provide the offset values
System is capable of overriding initial offset values
temperature senros on CXL memory controller and PMIC as well for die temperature measurement
Temperature Sensors
7.2 CXL Memory Temperature Reporting Interface
system soft/host에게 report하기 위해 2개의 인터페이스가 있음
CXL CCI (Component Command Interface) reporitng an aggregate device temperature
PLDM (Platform Level Data Model) with aggregate and granular reporintg
7.2.1 CXL CCI
Byte 4('Device Temperature') of the ‘Get Health Info’ utilize an aggregate temperature (T_aggregate)
reporting Warning and Critical device temperature status (e.g. Bits[3:2] in Byte Offset 2 of ‘Get Health Info’)
In general, all PLDM temperature sensors measuring for a critical component temperature limit (e.g. DRAM, CMC, PMIC, etc) should be calculated as part of the CCI aggregate temperature reading.
64ms 마다 measure
T_aggregate = 85 - min ( (T_sensor1_thresh - T_sensor1_cur), (Tsensor2_thresh - T_sensor2_cur)…)
Taggregate = 85 – min[(85-80), (100-70), (80-60), (90-50)] = 85 – min[(5), (30), (20), (40)] = 80
7.2.2 PLDM
CXL memory controller는 모든 temperature sensors들으리 값을 읽어서 BMC와 같은 관리 주체의 레지스터들에 기록을 해둔다.
Each individual PLDM sensor on CMDs has it’s own PLDM Warning/Critical/Fatal threshold temperature which is configurable.
7.3 CXL Memory Thresholds and Alert Configuration
아래는 가능한 예제
7.3.1 CXL CCI
‘Set Alert Config’ to set warning threshold for aggregate temperature
85’c~95’c를 Warning and Critical thresholds 로 추천함
7.3.2 PLDM
Platform Monitoring and Control Specification (DSP0248)
7.4 Thermal Event Handling Examples
7.4.1 CXL CCI Based
Management entities set FRU temperature thresholds
- Controller pools all the snsors, calculates aggregate temperature, report aggregate temperature vis CCI Get HEalth Info and keep register values
- Management entities may poll via CCI Get health status and/or PLDM FRU temperature
When Threshold exceeds for a single temperature sensor
Controller continues to report aggregate temperature, report aggregate temperature via CCI Get Health Info and report PLDM event for the temperature sensor that exceeded
Management entities may start any system action (fan speed) and/or start polling the particular sensor
When Threshold exceeds for an aggregate temperature values
Controller report CCI event for aggregate temperature, and report PLDM event for the temperature sensor that exceeded
Management entities may start any system action (fan speed) and/or start polling the particular sensor
7.4.2 PLDM Based
위 CXL CCI based와 유사
8. FRU Information (VPD) Content
Vital Product Data (VPD) is data that describes a CXL memory device Field Replaceable Unit (FRU) to a host system to enable discovery and configuration.
8.1 VPD Formats
8.1.1 IPMI VPD Structure
8.2 JEDEC Record Type 0xC0 - JEDEC Record for CXL Memory Devices
8.2.1 FRU Record Element Descriptor
8.2.2 FRU Record Element Data Field Format
8.2.3 Notes Regarding FRU Records
8.2.4 CXL Memory Device FRU Content
8.3 JEDEC Record Type 0xE1 - Thermal Record
8.4 JEDEC Record Type 0xE2 - Poiwer Record
'Data Handling' 카테고리의 다른 글
| Assign a variable at the end of DRAM region for ARM (0) | 2023.12.01 |
|---|