Keraunos PCIe Tile Documentation

Welcome to the Keraunos PCIe Tile SystemC/TLM2.0 implementation documentation.

Architecture & Design:

• Keraunos System Architecture
  • PCIe Tile Integration in Keraunos-E100 Chiplet Ecosystem
  • Executive Summary
  • Table of Contents
  • 1. System Overview
    • 1.1 Grendel Chiplet Ecosystem
    • 1.2 Keraunos-E100 Role
  • 2. Keraunos-E100 Chiplet Architecture
    • 2.1 High-Level Block Diagram
    • 2.2 Key Subsystems
  • 3. PCIe Tile Position in the System
    • 3.1 PCIe Tile Overview
    • 3.2 Architectural Position
    • 3.3 Key Interfaces
    • 3.4 Model Integration: Host–RC–EP–PCIe Tile Connection Diagram
    • 3.5 VDK Integration: PCIe Tile and Synopsys PCIe Controller in the Virtualizer
  • 4. Connectivity Architecture
    • 4.1 Inbound Data Path (Host → Chip)
    • 4.2 Outbound Data Path (Chip → Host)
    • 4.3 Configuration Path (SMN → PCIe Tile Registers)
    • 4.4 MSI Interrupt Path (Chip → Host)
  • 5. Data Flow Paths
    • 5.1 End-to-End Data Flow Example: Host DMA to Quasar
    • 5.2 Multi-Hop Data Flow: Quasar → PCIe → Host → PCIe → Quasar
  • 6. Address Space Integration
    • 6.1 System Address Map
    • 6.2 PCIe BAR (Base Address Register) Mapping
    • 6.3 Address Translation Stages
  • 7. System Use Cases
    • 7.1 Use Case 1: Model Initialization
    • 7.2 Use Case 2: Inference Execution
    • 7.3 Use Case 3: Package-to-Package Communication
    • 7.4 Use Case 4: System Management
  • 8. Final VDK Platform: Linux-Booting PCIe Tile Integration
    • 8.1 Overview
    • 8.2 Dual-Chiplet VDK Topology
    • 8.3 Host Memory Map
    • 8.4 Device (Keraunos_PCIE_Chiplet) Memory Map
    • 8.5 End-to-End Data Path
    • 8.6 Linux Boot Flow
    • 8.7 PCIe Enumeration
    • 8.8 The pcie_xfer Application
    • 8.9 Device-Side Firmware (pcie_bringup)
    • 8.10 VP Configuration
    • 8.11 Sideband Connections in the Final Platform
  • 9. Appendices
    • 9.1 Acronyms and Abbreviations
    • 9.2 Reference Documents
    • 9.3 Revision History
• Keraunos PCIe Tile - High-Level Design
  • 1. Introduction
    • 1.1 Purpose
    • 1.2 Scope
    • 1.3 Key Terms
  • 2. Architecture Overview
    • 2.1 Top-Level Architecture
    • 2.2 Design Principles
    • 2.3 System Connection Diagrams
  • 3. Theory of Operation
    • 3.1 Operational Modes
    • 3.2 Initialization and Boot Sequence
    • 3.3 Inbound Data Flow (PCIe to NOC)
    • 3.4 Outbound Data Flow (NOC to PCIe)
    • 3.5 Configuration Access Flow
    • 3.6 MSI Interrupt Generation Flow
    • 3.7 CII Tracking and Configuration Update
    • 3.8 Isolation and Recovery
    • 3.9 Error Handling
    • 3.10 Operating Mode Summary
  • 4. Detailed Block Diagram
    • 3.1 Complete System Architecture
  • 5. Data Flow Paths
    • 4.1 Inbound Data Flow (PCIe → NOC)
    • 4.2 Outbound Data Flow (NOC → PCIe)
    • 4.3 Configuration Access Flow (SMN → Config)
    • 4.4 MSI Generation Flow
    • 4.5 CII Interrupt Flow
  • 6. Module Descriptions
    • 5.1 NOC-PCIE Switch
    • 5.2 Translation Lookaside Buffers
    • 5.3 MSI Relay Unit
    • 5.4 System Information Interface (SII)
  • 7. Address Map
    • 6.1 Inbound Address Routing
    • 6.2 Configuration Address Map
  • 8. Clock and Reset Strategy
    • 7.1 Reset Hierarchy
    • 7.2 Reset Effects
    • 7.3 Isolation Sequence
  • 9. Interface Specifications
    • 8.1 External TLM Sockets
    • 8.2 Control and Status Signals
  • 10. Known Limitations and Findings
    • 9.1 Architecture Limitations
    • 9.2 Documented Findings
  • 11. Test Coverage Summary
    • 10.1 Test Suite Overview
    • 10.2 Coverage Metrics
  • 12. References
    • 11.1 Related Documentation
    • 11.2 Source Code Organization
• Keraunos PCIE Tile SystemC/TLM2.0 Design Document
  • Sphinx Setup Instructions
    • 1. Install Required Extensions
    • 2. Configure conf.py
    • 3. Build HTML Documentation
    • 4. Alternative: Use make
  • ⭐ Key Implementation Features
  • Table of Contents
  • 1. Introduction
    • 1.1 Purpose
    • 1.2 Scope
    • 1.3 References
    • 1.4 Implementation Version
  • 1.5 Refactored Architecture Overview ⭐ NEW
    • 1.5.1 Why Refactoring Was Necessary
    • 1.5.2 Refactored Architecture Pattern
    • 1.5.3 Function Callback Communication Pattern
    • 1.5.4 Smart Pointer Memory Management
    • 1.5.5 SCML2 Memory Integration
    • 1.5.6 Temporal Decoupling Support
    • 1.5.7 Modern C++ Best Practices Applied
    • 1.5.8 File Organization
    • 1.5.9 Component Communication Pattern
    • 1.5.10 Null Safety Pattern
    • 1.5.11 Performance Characteristics
    • 1.5.12 Code Example - Complete Transaction Path
  • 2. System Overview
    • 2.1 Keraunos PCIE Tile Context
    • 2.2 Modeled Components
    • 2.3 Design Objectives
  • 3. Architecture
    • 3.1 Overall Structure
    • 3.2 Component Hierarchy
    • 3.3 Data Flow
  • 4. Component Design
    • 4.1 TLB Common Structures
    • 4.2 Inbound TLB Design
    • 4.3 Outbound TLB Design
    • 4.4 MSI Relay Unit Design
    • 4.5 Intra-Tile Fabric Switch Design
    • 4.6 System Information Interface (SII) Block
    • 4.7 Configuration Register Block
    • 4.8 Clock & Reset Control Module
    • 4.9 PLL/CGM (Clock Generation Module)
    • 4.10 PCIE PHY Model
    • 4.11 External Interface Modules
    • 4.12 Top-Level Keraunos PCIE Tile Module
  • 5. Interface Specifications
    • 5.1 TLM2.0 Interfaces
    • 5.2 SystemC Signals
    • 5.3 Address Translation Interfaces
  • 6. Implementation Details
    • 6.1 Address Translation Algorithms
    • 6.2 Error Handling
    • 6.3 MSI Relay Unit State Machine
    • 6.4 Threading Model
    • 6.5 Memory Modeling
  • 7. Modeling Approach
    • 7.1 Abstraction Level
    • 7.2 SCML2 Usage
    • 7.3 TLM2.0 Compliance
    • 7.4 Design Patterns
  • 8. Performance Considerations
    • 8.1 Simulation Performance
    • 8.2 Optimization Opportunities
    • 8.3 Scalability
  • 9. Dependencies and Requirements
    • 9.1 Software Dependencies
    • 9.2 Hardware Dependencies
    • 9.3 Integration Requirements
  • 9. Detailed Implementation Architecture
    • 9.1 Class Hierarchy and Relationships
    • 9.2 Communication Architecture
    • 9.3 Memory Management Architecture
    • 9.4 Callback Wiring Implementation
    • 9.5 SCML2 Memory Usage Pattern
    • 9.6 Component Lifecycle
    • 9.7 Transaction Processing Flow
    • 9.8 Routing Decision Implementation
    • 9.9 TLB Translation Implementation
    • 9.10 Error Handling Strategy
    • 9.11 Configuration Register Implementation
  • 10. Implementation Guide
    • 10.1 Building the Design
    • 10.2 Running Tests
    • 10.3 Adding New Components
    • 10.4 Debugging and Troubleshooting
    • 10.5 Performance Tuning
    • 10.6 Test Development Guide
    • 10.7 Configuration Management
    • 10.8 Integration with VDK Platform
    • 10.9 Memory Management Best Practices
    • 10.10 Coding Standards Applied
  • 11. Test Infrastructure
    • 11.1 Test Framework Overview
    • 11.2 Test Categories (33 Tests)
    • 11.3 Test Execution Results
    • 11.4 Test API Examples
    • 11.5 Coverage Goals
  • 12. Migration from Original Design
    • 12.1 For Developers Familiar with Original
    • 12.2 API Migration Guide
    • 12.3 Backward Compatibility Notes
  • 13. Known Limitations and Future Work
    • 13.1 Current Limitations
    • 13.2 Future Enhancements
  • 14. Lessons Learned and Best Practices
    • 14.1 Architecture Decisions
    • 14.2 Design Patterns Applied
    • 14.3 Recommendations for Similar Projects
  • Appendix A: Implemented Components Summary
    • A.1 Complete Component List
    • A.2 Component Statistics
    • A.3 SCML Compliance
  • Appendix B: Address Map Summary
    • B.1 TLB Configuration Space
    • B.2 MSI Relay Unit Address Map
    • B.3 SII Block Address Map
    • B.4 Config Register Block Address Map
    • B.5 SMN-IO Switch Address Map
    • B.6 NOC-IO Switch Address Map
    • B.7 NOC-PCIE Switch Routing Map
  • Appendix C: Acronyms and Abbreviations

VDK Platform & Integration:

• VDK Platform Guide: Linux-Booting Keraunos PCIe Tile
  • 1. Introduction
    • 1.1 What This Platform Demonstrates
  • 2. Platform Architecture
    • 2.1 Two-Chiplet Topology
    • 2.2 PCIe Link Wiring
    • 2.3 PCIE_TILE Interface Connections
  • 3. Memory Maps
    • 3.1 Host_Chiplet CPU Address Space
    • 3.2 Keraunos_PCIE_Chiplet SharedMemoryMap Decode
  • 4. Linux Boot
    • 4.1 Boot Flow
    • 4.2 Device Tree Highlights
    • 4.3 Key Boot Parameters
    • 4.4 Toolchain Requirements
  • 5. PCIe Enumeration and BAR Assignment
    • 5.1 Enumeration Topology
    • 5.2 iATU Configuration
  • 6. Data Transfer Application (pcie_xfer)
    • 6.1 Purpose
    • 6.2 Data Path
    • 6.3 Supported Commands
    • 6.4 EP Auto-Detection
    • 6.5 Building pcie_xfer
  • 7. Bare-Metal Test (pcie_e2e_test)
  • 8. VP Configuration Reference
    • 8.1 Available Configurations
    • 8.2 Critical VPCFG Parameters
    • 8.3 Image Loading
  • 9. Sideband Signal Mapping
    • 9.1 EP ↔ PCIE_TILE Sideband
    • 9.2 Clocks and Resets
    • 9.3 MSI Interrupt Path
  • 10. Troubleshooting
    • 10.1 Host Crashes or Nonsense PC
    • 10.2 PCIe EP Not Detected
    • 10.3 BAR0 Access Returns All-Ones
    • 10.4 Data Mismatch on Read-Back
  • 11. Related Documents

Verification:

• Keraunos PCIE Tile SystemC/TLM2.0 Testplan
  • Table of Contents
  • 1. Introduction
    • 1.1 Purpose
    • 1.2 Scope
    • 1.3 Test Objectives
    • 1.4 TLB Translation and Sys In/Out (Reference)
    • 1.5 References
  • 1.5 Test Traceability Matrix
  • 2. Test Environment
    • 2.1 Testbench Structure
    • 2.2 Test Components
    • 2.3 Test Tools
  • 3. Test Strategy
    • 3.1 Test Levels
    • 3.2 Test Methodology
    • 3.3 Test Phases
  • 4. Inbound TLB Test Cases
    • 4.1 TLBSysIn0 Test Cases
    • 4.2 TLBAppIn0 Test Cases
    • 4.3 TLBAppIn1 Test Cases
  • 5. Outbound TLB Test Cases
    • 5.1 TLBSysOut0 Test Cases
    • 5.2 TLBAppOut0 Test Cases
    • 5.3 TLBAppOut1 Test Cases
  • 6. MSI Relay Unit Test Cases
    • 6.1 Register Access Tests
    • 6.2 MSI Generation Tests
  • 7. Integration Test Cases
    • 7.1 TLB + MSI Relay Integration
    • 7.2 Multiple TLB Integration
  • 8. Switch Test Cases
    • 8.1 NOC-PCIE Switch Test Cases
    • 8.2 NOC-IO Switch Test Cases
    • 8.3 Bus Master Enable (BME) Test Cases
    • 8.4 SMN-IO Switch Test Cases
  • 9. SII Block Test Cases
    • 9.1 Register Access Tests
  • 10. Config Register Block Test Cases
    • 10.1 Register Access Tests
  • 11. Clock & Reset Test Cases
    • 11.1 Clock Generation Tests
    • 11.2 Reset Sequence Tests
  • 12. PLL/CGM Test Cases
    • 12.1 PLL Lock Tests
  • 13. PCIE PHY Test Cases
    • 13.1 PHY Configuration Tests
  • 14. External Interface Test Cases
    • 14.1 NOC-N Interface Tests
    • 14.2 SMN-N Interface Tests
  • 15. Top-Level Integration Test Cases
    • 15.1 Full Tile Integration
  • 16. End-to-End Test Cases
    • 16.1 Inbound Data Paths (PCIe → NOC/SMN)
    • 16.2 Outbound Data Paths (NOC/SMN → PCIe)
    • 16.3 Configuration Paths (SMN → TLB/SII/MSI Config)
    • 16.4 MSI Interrupt Flows
    • 16.5 Status Register Access
    • 16.6 Isolation and Error Handling
    • 16.7 Multi-Path Concurrent Traffic
    • 16.8 Reset and Initialization Sequences
    • 16.9 Complete Transaction Flows
    • 16.10 Error Injection and Recovery
    • 16.11 MSI-X End-to-End Flow
    • 16.12 Clock Domain Crossing
    • 16.13 Performance and Stress Tests
    • 16.14 Power Management
    • 16.15 System Integration Scenarios
    • 16.16 Refactored Architecture Validation
  • 17. Coverage Goals
  • 16. Coverage Goals
    • 16.1 Functional Coverage
    • 16.2 Code Coverage
    • 16.3 Specification Coverage
  • 17. Test Infrastructure
    • 9.1 Testbench Components
    • 9.2 Test Utilities
    • 9.3 Test Configuration
  • 18. Regression Testing
    • 10.1 Regression Suite
    • 10.2 Regression Criteria
    • 10.3 Continuous Integration
  • Appendix A: Unit/Integration Test Case Summary (Sections 4-15)
  • Appendix A.2: End-to-End Test Case Summary (Section 16) — Implemented in Keranous_pcie_tileTest.cc
  • Appendix B: Test Execution Plan
    • Phase 1: Unit Tests (Weeks 1-3)
    • Phase 2: Functional Tests (Weeks 4-5)
    • Phase 3: Integration Tests (Weeks 6-7)
    • Phase 4: System Tests (Week 8)
    • Phase 5: Regression (Week 9)
• Enable Gating Feature - Design Documentation
  • Overview
  • Register Interface
  • Implementation
    • Signal Propagation
    • Key Features
  • Test Coverage

Overview

This documentation describes the SystemC/TLM2.0 implementation of the Keraunos PCIe Tile, featuring:

• Linux-Booting VDK Platform - Full RISC-V Linux on host with PCIe endpoint enumeration

• End-to-End PCIe Data Path - Host to Target_Memory via RC, EP, PCIE_TILE, and noc_n_initiator

• pcie_xfer Application - Userspace tool for interactive BAR0 read/write operations

• Test Suite - 86 tests (41 E2E + 45 directed), 344 checks, including BME (Bus Master Enable) tests

• Zero Memory Leaks - Smart pointer (RAII) based design

• Modern C++17 - Best practices throughout

• SCML2 Memory - Proper persistent storage

• Temporal Decoupling - Full TLM-2.0 LT support

Key Features

The refactored architecture (v2.0) introduces:

1. Dual-Chiplet VDK Platform - Host_Chiplet (RC + Linux) and Keraunos_PCIE_Chiplet (EP + Tile + Memory)

2. Linux PCIe Stack Integration - snps,dw-pcie driver enumerates EP, assigns BARs

3. Function Callback-Based Communication - Eliminates internal socket bindings

4. Smart Pointer Memory Management - Zero memory leaks guaranteed

5. SCML2 Integration - Proper configuration persistence

6. Comprehensive Testing - 86 test cases (incl. 5 BME tests) with cross-socket data verification

7. Sparse Backing Memory - Custom sparse_backing_memory with 256TB address range

8. Initiator Socket Architecture - DUT output ports use simple_initiator_socket for outbound traffic

Platform Architecture

The validated VDK platform demonstrates the complete data path:

Host CPU (Linux) → RC AXI_Slave → iATU → PCIe TLP
  → EP PCIMem_Slave → EP BusMaster
    → PCIE_TILE.pcie_controller_target
      → noc_n_initiator → Target_Memory (16 MB)

See VDK Platform Guide: Linux-Booting Keraunos PCIe Tile for full platform details.

Components

The tile includes:

• Translation Lookaside Buffers (TLBs) - 6 inbound, 3 outbound

• MSI Relay Unit - Interrupt management

• Intra-Tile Fabric Switches - NOC-PCIE, NOC-IO, SMN-IO

• System Information Interface (SII)

• Configuration Register Block

• Clock & Reset Control

• PLL/CGM (Clock Generation Module)

• PCIe PHY Model

Documentation Sections

System Architecture (Keraunos_System_Architecture.md):

• Grendel chiplet ecosystem and Keraunos-E100 role

• PCIe Tile position and interfaces

• Model integration (Host-RC-EP-Tile connections)

• VDK integration with Synopsys DesignWare PCIe

• Final VDK platform: Linux boot, PCIe enumeration, data transfer

VDK Platform Guide (VDK_Platform_Linux_PCIe_Guide.md):

• Dual-chiplet topology and wiring

• Memory maps (host and device)

• Linux boot flow via OpenSBI

• PCIe enumeration and BAR assignment

• pcie_xfer application usage

• VP configuration reference

• Troubleshooting guide

Indices and tables

• Index

• Search Page