Fundamentals of Accelerated Computing with CUDA C/C++ Training Course

GPU Computing and CUDA Architecture

• Architectural distinctions between CPUs and GPUs
• NVIDIA GPU streaming multiprocessor architecture
• Overview of the CUDA programming model
• Heterogeneous computing and the host-device paradigm

Setting Up the CUDA Development Environment

• Installation of the CUDA Toolkit 13.x
• NVCC compiler and build workflow
• Verifying the environment using device queries
• IDE integration and development tools

Writing and Launching CUDA Kernels

• Syntax and qualifiers for kernel functions
• Launch configuration and execution
• Vector addition and fundamental data-parallel patterns
• CUDA error-checking macros

CUDA Thread Hierarchy and Execution Model

• Organization of grids, blocks, and threads
• Thread indexing and global ID calculation
• Warp execution and the SIMT model
• Occupancy and resource utilization

GPU Memory Architecture and Management

• Memory types: global, shared, constant, and registers
• Allocating and freeing device memory
• Transfers between host and device
• Using shared memory for intra-block collaboration

Unified Memory and Data Migration

• The unified memory model and managed allocations
• Page migration and on-demand paging
• Asynchronous prefetching using cudaMemPrefetchAsync
• Memory advice hints for access patterns

System-Wide Profiling with Nsight Systems

• Timeline analysis in Nsight Systems
• Identifying CPU-GPU synchronization points
• Visualizing kernel execution and memory transfers
• Interpreting system-level performance data

Kernel Optimization with Nsight Compute

• Interactive kernel profiling in Nsight Compute
• Analysis of memory throughput and bandwidth
• Compute utilization and warp state statistics
• Guided analysis and optimization guidelines

Concurrent Streams and Asynchronous Operations

• CUDA streams and the default stream
• Overlapping kernel execution with data transfers
• Stream synchronization and CUDA events
• Multi-stream pipeline design patterns

Error Handling and Debugging Tools

• CUDA API error codes and recovery strategies
• Compute-sanitizer for memory access verification
• cuda-gdb for kernel debugging
• Assertions and synchronous error detection

Profile-Driven Optimization Workflow

• Iterative profiling methodology
• Bottleneck identification and prioritization
• Performance regression testing
• Documenting optimization decisions

End-to-End Accelerated Application Project

• Designing a complete GPU-accelerated solution
• Integrating profiling throughout development
• Performance benchmarking and reporting
• Deployment considerations for production environments