Performance Analysis with Vampir Bert Wesarg Technische Universität Dresden
Outline Part I: Welcome to the Vampir Tool Suite Mission Event trace visualization Vampir & VampirServer The Vampir displays Part II: Vampir hands-on Visualizing and analyzing NPB-MZ-MPI / BT PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 2
Event trace visualization with Vampir Alternative and supplement to automatic analysis Show dynamic run-time behavior graphically at any level of detail Provide statistics and performance metrics Timeline charts Show application activities and communication along a time axis Summary charts Provide quantitative results for the currently selected time interval PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 3
Visualization modes (1) Directly on front end or local machine % vampir Core Core Core Core Multi-Core Program Core Core Core Core Score-P Trace File Vampir Small/Medium sized trace Thread parallel PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 4
Visualization Modes (2) On local machine with remote VampirServer % vampirserver start % vampir VampirServer Vampir CPU CPU CPU CPU CPU CPU CPU CPU Core Core Core Core CPU CPU CPU CPU CPU CPU CPU CPU Core Core Core Core Many-Core CPU CPU CPU CPU CPU CPU Program CPU CPU Core Core Core Core Score-P Trace File LAN/WAN Large trace file (stays on remote machine) CPU CPU CPU CPU CPU CPU CPU CPU Core Core Core Core Parallel application PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 5
The main displays of Vampir Timeline Charts: Master Timeline Process Timeline Counter Data Timeline Performance Radar Summary Charts: Function Summary Message Summary Process Summary Communication Matrix View PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 6
Hands-on: Visualizing and analyzing NPB-MZ-MPI / BT
Start Vampir % vampir scorep_bt-mz_w_4x4_trace/traces.otf2 PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 8
Visualization of the NPB-MZ-MPI / BT trace Navigation Toolbar Function Summary Master Timeline Function Legend PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 9
Visualization of the NPB-MZ-MPI / BT trace Master Timeline Detailed information about functions, communication and synchronization events for collection of processes. PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 10
Visualization of the NPB-MZ-MPI / BT trace Process Timeline Detailed information about different levels of function calls in a stacked bar chart for an individual process. PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 11
Visualization of the NPB-MZ-MPI / BT trace Typical program phases Initialisation Phase Computation Phase PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 12
Visualization of the NPB-MZ-MPI / BT trace Counter Data Timeline Detailed counter information over time for an individual process. PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 13
Visualization of the NPB-MZ-MPI / BT trace Performance Radar Detailed counter information over time for a collection of processes. PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 14
Visualization of the NPB-MZ-MPI / BT trace Zoom in: Inititialisation Phase Context View: Detailed information about function initialize_. PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 15
Visualization of the NPB-MZ-MPI / BT trace Find Function Execution of function initialize_ results in higher page fault rates. PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 16
Visualization of the NPB-MZ-MPI / BT trace Computation Phase Computation phase results in higher floating point operations. PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 17
Visualization of the NPB-MZ-MPI / BT trace Zoom in: Computation Phase MPI communication results in lower floating point operations. PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 18
Visualization of the NPB-MZ-MPI / BT trace Zoom in: Finalisation Phase Early reduce bottleneck. PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 19
Visualization of the NPB-MZ-MPI / BT trace Process Summary Function Summary: Overview of the accumulated information across all functions and for a collection of processes. Process Summary: Overview of the accumulated information across all functions and for every process independently. PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 20
Visualization of the NPB-MZ-MPI / BT trace Process Summary Find groups of similar processes and threads by using summarized function information. PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 21
Serial exercises /home/docretreat/share/scorep+vampir/scorep_bt_c_serial_trace/traces.otf 2 What are the functions with the longest running time in total? Color the top 3 with different colors Which function called these functions? How often is this function called? What is the average call time of this function? PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 22
OpenMP exercises /home/docretreat/share/scorep+vampir/scorep_bt_c_omp_4_trace/traces.otf2 Total time spent in OpenMP barriers? Which threads spent the most time in the barriers? PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 23
MPI exercises /home/docretreat/share/scorep+vampir/scorep_bt_c_mpi_16_trace/traces.otf 2 Find MPI_Init How many nodes did the job used? Total wait time in MPI functions? Communication pattern and cliques? MPI collective used? PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 24
Summary and Conclusion
Summary Vampir & VampirServer Interactive trace visualization and analysis Intuitive browsing and zooming Scalable to large trace data sizes (20 TiB) Scalable to high parallelism (200,000 processes) Vampir is available for Linux, Windows, and Mac OS X PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 26
Vampir is available at http://www.vampir.eu Get support via vampirsupport@zih.tu-dresden.de PERFORMANCE ENGINEERING WITH SCORE-P AND VAMPIR, PASSAU, SEPTEMBER 15, 2015 27