# Computing competencies

HPC University has defined lists of competencies important for computational science and learning the words helps in discussions with computational researchers

The following very detailed lists were compiled by a number of computational institutions and were designed to serve as guidance for designing computational courses at various levels. You are **not** expected to know all these in detail (as you are not doing a computational science course). However, it can be very helpful to understand what these words mean to facilitate discussions with computational scientists. Feel free to add explanations to the Glossary.

- To give you a rough overview, below we give the 7 areas important for undergraduates in computational science as identified by the HPC University website. This is followed by he fully unfolded lists of all competencies for your convenience; if you want a more structured overview, that allows folding, then go to the HPC University website.
- These lists were made to help design curricula for a student workforce preparing to address these challenges in the modern field of Computational Science and Engineering (also called High Performance Computing)
- Resources for teaching a good part of these competencies are available here in searchable form.
- A training roadmap is available for establishing a sequence in which to teach this material.

## Minor Program in Computational Science Competency/Topic Overview (Draft as defined by HPC University)

### Overview:

### Area 1: Simulation and Modeling

### Area 2: Programming and Algorithms

### Area 3: Differential Equations and Discrete Dynamical Systems

### Area 4: Numerical Methods

### Area 5: Optimization

### Area 6: Parallel Programming

### Area 7: Scientific Visualization

### Details

### Area 1: Simulation and Modeling [-]

**Explain the role of modeling in science and engineering:[-]**- Discuss the importance of modeling to science and engineering
- Discuss the history and need for modeling
- Discuss the cost effectiveness of modeling
- Discuss the time-effect of modeling (e.g. the ability to predict the weather)
- Define the terms associated with modeling to science and engineering
- List questions that would check/validate model results
- Describe future trends and issues in science and engineering
- Identify specific industry related examples of modeling in engineering (e.g., Battelle; P&G, material science, manufacturing, bioscience, etc.)
- Discuss application across various industries (e.g., economics, health, etc.)

**Analyze modeling and simulation in computational science:[-]****Create a conceptual model:[-]**- Illustrate a conceptual modeling process through examples
- Identify the key parameters of the model
- Estimate model outcomes
- Utilize modeling software and/or spreadsheets to implement model algebraic equations (e.g. Vensim, Excel, MATLAB, Mathematica)
- Construct a simple computer visualization of the model results (e.g. infectious disease model, traffic flow, etc.)
- Validate the model with data
- Discuss model quality and the sources of errors

**Examine various mathematical representations of functions:[-]**- Describe linear functions
- Define non-linear functions (e.g., polynomials, exponential, periodic, parameterized, etc.)
- Visualize functions utilizing software (e.g. Excel, Function flyer, etc.)
- Determine appropriate functional form to fit the data
- Demonstrate essential mathematical concepts related to modeling and simulation

**Analyze issues in accuracy and precision:[-]****Understand discrete and difference-based computer models:[-]**- Explain the transition of a continuous function to its discrete computer representation
- Represent "rate of change" using finite differences
- Cite examples of finite differences
- Explain derivatives and how they relate to model implementation on a computer
- Write pseudo-code for finite difference modeling

**Demonstrate computational programming utilizing a higher level language or modeling tool (e.g. Maple, MATLABTM, Mathematica, other):[-]**- Describe the system syntax (e.g., menus, toolbars, etc.)
- Define elementary representations, functions, matrices and arrays, script files, etc.
- Explain programming and scripting processes (e.g., relational operations, logical operations, condition statements, loops, debugging programs, etc.)
- Create tabular and visual outputs (e.g., 2-D and 3-D subplots)
- Translate the conceptual models to run with this system and assess the model results (e.g. traffic flow and/or "spread of infectious disease")
- Illustrate other people's models utilizing the modeling program

**Assess computational models:[-]**- Assess problems with algorithms and computer accuracy
- Discuss techniques and standards for reviewing models
- Verify and validate the model
- Discuss the differences between the predicted outcomes of the model and the computed outcomes and relevance to the problem
- Discuss the suitability and limits of the model to address the problem for which the model was designed

**Verification, Validation, and Accreditation:[-]****Complete a team-based, real-world model project:[-]****Demonstrate technical communication:[-]**

### Area 2: Programming and Algorithms [-]

**Describe the fundamentals of problem solving[-]**- Understand Top-Down thinking and program design Discuss breaking up a problem into its component tasks Understand how tasks acquire data
- Describe how tasks should be ordered
- Represent tasks in a flow-chart style format
- Understand the difference between high-level languages (for example Mathematica, Maple or MATLAB), medium level languages (for example FORTRAN or C) and low-level languages (assembler) and when each should be used

**Understand and write Pseudo code:[-]**- List the basic programming elements of Pseudo code
- Explain the logic behind an if/then/else statement
- Understand the iterative behavior of loops
- Describe the difference between several looping constructs
- Write Pseudo code to solve basic problems
- Understand how to represent data flow in and out of subprograms

**Use subprograms in program design:[-]**- Describe how logical tasks can be implemented as subprograms
- Understand the logical distinction between functions and subroutines
- Explain the control flow when a function is called
- Define dummy and actual arguments
- Discuss the different relationships dummy and actual arguments
- Explain how function output is used
- Understand how languages handle passed data into functions and subprograms, especially one- and two-dimensional arrays

**Write code in a Programming language:[-]**- Understand the concept of syntax in a programming language
- Describe the syntax of the programming language constructs
- List the type of subprograms available in the language
- Explain the concepts of argument pass-by-value and pass-by-reference
- Understand what a compiler and linker do
- Understand the difference between a compiled and interpreted language
- Understand the difference between a typed and an un-typed language
- Understand the difference between a source file and an executable file
- Write and run basic programs in the language of choice
- Understand how to de-bug code and how to "sanity check" code
- Understand the importance of user-interfaces: clear input instructions including physical units if needed and clearly formatted and labeled output
- Understand the numerical limits of various data types and the implications for numerical accuracy of results

**Use different approaches to data I/O in a program:[-]****Understanding and use of fundamental programming Algorithms:[-]**- Explain an algorithm as an ordered series of solution steps
- Describe an algorithm for a simple programming problem
- Learn and use "classic" programming algorithms from a field of interest to the student. If possible, these should be algorithms used in the student's discipline.
- Describe what a software library is
- Understand how library functions implement algorithms
- Write code to implement your own version of "classic" algorithm
- Compare with code using a library function
- Understand data flow into library functions and implications of selecting any "tuning parameters" or options that may be required

**Explain various approaches to Program Design:[-]**- Describe Functional decomposition (Top-down Problem Solving)
- Be familiar with different programming styles (e.g. function, procedural, rule based)
- Understand how to modularize code
- Understand the benefits of code re-use
- Explain the operation of a Boss-Worker design
- Compare designs based on Global Variables vs. self-contained functions
- Define Object-Oriented Programming (OOP)
- Contrast OOP with functional decomposition
- Explain the power of Inheritance in OOP
- Understand how to document code
- Understand how to write and when to use stubs and drivers

**Possible Additions - Understand basic concepts of parallel programming:[-]**- Identify independent operations or tasks in a computational problem
- Understand how to represent a computational problem as a task graph
- Explain the difference between a process and a thread
- Become familiar with a thread-parallel programming model such as OpenMP
- Write a thread-parallel program using OpenMP or language constructs
- Understand the difference between shared and local data
- Understand data dependencies and race conditions
- Use synchronization primitives to eliminate race conditions in a thread-parallel program

### Area 3: Differential Equations and Discrete Dynamical Systems [-]

**Describe the solution methodology for first order linear differential and difference equations:[-]**- Analyze modeling problems with first order differential equations and present their solution methodology (e.g. linear, homogeneous, exact)
- Analyze modeling problems with first order difference equations and present their solution methodology (e.g. homogeneous, non-homogeneous). Analyze long term behavior

**Describe the solution methodology for systems of linear first order differential and difference equations:[-]**- Describe modeling problems with systems of first order differential equations and present their solution methodology (e.g., homogeneous with constant coefficients, variation of parameters)
- Describe modeling problems with systems of first order difference equations and their solution methodology (e.g., homogeneous with constant coefficients)

**Describe the solution methodology for higher order differential and difference equations:[-]**- Describe modeling problems with higher order differential equations analyze their solution methodology (e.g., homogeneous, non-homogeneous, undetermined coefficients, variation of parameters)
- Describe modeling problems with higher order difference equations analyze their solution methodology (e.g., homogeneous, non-homogeneous). Analyze the long-term behavior.

**Describe the solution methodology for differential equations using the Laplace Transforms:[-]****Describe the solution methodology for non-linear differential equations:[-]****Describe the solution methodology for non-linear difference equations:[-]**- Describe the method of linearization
- Describe the concepts of Logistic and Henon Maps
- Model with non-linear difference equations and demonstrate understanding of fundamental concepts from Bifurcation theory (e.g., fixed, periodic points, chaos)
- Describe techniques for controlling chaos
- Understand concepts of numerical accuracy applied to each solution approach

### Area 4: Numerical Methods [-]

**Understand number representation and computer errors:[-]****Analyze methods for solving non-linear equations:[-]****Describe techniques for solving systems of linear equations:[-]**- Describe the naive Gauss elimination and the partial pivoting method
- Understand the concepts of condition number and ill-conditioning problems
- Discuss and contrast factorization methods (e.g., LU, QR, Cholesky, SVD)
- Discuss and contrast iterative methods (e.g., Jacobi, Gauss Siedel)
- Describe convergence and stopping criteria of iterative methods

**Analyze techniques for computing eigenvalues/eigenvectors (Optional):[-]**- Describe and give examples of eigenvalue/eigenvector problems using specific, applied examples and their significance
- Describe canonical forms of matrices
- Describe and contrast direct methods for computing eigenvalues (e.g., power method, inverse power method)
- Describe and contrast transformation methods (e.g., QR algorithm)

**Describe interpolation and approximation methods:[-]****Describe numerical methods for Ordinary Differential Equations:[-]**- Describe and compare basic methods for IVPs (e.g., Euler, Taylor, Runge-Kutta)
- Describe and compare predictor-corrector methods
- Describe and compare multistep methods
- Discuss and contrast numerical methods for BVPs (e.g., shooting method, finite difference method)
- Compare the accuracy, memory requirements, and precision of each of the approaches

**Describe numerical methods for Partial Differential Equations:[-]**- Describe and compare numerical methods for parabolic PDEs (e.g., finite difference, Crank-Nicolson)
- Describe numerical methods for hyperbolic PDEs
- Describe numerical methods for elliptic PDEs (e.g. finite difference, Gauss- Seidel)
- Discuss the finite element method for solving PDEs
- Describe Monte Carlo Methods
- Describe applications of Monte Carlo models with examples Discuss algorithms for Monte Carlo methods

### Area 5: Optimization [-]

**Describe and use Optimization techniques:[-]**- Describe and contrast unconstrained optimization methods (e.g., Golden section search, Steepest descent, Newton's method, conjugate gradient, simulated annealing, genetic algorithms)
- Describe and contrast constrained optimization methods (e.g., Lagrange multiplier, quasi-Newton, penalty function method

**Implement linear and non-linear programs:[-]**

### Area 6: Parallel Programming [-]

**Describe the fundamental concepts of parallel programming and related architectures:[-]****Demonstrate parallel programming concepts using MPI:[-]**- Describe the MPI programming model
- Create, compile, and run an MPI parallel program
- Create MPI programs that utilize point-to-point communications
- Create an MPI program that uses point-to-point blocking communications
- Create an MPI program that uses point-to-point non-blocking communications
- Create an MPI program that uses collective communications
- Create an MPI programs that use parallel I/O
- Create MPI programs that use derived data types
- Create MPI programs that use vector derived data type
- Create MPI programs that use structure derived data type

**Demonstrate knowledge of parallel scalability:[-]****Demonstrate knowledge of parallel programming libraries and tools:[-]**

### Area 7: Scientific Visualization [-]

**Define SciVis needs; relationships to human visualization; basic techniques:[-]**- Define Scientific Visualization (Sci Vis)
- Discuss needs of SciVis (in the framework of a large variety of possible application areas)
- Survey different platforms for Visualization (e.g. AVS, VTK, OpenGL, VRLM)
- Discuss the different techniques and visualization methods used in SciVis
- Explain the human visualization system capabilities and perceptions
- Explain the different steps in the visualization pipeline
- Discuss different sources of data for SciVis and explain the terms applied to data types (i.e. scalar, vector, normal, tensor)
- Discuss different types of grids (e.g., regular vs. irregular grids)
- Discuss the different methods used to gather data
- Describe and explore the use of different file formats for sharing data (netCDF, XML, TIFF, GIF, JPEG, Wavefront OBJ)
- Discuss limitations of different methods
- Discuss future applications in emerging fields
- Metadata needs for graphics libraries

**Overview of computer graphic concepts:[-]**- Overview of SciVis concepts (pixels, rgb colors, 3D coordinate system, mapping 3D data to a 2Dscreen, continuous vs. discrete)
- Discuss polygonal representation
- Discuss lighting/shading
- Overview of classification/segmentation and transfer functions
- Discuss concept of rendering pipeline (no details about matrices)
- Discuss hardware rendering (mainly for polygonal models, few specialized volumetric hardware cards)
- Identify terms used in virtual space and in graphics elements
- Navigate in virtual space and manipulate primitive objects
- Explore colormaps and examine conceptual definitions for different color maps (pertaining to color spaces HSV, RGB, etc.) as related to representing data and relationships to perception

**Describe approaches to visualization for different scientific problems:[-]**- Examine different computational solutions to scientific problems
- Explain the different techniques used in visualization (i.e. glyphs, iso-contours, streamlines, image processing, volume-data)
- Examine the application of problems to visualization techniques
- Utilize software tools to implement visual image of a solution
- Discuss the use of time in animation

**Utilize software to implement grid representations of data:[-]**- Identify the various cell representations (i.e. points, polygons, 3d geometries)
- Discuss the application to different grid types (i.e. structured, unstructured, random)
- Discuss raycasting methods and texture mapping
- Examine the details of raycasting sampling FAT (low-resolution sampling), interpolation techniques)
- Examine algorithms: Direct Composite, SFP, use of transparency
- Identify the grid representation and data (color reps.) (regular grids, 1, 8, 24 and 32 bit color information)
- Discuss algorithms for manipulating images, distortion, fft's, enhancement, restoration, and frequency domains
- Utilize software to implement different grid types
- Discuss limitations of grids

**Use visualization software to display an isosurface:[-]**- Discuss different data types used: scalar vs. vector data
- Discuss the different grid types
- Discuss the different algorithms (Marching Cubes, etc.)
- Introduce details of the system being used in a course (e.g., VTK, AVS, etc.)
- Apply the system to extract and display an isosurface of some data set (could be tailored towards the teacher's and student's interests/application areas)
- Discuss limitations of these methods

**Use visualization software to complete a volumetric rendering:[-]**- Discuss direct volumetric rendering (raycasting and texture mapping) and its advantages/disadvantages vs. surface rendering
- Discuss segmentation/classification and transfer functions
- Discuss and illustrate how to use a system (VTK, AVS, etc.) to do volumetric rendering)
- Using the system, visualize a data set using raycasting
- Using the system, visualize a data set using texture mapping
- Discuss limitations of this method

**Utilize visualization software to visualize a vector dataset:[-]**- Discuss vector data
- Discuss different methods for vector visualizing (particles, stream ribbons, vector glyphs, etc.)
- Discuss the use of structured grid types: (ir)regular, cylindrical, spherical Discuss application areas for vector visualization (air flow, etc.)
- Using a system (VTK, AVS, etc.), visualize a vector data set
- Discuss limitations of this method

**Explore examples of image processing:[-]**- Discuss basic steps and goals in image processing
- Discuss variety of data sources of images and how they can be represented
- Discuss algorithms used for image processing
- Explore examples of image processing (e.g., noise reduction, image enhancement, feature extraction etc)
- Discuss challenges and limitations in image processing

**Use advanced techniques applied to a real problem:[-]****Examine SciVis problems for Biological Sciences "OMICS" applications:[-]**- Examine different problems existing in OMICS sciences that require visualization solution (overview)
- Discuss challenges of representing biomedical/biological data (i.e., representing protein structure or genomic sequence with all their attributes as a visual metaphor)
- Discuss challenges associated with visualization of scattered data such as text information and bioinformatics data (e.g., phylogenetic information)
- Gene finding in genomic sequences
- Protein folding and protein structure prediction
- Discuss the differences between protein folding and protein structure prediction
- Explore different methods used in protein folding and structure prediction
- Apply different methods of protein structure prediction and compare the results
- Construct, visualize and examine structure-based protein alignment
- Biological networks (e.g., protein-protein or protein-DNA interaction networks)
- Visualization of various types of expression data
- Discuss and contrast different types of expression data e.g., microarray gene expression data, protein expression data
- Discuss different visualization (and analyses) techniques used for expression data
- Apply (and compare outcomes) hierarchical clustering and k- means clustering to the same gene microarray expression data
- Discuss pros and cons of different clustering methods, their shortcomings, and ways to access the quality of clusters

- Discuss potential applications of SciVis techniques in biomedical and drug design fields
- Utilize MATLAB to implement/solve the above problems

**Explore SciVis techniques in BioMedical applications:[-]**- Explore a variety of biomedical applications of SciVis to explore large datasets such as MRI and confocal microscopy data
- Overview of volume visualization techniques in biomedical problems
- Examine and different ways MRI (Magnetic Resonance Imaging) data can be visualized (e.g., 2-D image versus isocontour slices)
- Discuss potential applications (interpretation) of each of the techniques
- Utilize software tools (MATLAB, VTK) to apply the techniques above

## Graduate Level Educational Competencies for Computational Science Overview (Draft as defined by HPC University)

### Prerequisites [-]

- The specification of the competencies assumes that students have come from undergraduate backgrounds with introductory computational science skills
- These could come from a formal program such as the minor program in computational science at the Ralph Regula School of Computational Science (RRSCS) collaborative institutions (see the Undergraduate Competencies tab above)
- This and other related programs generally require the following courses for undergraduates:
- This assumes that students interested in graduate computing will start with these skills or acquire them before taking courses to enhance their skills with the competencies in this document

### Area 1: Intermediate-Level Scientific Computing [-]

**Prereqs:[-]**- Introduction to modeling and simulation, introductory programming (any language), numerical methods, one discipline-oriented computational science course.
- This course will use either Fortran or C/C++ and will emphasize topics commonly encountered in scientific computing/computational science. It primarily addresses serial computing competencies and is a prerequisite to the HPC/parallel computing area.
- This course includes programming skills at an intermediate level and focuses specifically on scientific computing skills.

**Ability to contribute code in the programming language of greatest importance in the research domain (Fortran or C):[-]**- Students will understand basic language features and concepts
- Students will be able to understand code written by others
- Students will be able to use input/output functions effectively
- Students will complete a project to implement a new code and add a new capability to an existing code
- Students will be introduced to user-defined types, structures, classes, or similar mechanisms in their primary HPC programming language
- Students will be introduced to the concepts of object-oriented design and programming
**Example Activity:**Students will write a few small programs using common sort and search algorithms

**Students will be introduced to basic debugging techniques:[-]**- Students will recognize common runtime errors and program failures and will be able to describe some common mistakes that could cause them
- Students will be introduced to debugging strategies and rules of thumb
- Students will understand capabilities of commonly available symbolic debugging tools such as gdb
- Students will understand some common memory problems and how to fix them, for example, array overruns, invalid pointers, mismatched parameters in function calls
**Example Activity:**Students will follow a script to start a debugging session using a symbolic debugger on a sample code containing one or more common errors. The script will demonstrate useful debugging features such as setting break or watch points and inspecting variable values

**Understand number representation:[-]**- Students will understand the uses of floating point and integer arithmetic
- Students will understand the range limits of different size integers and will know how to select the appropriate size
- Students will understand the range, precision, and memory requirements of single and double precision floating point numbers and will know how to select the appropriate precision
- Students will be able to explain the concepts of floating point representation, including range, precision, overflow, and underflow
**Example Activity:**Students will do some simple exercises to illustrate the differences among the data types, including allocating large arrays and checking their sizes; experimentally finding the largest representable number (approximately); and for floating point numbers experimentally finding both the smallest representable positive number (approximately) and the machine precision

**Understand numerical errors:[-]**- Students will be able to describe various kinds of numerical errors (roundoff, overflow, underflow)
- Students will be able to describe absolute and relative error
- Students will discuss error propagation
- Students will understand loss of significance and methods to avoid loss of significance
- Students will be able to describe the effect of problem conditioning or sensitivity on the correctness of a computed solution
**Example Activity:**Students will experiment with solving systems of linear equations involving well-conditioned and ill-conditioned matrices

**Students will be introduced to software engineering best practices:[-]**- Students will understand how to define software requirements
- Students will be introduced to the concepts of software design
- Students will be introduced to the concepts of unit and regression testing
- Students will understand the purpose and concepts of source control
- Students will use source control for a class project
- Students will include useful comments in their code
- Students will format code consistently
- Students will understand and use the build process
**Example Activity:**Students will review the software design process with a case study of a computational science code

**Ability to identify efficient file formats:[-]**- Students will understand the difference between text and binary file formats and the uses and limitations of each
- Students will be able to estimate output file sizes and storage space requirements for the results of a simulation in their area of interest
**Example Activity:**Students will do textbook style exercises to estimate storage requirements and data transfer times for storing and moving large files in both text and binary formats

**Create a program that uses at least one widely used numerical library:[-]**- Students will demonstrate how to include the appropriate header files
- Students will demonstrate how to call the library routines, pass in arguments, and use the output values
- Students will demonstrate how to compile, link and run the program
- Students will understand the effect of environment variables or other settings on library performance
- Students will be able to discuss library quality: differences between excellent and questionable libraries
**Example Activity:**Students will compare the performance of naively coded matrix multiply to that of a well-crafted library implementation for a variety of matrix sizes

**Students will be introduced to the concepts of algorithm complexity:[-]**- Students will understand at the overview level the concept of algorithm complexity and will know how some important algorithms compare to each other
- Students will be aware of big-O notation for algorithm complexity
**Example Activity:**Students will do textbook problems on complexity analysis for common linear algebra and search algorithms

**Students will be introduced to serial program optimization concepts and techniques:[-]**- Students will understand basic compiler optimization options and how to use them
- Students will experiment with compiler optimization flags to see the effect on performance of selected codes
- Students will understand the dangers of using compiler optimizations on floating-point code, where common optimizations can change results
- Students will use timing functions or profiling tools to measure the time spent in different sections of their code
- Students will understand at an overview level the concepts of efficient cache utilization including unit stride and cache reuse
- Students will explore these concepts by examining sample codes that have different memory access patterns and measuring the execution time of the different versions. Sample code may include matrix-matrix multiplication
**Example Activity:**Students will experiment with simple example codes as outlined above

**Students will understand verification and validation principles:[-]**- For this competency, students may use a simple model from their field of study or an example provided by the instructor
- Students will develop test cases to verify that their programs correctly implement their models
- Students will validate their models using one or more of the following approaches: a) identify simple or analytical problems that can be used to validate a new method; b) make a statistical comparison to experimental data; c) use a statistical comparison to previous approaches or other models
**Example Activity:**Students will experiment with a simple model as outlined above and write a brief report explaining their model, approach to verification and validation and the results

**Students will be introduced to Monte Carlo methods:[-]**

### Area 2: Intro to High Performance Scientific Computing [-]

**Prereqs - Intermediate-Level Scientific Computing:[-]**- This area will use either Fortran or C/C++ and addresses HPC and parallel computing topics
- The emphasis is on concepts rather than expert-level parallel programming
- The course is intended for all users of HPC systems, not jlinearust developers
- Students should understand at a conceptual level the various forms of parallel computing and parallel programming models
- This course involves concepts that are to a large degree independent of specific technologies

**Students will be introduced to parallel architectures and execution models:[-]**- Students will understand the concepts of the Single-Instruction-Multiple-Data (SIMD) execution model
- They will understand at a conceptual level the architectures associated with this model (streaming and vector processors, including GPUs)
- Students will understand the concepts of the Multiple-Instruction-Multiple-Data (MIMD) execution model
- They will understand at a conceptual level the architectures associated with this model (multicore processors, clusters)
- Students will understand the concepts of the Single-Program-Multiple-Data (SPMD) execution model
- They will understand that SPMD is a subset of the MIMD model used by MPI, UPC and Co-Array FORTRAN applications
**Example Activity:**Students will take a brief quiz on this terminology

**Students will be introduced to memory models for parallel programming:[-]**- Students will understand the concepts of shared and distributed memory and their advantages and disadvantages
- Students will be aware of the differences in programming approaches for shared and distributed memory models
- Students will be introduced to the concept of a global address space with distributed memory
**Example Activity:**Students will take a brief quiz on this terminology

**Students will understand the principles of how to match algorithms, applications, and architectures:[-]**- Students will be introduced to the concepts of data parallelism, functional parallelism, and task-level parallelism (embarrassingly parallel problems)
- Students will understand which types of algorithms parallelize well, at what granularity they are parallelized, and how this relates to different architectures
- Students will understand what algorithmic and program constructs work well with each of the execution models introduced (SIMD, MIMD, SPMD)
**Example Activity:**Students will do textbook type problems, in which they will identify appropriate models and target architectures for different types of problems

**Students will understand the concept of application scalability:[-]**- Students will be able to define weak scalability and give examples of problems that are weakly scalable
- Students will be able to define strong scalability and give examples of problems that are weakly scalable
- Students will be aware of the factors that limit scalability
- Students will understand the limits of parallelism as described by Amdahl's law
**Example Activity:**Students will do textbook exercises to compute speedup, serial fraction, etc.

**Students will understand code performance metrics:[-]**- Students will know how to measure, interpret, and report the performance of their code
- Students will know how to measure, interpret, and report speedup as the number of processors is increased
**Example Activity:**Students will run a sample parallel program on varying numbers of processors and will calculate speedup

**Students will be introduced to parallel programming methods and concepts:[-]**- Students will be able to identify parallelism in an application and discuss approaches for exploiting it
- Students will be introduced to the concept of message passing and its implementation using MPI
- Students will be introduced to the concept of multithreading and its implementation using OpenMP
- Students will be introduced to the concept of vectorization and will understand how to take advantage of it using compiler flags
- Students will be introduced to the concepts of streaming and many-core accelerators such as GPUs
- Students will be aware of various concurrency issues, the problems they can cause, and some mechanisms for avoiding them, including race conditions, deadlocks, critical sections, and data dependencies
- Students will discuss the generation of parallel random number streams
**Example Activity:**Students will review and run MPI and OpenMP example programs on parallel computer system

**Data Intensive Computing:[-]**- Students will be able to define data intensive computing and explain how it differs from traditional high performance computing
- Students will describe an application, preferably related to their field of study, that involves data intensive computing
- Students will be able to transfer a dataset from one system to another using a parallel data transfer method
- Students will understand how data intensive computing impacts algorithm design
**Example Activity:**Students will experiment with file transfer tools such as gridftp

**Data management:[-]**- Students will be familiar with the file compression and archiving tools available on their systems
- Students will know what tool to use to decompress a file in any of the formats commonly used in the HPC community
- Students will understand how metadata helps make sense of data and will recognize at least one metadata format, possibly XML
**Example Activity:**Students will read and write HDF5 files with a provided application and will use HDF5 commands to inspect the files

**Understanding fault tolerance:[-]**- Students will be able to explain the concept of mean time between failures (MTBF)
- Students will be able to explain the concept of fault tolerance
- Students will identify some causes of failures in HPC systems
- Students will understand why fault tolerance is a bigger issue on extremely large scale systems than on smaller systems
**Example Activity:**Students will do textbook exercises to compute MTBF of computer system components

**Students will understand basic scientific visualization concepts[-]**- Students will understand the difference between scientific visualization and information visualization
- Students will understand basic techniques to visualize scalar fields
- Students will understand basic techniques to visualize vector fields
**Example Activity:**Students will display a two-dimensional scalar dataset using a widely available visualization tool, for example, MATLAB, Excel, or VisIt

### Specialty Area 1: HPC Software Development [-]

**Prereqs - Intro to High Performance Scientific Computing:[-]**- This area is intended for researchers who develop HPC software for their own use or community use
- Students coming out of this course should understand the topics of the first two areas at a mastery level in addition to mastery of the competencies listed here
- There is some redundancy in the competencies listed for this area and those above; this area is intended to provide more depth
- This area includes more advanced programming skills, software engineering practices, and parallel programming
- Students should understand and have a working knowledge of alternative approaches to parallel programming and how they relate to current and emerging parallel programming models

**Advanced programming skills[-]**- Students will be able to use advanced language features that support scientific computing
- Students will be able to create and use user-defined types, structures, classes, or similar mechanisms in their primary HPC programming language
- Students will be familiar with object-oriented design and programming
- Students will demonstrate the use of gdb or some other commonly available debugger, including setting breakpoints, stepping, and examining the contents of variables

**Students will understand and follow software engineering best practices:[-]**- Students will define software requirements for one or more of their class projects
- Students will be required to produce a software design for one or more of their class projects
- Students will use source control for all their class projects and will understand the capabilities of the source control tool they are using
- Students will create a makefile for each of their programs or use some other method for one-step builds
- Students will develop unit and regression test cases for their programs
- Students will be introduced to testing frameworks

**Students will demonstrate techniques for understanding and interpreting existing code:[-]****Students will be proficient in serial program optimization:[-]**- Students will correctly use compiler flags to optimize their code
- They will understand the benefits and limitations of the compiler optimizations
- Students will understand the dangers of using compiler optimizations on floating-point code, where common optimizations can change results
- Students will use timing functions and code profiling tools such as gprof to measure the time spent in different sections of their code
- Students will understand the concepts of efficient cache utilization including unit stride and cache reuse
**Example Activity:**Students will explore these concepts by modifying sample code to have different memory access patterns and measuring the execution time of the different versions. Sample code may include matrix-matrix multiplication.

**Students will use parallel concepts/algorithms in developing software:[-]**- Students will complete a class project using each of these models: SIMD, MIMD, SPMD
- Students will discuss the architectural concepts associated with each of these models and give a current example of each
- Students will discuss the algorithmic and program constructs that work well with each
- Students will understand which types of algorithms parallelize well, at what granularity they are parallelized, and how this relates to different architectures
- Students will understand various concurrency issues, the problems they can cause, and how to handle them properly, including race conditions, deadlocks, critical sections, and data dependencies

**Students will demonstrate skill in application scaling:[-]**- Students will be able to define weak and strong scalability and give examples of algorithms or applications exhibiting each
- Students will be able to identify parallelism in an application and decide on the best approach to exploit it
- Students will describe several factors that limit scalability
- Students will complete a class project in which they measure speedup of their code and present the information in a meaningful way
- Students will explain the limits of parallelism as described by Amdahl's law and give examples

**Students will be able to write a parallel program using MPI:[-]**- Students will understand the concepts of message passing and communicators
- Students will know how to use point-to-point communication and collective communication functions
- Students will be able to explain the semantics of blocking and non-blocking send and receive operations
- Students will be able to explain the standard, buffered, synchronous, and ready communication modes
- Students will recognize common errors that can cause a program to deadlock and will suggest ways to correct them
- Students will understand why a program that works correctly with one MPI implementation may fail when run with another one
- Students will be introduced to techniques for communicating non-contiguous data or mixed data types
- Students will be introduced to virtual topologies in MPI
- Students will understand how data distribution affects communication loads (data movement)
- Students will investigate the use of non-blocking and one-sided communication for overlapping computation and communication

**Students will be able to write a program using OpenMP:[-]**- Students will use OpenMP for loop-level parallelization in an example program
- Students will understand what types of data dependencies inhibit loop parallelization
- Students will be introduced to additional worksharing constructs in OpenMP including Parallel Regions and Parallel Sections
- Students will understand the concepts of private, shared and reduction variables in OpenMP and will be able to correctly determine when to use each attribute
- Students will be introduced to synchronization mechanisms available in OpenMP
- Students will recognize race conditions and use synchronization to prevent errors
- Students will understand the relationship between threads and cores and how OpenMP relates to both
- Students will understand OpenMP scheduling options and will experiment with them in class exercise

**Students will be introduced to hybrid or mixed mode MPI-OpenMP:[-]**- Students should know why and when it is appropriate to mix MPI and OpenMP
- Students will be able to explain the benefits and pitfalls of mixing MPI and OpenMP
- Students will create and run a program that combines MPI and OpenMP parallelization techniques
- Students will be able to explain the threading models in the MPI standard and the strengths and pitfalls of each

**Students will learn techniques for load balancing:[-]**- Students will be introduced to data distribution strategies such as blocked partitioning, cyclic partitioning, graph partitioning, etc.
- Students will understand the effect of data distribution on load balance
- Students will know how to measure load imbalance
- Students will understand static and dynamic load balancing

**Students will be introduced to frameworks for large-scale parallel code development:[-]****Students will be introduced to many-core programming:[-]**- Students will explain the difference between many-core and multi-core computing
- Students will describe the characteristics, benefits, and programming challenges of at least one many-core device
- Students will run a sample CUDA or OpenCL (or similar language) program on a GPU or other many-core device and will compare execution time to that of a CPU version of the same computation

**Students will be able to write a simple Partitioned Global Address Space (PGAS) parallel program:[-]****Students will understand how to implement checkpointing:[-]****Students will demonstrate skill in debugging parallel programs:[-]**- Students will be proficient in the use of a parallel debugger, for example Totalview, DDT, or the debugging capabilities of an IDE they are using
- Students will be aware of some common types of bugs in parallel programs and will understand how to find and correct them
- Students will understand how concurrency issues outlined earlier (data dependencies, race conditions, deadlocks) manifest as bugs

**Students will be able to improve the efficiency of a program:[-]**- Students will know how to improve efficiency by improving data locality and memory access patterns
- Students will know how to improve efficiency by finding hotspots in their code and optimizing small sections of code
- Students will understand the factors that affect the performance of a parallel program
- Students will be introduced to techniques for efficient scheduling: detecting load imbalance, appropriately sized tasks, etc.
- Students will be introduced for techniques for reducing communication overhead such as message coalescing and latency hiding

**Students will know how to use performance analysis tools:[-]**- Students will understand how to use performance analysis tools such as gprof, IPM and TAU to find performance problems in their code
- Students will be able to interpret the output of performance analysis tools
- Students will be able to use profiling tools such as Oprofile which report Hardware Performance Counter data to measure CPU performance metrics such as cache misses

**Students will understand HPC workflows and know how to automate them:[-]****Students will understand parallel I/O:[-]**- Students will understand how to implement parallel I/O, for example using MPI I/O, HDF5, or ADIOS
- Students will create a program that uses MPI-IO or some other library to perform parallel I/O
- Students will understand the effect of network topology on parallel file I/O
- Students will investigate leveraging parallel filesystems for data-intensive applications