Prof. Suarez is in sabbatical. This seminar (MA-INF 1322) is given by substitute lecturers in SoSe-2025.

This Website is now up to date. It has been actualized for Summer Semester 2025. Both dates, topics and references are specific for SoSe2025.

High Performance Computing (HPC) refers to the use of large cluster computers to solve scientific and technical problems unsolvable on small-scale systems, and doing it so that the hardware resources are efficiently employed. HPC systems are designed to achieve the maximum computing performance at the lowest possible power consumption. They are parallel computers made of hundreds to thousands of compute nodes connected to each other via a high-speed network. Operating HPC systems requires specific software distributions, programming models, and tools. Applications must be parallelized, meaning they must be programmed so that the problem to be solved is split into many individual operations that can be executed in parallel.

This seminar addresses important topics and challenges on today’s HPC. It is designed to cover a wide range of areas allowing to get a glimpse of the latest developments in specific areas of HPC.

All the above dates are strict. Missing any of those leads to expulsion from the seminar or failure (score 5.0).

The topics below are concrete examples from a variety of current HPC topics. Based on reading the references provided for each topic, and the related ones (papers citing them or cited by them), you should be able to write a focused seminar report:

• Focus paper: shortly describe the general overview (“background” section of your report) and then focus on a specific research (e.g. a specific publication) within the given topic.

• Application period will be open according to the timetable above
• Application is done by writing an email to Dr. Ujjwal Sinha and Dr. Bartosz Kostrzewa, which must contain
  • your name,
  • your email address (@uni-bonn.de),
  • your matriculation number,
  • and your desired subjects: rank 3 or more subjects, indicating for each one the letter (A, B, C…) and the topic name from the list above.
• Please note, that the number of participants is limited to 10 and only the first 10 applications will be considered.
  • You will receive a feedback mail in all cases.
• Subject assignment will be done at the preliminary discussion.
  • The preliminary discussion is mandatory.
  • Absence will lead to exclusion from the seminar.
• Basis registration according to official deadline for Seminars
• Submissions after the given deadlines will not be accepted.

A selection of scientific publications / subjects is given via this website. The subject will be assigned at the preliminary discussion (see section Dates).

During the semester each participant will:

• Create a written report in 3 steps: concept/structure, full version, final version including rebuttal.
• Participate in the double-blind peer review process.
• Create presentation slides (structure, final version) and present them on the final seminar day.

Two intermediate meetings are organized: 1st to discuss the paper concept, 2nd to discuss the outcome of the review process. Further than that there are no group meetings and most communication is done via email. The tutors Dr. Ujjwal Sinha and Dr. Bartosz Kostrzewa will be available for any organizational or content-related questions. Feel free to drop an email anytime, also if if you'd like to setup a meeting.

The course is organized in following steps:

1. step: concept: a first skeleton/concept of the written report must be created and uploaded to EasyChair. This should contain the projected structure of the report as well as a basic description of the contents of each section. Furthermore, it should contain the used literature. To avoid misconceptions and unnecessary work, the conception should be discussed with the tutor before the actual work on the written report is started (a meeting takes place for this).

2. step: full report: after receiving feedback on the concept, the full paper is written and uploaded to EasyChair. The written report must span 8-10 pages (excluding table of contents and bibliography) and must be created using LaTeX ( Report_Template.zip). Generally, it is necessary to make a selection and prioritization of topics discussed in the source literature. The content of the written report should match the later presentation, although a different depth, ordering, and prioritization is possible. It is advantageous to incorporate other scientific sources. A scientific complete reference to all used sources is mandatory. It is expected to critically review the subject and literature at hand. A complete and successful written report is necessary to continue with the seminar. Simple rephrasing will be considered unsuccessful.

3. step: peer-review. Each participant can report conflicts of interests and bid via EasyChair on the papers that she/he is most interested on. Then, two papers are assigned to each participant for review in a double-blind fashion. The reviews of the two papers must be uploaded via EasyBuild. Each participant receives at the end of this process two reviews for her/his paper from two different colleagues, and writes a rebuttal.

• NOTE: For you, the peer-review process contributes to your score in this course based on only two aspects: 1) The quality of the 2 reviews that you provided, and 2) The quality of the rebuttal that you have done to the reviews that you received. For the avoidance of doubt: whether the reviews that you receive from your colleagues are positive or negative has NO impact on your score.

4. step: final report submission: you complete your report by making any changes on the report that are necessary to address the reviews that you received, and include the rebuttal as an Annex to the report. The final report is uploaded to EasyChair.

• Only this final version of the report is graded.
• Presentation slides will not be accepted for review by the tutors if the written report is unsuccessful.

5. step: slides preparation: The projected structure of the presentation slides should be submitted to the tutor in advance to the complete version for discussion. It is possible that the tutor will request further changes to the presentation slides after the submission of the complete slides. Be prepared to incorporate these changes before the seminar takes place.

6. step: presentation: Finally, the slides are presented in a seminar day with participation of all students. All participants will present their subject in a 30 minute presentation during the seminar. After each presentation, there is a 15 minute time slot to discuss the presentation and the subject.

• Attendance to all presentations is mandatory.

NOTE that: Submissions after the given deadlines will not be accepted.

To write your report, please use the following LaTeX template: Report_Template.zip

• Layout and formal requirements: citation style and appropriate citation usage, correct mathematical notion, grammatical correctness, spelling, punctuation, formatting, optical appearance.
• Style and structure: spelling style, well defined technical terms, well structured, concise content representation, correct usage of LaTeX environments and theorems.
• Content: adequate selection and prioritization of the content, usage of additional literature, content related correctness, mathematical correctness, correct definitions / theorems / proofs, suitable self-provided examples, precise phrasing, critical evaluation and discussion of the content.
• Independent work style: preparation of good questions for meetings with the tutor, performing literature search for open questions and a deep understanding of the content, justified prioritization and content selection. Attention: Questions and discussions with the tutor are recommended and welcome. They will not lead to lesser grades. On the contrary, they will typically enhance the overall quality of the submissions. An independent work style means, that you think over your problem on your own in advance to such discussions and that you do not rely on your tutor to make trivial corrections.
• Bibliography: for correct bibliographic referencing, see more information in this Bibliography Guideline

• Review Feedback: thoroughness of the review, structure and content, respectful wording, accurate feedback, objective and reasoned observation, clear recommendations for improvement
• Rebuttal: respectful feedback to reviewers, accurate reply to their comments, comprehensive integration of improvements on the manuscript.

• Content: structure, adequate selection and prioritization of the content, correctness, self-prepared examples, graphics, critical evaluation and discussion
• Presentation: presentations style (free, smooth, adequate and precise phrasing, understandability), reasonable and supportive presentation slides / examples / graphics, that help the audience to understand the problems / definitions / evaluations, timing.
• Some general guidelines: for further information, see this Presentation Guideline

These are the possible sources on the topics of the Seminar. In order to get an overview of the topic that you selected, it is a good approach to start by the given paper(s) and look further into other papers that have cited it, and on its own references (especially those given as related work). If you cannot get access to the papers via the UniBonn library license, please contact the people holding this course (mails at the top of this page).

A - Modular Supercomputing Architecture

• [01] Suarez, E., Eicker, N., & Lippert, T. (2019). Modular Supercomputing Architecture: From Idea to Production. In Contemporary High Performance Computing: From Petascale toward Exascale (Vol. 3, pp. 223–251). FL, USA: CRC Press. ISBN: 9781138487079. https://juser.fz-juelich.de/record/862856.
• [02] Suarez, E., Eicker, N., Moschny, T., & Lippert, T. (2021). Critical Analysis of the Modular Supercomputing Architecture. In Porting applications to a Modular Supercomputer - Experiences from the DEEP-EST project (Vol. 48, pp. 233-245). Jülich: Forschungszentrum Jülich GmbH Zentralbibliothek, Verlag Jülich. https://juser.fz-juelich.de/record/905854.

B – Performance analysis of new processor technologies (Arm)

• [03] Ruhela A. et al. (2024). Performance Analysis of Scientific Applications on an NVIDIA Grace System, In SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 558-566). IEEE. DOI:10.1109/SCW63240.2024.00078, https://www.computer.org/csdl/proceedings-article/sc-workshops/2024/555400a558/23l2DKLR8U8

C – Memory bandwidth and AI

• [04] Gholami, A., Yao, Z., Kim, S., Hooper, C., Mahoney, M. W., & Keutzer, K. (2024). AI and Memory Wall. IEEE Micro, 44(3), 33–39. https://ieeexplore.ieee.org/document/10477550
• [05] K. Kim and M. -j. Park, Present and Future, Challenges of High Bandwith Memory (HBM), 2024 IEEE International Memory Workshop (IMW), Seoul, Korea, Republic of, 2024, pp. 1-4, DOI: 10.1109/IMW59701.2024.10536972

D - Interconnection networks

• [06] Y. Ajima et al. (2018), The Tofu Interconnect D, 2018 IEEE International Conference on Cluster Computing (CLUSTER), pp. 646-654 DOI:CLUSTER.2018.00090, https://www.computer.org/csdl/proceedings-article/cluster/2018/831900a646/17D45WYQJ9c
• [07] Y. Ajima et al. (2011), The Tofu Interconnect, Symposium on High-Performance Interconnects, pp. 87-94, DOI: 10.1109/HOTI.2011.21, https://www.computer.org/csdl/proceedings-article/hoti/2011/4537a087/12OmNAL3B8Q

E – Malleability in HPC systems

• [08] A. Tarraf et al., Malleability in Modern HPC Systems: Current Experiences, Challenges, and Future Opportunities, in IEEE Transactions on Parallel and Distributed Systems, vol. 35, no. 9, pp. 1551-1564, Sept. 2024, DOI: 10.1109/TPDS.2024.3406764

F - Performance portability

• [09] Trott, C. et all, (2022), \\Kokkos 3: Programming Model Extensions for the Exascale Era\\, IEEE Transactions on Parallel and Distributed Systems, DOI: 10.1109/TPDS.2021.3097283, https://www.computer.org/csdl/journal/td/2022/04/09485033/1veoqTgRICA
• [10] Asahi, Y. et al. (2022), \\Performance portable Vlasov code with C++ parallel algorithm\\, 2022 IEEE/ACM International Workshop on Performance, Portability and Productivity in HPC (P3HPC), DOI: 10.1109/P3HPC56579.2022.00012,https://www.computer.org/csdl/proceedings-article/p3hpc/2022/602100a068/1KmFiviTt8k

G - Productivity, portability, performance

• [11] Ziogas, A. N. (2021), \\Productivity, Portability, Performance: Data-Centric Python\\, SC21: International Conference for High Performance Computing, Networking, Storage and Analysis, https://www.computer.org/csdl/proceedings-article/sc/2021/09910042/1HzBK9BqrpC

H - Machine Learning on HPC systems

• [12] S. Farrell et al., MLPerf™ HPC: A Holistic Benchmark Suite for Scientific Machine Learning on HPC Systems, 2021 IEEE/ACM Workshop on Machine Learning in High Performance Computing Environments (MLHPC), St. Louis, MO, USA, 2021, pp. 33-45, DOI: 10.1109/MLHPC54614.2021.00009

I – Deep Neural Networks

• [13] W. Samek, G. Montavon, S. Lapuschkin, C. J. Anders and K. -R. Müller, Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications, in Proceedings of the IEEE, vol. 109, no. 3, pp. 247-278, March 2021, DOI: 10.1109/JPROC.2021.3060483
• [14] B. Hasheminezhad, S. Shirzad, N. Wu, P. Diehl, H. Schulz and H. Kaiser, Towards a Scalable and Distributed Infrastructure for Deep Learning Applications, 2020 IEEE/ACM Fourth Workshop on Deep Learning on Supercomputers (DLS), Atlanta, GA, USA, 2020, pp. 20-30, DOI: 10.1109/DLS51937.2020.00008
• [15] S. Shams, R. Platania, K. Lee and S. -J. Park, Evaluation of Deep Learning Frameworks Over Different HPC Architectures, 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), Atlanta, GA, USA, 2017, pp. 1389-1396, DOI: 10.1109/ICDCS.2017.259

J – I/O in HPC systems

• [16] A. Bagbaba, A Comparative Study of MPI-IO Libraries for Offloading of Collective I/O Tasks, 2021 International Conference on Engineering and Emerging Technologies (ICEET), Istanbul, Turkey, 2021, pp. 1-6, DOI: 10.1109/ICEET53442.2021.9659767
• [17] W. Frings, F. Wolf and V. Petkov, Scalable massively parallel I/O to task-local files, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, Portland, OR, USA, 2009, pp. 1-11, DOI: 10.1145/1654059.1654077

K — Network performance of HPC systems

• [18] Bartelheimer, N., Zhu, Z., & Neuwirth, S. (2024). Automated Network Performance Characterization for HPC Systems. International Journal of Networking and Computing, 14(1), 2–25. Available at http://www.ijnc.org/index.php/ijnc/article/view/312.