Replication Data for: Realizing Joint Extreme-Scale Simulations on Multiple Supercomputers - Two Superfacility Case Studies

Version 1.0

Pollinger, Theresa; Van Craen, Alexander; Offenhäuser, Philipp, 2024, "Replication Data for: Realizing Joint Extreme-Scale Simulations on Multiple Supercomputers - Two Superfacility Case Studies", https://doi.org/10.18419/darus-3707, DaRUS, V1

Learn about Data Citation Standards.

Contact Owner

Dataset Metrics

461 Downloads

Description	This data repository contains input and output files for large-scale experiments conducted on the three German national supercomputers HAWK, SuperMUC-NG, and JUWELS. More structure can be seen when switching to "Tree" view below. Then, the files are structured into three folders (`io-benchmark`, `two-systems, `three-systems`), and the outputs were generated with the following software versions: IO-Benchmark: DisCoTec version e8fad8ae, inputs and outputs both compressed into the same file. Includes file preparation scripts and darshan outputs. The darshan files are also uploaded to https://hpcioanalysis.zdv.uni-mainz.de/traces (search `DisCoTec` in "Application" or `HAWK` in "Cluster), where they can be explored interactively. Two-system experiment: DisCoTec version e81a10589, UFTP copy scripts (in the DisCoTec repository) version 125ce10a5. Three-system experiment: DisCoTec version 46127fe94c1, UFTP copy scripts (in the DisCoTec repository) version 58d1b9ba9. The input files were generated with https://github.com/SGpp/DisCoTec-combischeme-utilities version 88d11fb0, exact commands can be seen from the scrips in this repository. After the combination scheme is assigned to systems, the conjoint file sizes need to be generated as detailed in https://github.com/SGpp/DisCoTec/blob/main/tools/subspace_writer/README.md . The resulting `schemejson` and `conjoint.sizes` files are part of the input generator folders uploaded here. The measurements for the I/O benchmark were taken on October 6 2023. The measurements for the two-system setup were taken on September 4 2023 between 14:30 and 15:30. The measurements for the three-system setup were taken on November 8 2023 between 13:30 and 15:00. Raw data are uploaded as .tar.gz files in the respective folder.
Subject	Computer and Information Science; Physics
Keyword	Coupling HPC Systems, Combination Technique, Multi-Level Methods, Higher-Dimensional Simulation, UFTP
License/Data Use Agreement	CC BY 4.0

Change View

Table

Tree

Filter by

	1 to 10 of 34 Files	Download
	io-benchmark-discotec.tar.gz io-benchmark/Gzip Archive - 424.4 MB Published Jun 28, 2024 15 Downloads MD5: afdd8eaac59ec32243418ac11e987860	Access File File Access Public Download Options Gzip Archive Download Metadata Data File Citation EndNote XML RIS BibTeX
	Preparatory_Benchmarks_Multi-Files.xlsx io-benchmark/MS Excel Spreadsheet - 65.4 KB Published Jun 28, 2024 13 Downloads MD5: ef24674b05e4d2ed9ae0586d632ba319	Access File File Access Public Download Options MS Excel Spreadsheet Download Metadata Data File Citation EndNote XML RIS BibTeX
	XXL-Day_20231006.xlsx io-benchmark/MS Excel Spreadsheet - 10.5 KB Published Jun 28, 2024 12 Downloads MD5: bef7ef2a35b7929c8d3a707caf5b8105	Access File File Access Public Download Options MS Excel Spreadsheet Download Metadata Data File Citation EndNote XML RIS BibTeX
	cappuccino_day_hawk_3-systems_16x2048.tar.gz three-systems/Gzip Archive - 6.3 MB Published Jun 28, 2024 12 Downloads MD5: c9424e274b3030eda09227cbc5a5199c output	Access File File Access Public Download Options Gzip Archive Download Metadata Data File Citation EndNote XML RIS BibTeX
	cappuccino_day_juwels_3-systems_3x2048.tar.gz three-systems/Gzip Archive - 4.3 MB Published Jun 28, 2024 14 Downloads MD5: ee93e6d3a643ebb01eb93fa15289663e output	Access File File Access Public Download Options Gzip Archive Download Metadata Data File Citation EndNote XML RIS BibTeX
	cappuccino_day_ng_3-systems_8x2048.tar.gz three-systems/Gzip Archive - 7.3 MB Published Jun 28, 2024 15 Downloads MD5: a1736788a3f8fb830d6d066cbd492c40 output	Access File File Access Public Download Options Gzip Archive Download Metadata Data File Citation EndNote XML RIS BibTeX
	compile_hawk_three-system.tar three-systems/TAR Archive - 210.0 KB Published Jun 28, 2024 10 Downloads MD5: be9bdde54055a691c6b98d89388c7cdf	Access File File Access Public Download Options TAR Archive Download Metadata Data File Citation EndNote XML RIS BibTeX
	compile_juwels_three-system.tar three-systems/TAR Archive - 210.0 KB Published Jun 28, 2024 15 Downloads MD5: 7fe106cdfeae656e50895c945d4c69f3	Access File File Access Public Download Options TAR Archive Download Metadata Data File Citation EndNote XML RIS BibTeX
	compile_ng_three-system.tar three-systems/TAR Archive - 210.0 KB Published Jun 28, 2024 13 Downloads MD5: ad85b382fc46cceb61ab92301414ee6a	Access File File Access Public Download Options TAR Archive Download Metadata Data File Citation EndNote XML RIS BibTeX
	generate_metis_three-system.tar three-systems/TAR Archive - 20.0 KB Published Jun 28, 2024 15 Downloads MD5: 3da80eeb8a4defffbffefecd0ae1c5ff input	Access File File Access Public Download Options TAR Archive Download Metadata Data File Citation EndNote XML RIS BibTeX

Citation Metadata

Persistent Identifier	doi:10.18419/darus-3707
Publication Date	2024-06-28
Title	Replication Data for: Realizing Joint Extreme-Scale Simulations on Multiple Supercomputers - Two Superfacility Case Studies
Author	Pollinger, Theresa (Universität Stuttgart) - ORCID: 0000-0002-0186-4340 Van Craen, Alexander (Universität Stuttgart) - ORCID: 0000-0002-3336-7226 Offenhäuser, Philipp (HPE) - ORCID: 0009-0001-1674-7980
Point of Contact	Use email button above to contact. Pollinger, Theresa (Universität Stuttgart)
Description	This data repository contains input and output files for large-scale experiments conducted on the three German national supercomputers HAWK, SuperMUC-NG, and JUWELS. More structure can be seen when switching to "Tree" view below. Then, the files are structured into three folders (`io-benchmark`, `two-systems, `three-systems`), and the outputs were generated with the following software versions: IO-Benchmark: DisCoTec version e8fad8ae, inputs and outputs both compressed into the same file. Includes file preparation scripts and darshan outputs. The darshan files are also uploaded to https://hpcioanalysis.zdv.uni-mainz.de/traces (search `DisCoTec` in "Application" or `HAWK` in "Cluster), where they can be explored interactively. Two-system experiment: DisCoTec version e81a10589, UFTP copy scripts (in the DisCoTec repository) version 125ce10a5. Three-system experiment: DisCoTec version 46127fe94c1, UFTP copy scripts (in the DisCoTec repository) version 58d1b9ba9. The input files were generated with https://github.com/SGpp/DisCoTec-combischeme-utilities version 88d11fb0, exact commands can be seen from the scrips in this repository. After the combination scheme is assigned to systems, the conjoint file sizes need to be generated as detailed in https://github.com/SGpp/DisCoTec/blob/main/tools/subspace_writer/README.md . The resulting `schemejson` and `conjoint.sizes` files are part of the input generator folders uploaded here. The measurements for the I/O benchmark were taken on October 6 2023. The measurements for the two-system setup were taken on September 4 2023 between 14:30 and 15:30. The measurements for the three-system setup were taken on November 8 2023 between 13:30 and 15:00. Raw data are uploaded as .tar.gz files in the respective folder.
Subject	Computer and Information Science; Physics
Keyword	Coupling HPC Systems Combination Technique Multi-Level Methods Higher-Dimensional Simulation UFTP
Topic Classification	Computer Science (DFGFO) https://github.com/tibonto/dfgfo/409
Language	English
Producer	Pollinger, Theresa Van Craen, Alexander
Funding Information	Stuttgart Center for Simulation Science (SimTech) Gauss Centre for Supercomputing e.V.: WideDisCoTec Gauss Centre for Supercomputing e.V.: WideDisCoTecLRZ Gauss Centre for Supercomputing e.V.: widediscotecjsc
Depositor	Pollinger, Theresa
Deposit Date	2023-09-08
Date of Collection	Start Date: 2023-10-06 ; End Date: 2023-10-06 Start Date: 2023-09-04 ; End Date: 2023-09-04 Start Date: 2023-11-08 ; End Date: 2023-11-08
Software	DisCoTec, Version: e81a10589 (first case study) and 46127fe94c1 (second case study)
Related Dataset	Pollinger, Theresa, 2023, "Replication Data for: Leveraging the compute power of two HPC systems for higher-dimensional grid-based simulations with the widely-distributed sparse grid combination technique", DaRUS, https://doi.org/10.18419/darus-3393
Other Reference	https://hpcioanalysis.zdv.uni-mainz.de/traces

Process Metadata

Software	DisCoTec (https://github.com/SGpp/DisCoTec) LGPL-3.0 DisCoTec-combischeme-utilities Version: 88d11fb0 (https://github.com/SGpp/DisCoTec-combischeme-utilities) GPL-3.0

Dataset Terms

License/Data Use Agreement

Our Community Norms as well as good scientific practices expect that proper credit is given via citation. Please use the data citation shown on the dataset page.

Creative Commons Attribution 4.0 International License. CC BY 4.0

Restricted Files + Terms of Access

Dataset Version	Summary	Contributors	Published on
No records found.

Edit File

This file has already been deleted (or replaced) in the current version. It may not be edited.

Restrict Access

Restricting limits access to published files. People who want to use the restricted files can request access by default. If you disable request access, you must add information about access to the Terms of Access field.

Learn about restricting files and dataset access in the User Guide.

Request Access

Enable access request

You must enable request access or add terms of access to restrict file access.

Terms of Access for Restricted Files

Save Changes

Edit Embargo

The selected file or files have already been published. Contact an administrator to change the embargo date or reason of the file or files.

Edit Retention Period

The selected file or files have already been published. Contact an administrator to change the retention period date or reason of the file or files.

Delete Files

The file will be deleted after you click on the Delete button.

Files will not be removed from previously published versions of the dataset.

Select File(s)

Please select one or more files.

Share Dataset

Share this dataset on your favorite social media networks.

Continue

Dataset Citations

Citations for this dataset are retrieved from Crossref via DataCite using Make Data Count standards. For more information about dataset metrics, please refer to the User Guide.

Sorry, no citations were found.

Inaccessible Files Selected

The selected file(s) may not be downloaded because you have not been granted access or the file(s) have a retention period that has expired or the files can only be transferred via Globus.

You may request access to any restricted file(s) by clicking the Request Access button.

Ineligible Files Selected

The selected file(s) may not be transferred because you have not been granted access or the file(s) have a retention period that has expired or the files are not Globus accessible.

You may request access to any restricted file(s) by clicking the Request Access button.

Download Options

The files selected are too large to download as a ZIP.

You can select individual files that are below the 1.9 GB download limit from the files table, or use the Data Access API for programmatic access to the files.

Select File(s)

Please select a file or files to be downloaded.

Inaccessible Files Selected

The selected file(s) may not be downloaded because you have not been granted access or the file(s) have a retention period that has expired.

Click Continue to download the files you have access to download.

Ineligible Files Selected

Some file(s) cannot be transferred. (They are restricted, embargoed, with an expired retention period, or not Globus accessible.)

Click Continue to transfer the elligible files.

Delete Dataset

Are you sure you want to delete this dataset and all of its files? You cannot undelete this dataset.

Delete Draft Version

Are you sure you want to delete this draft version? Files will be reverted to the most recently published version. You cannot undelete this draft.

Unpublished Dataset Private URL

Private URL can only be used with unpublished versions of datasets.

Unpublished Dataset Private URL

Are you sure you want to disable the Private URL? If you have shared the Private URL with others they will no longer be able to use it to access your unpublished dataset.

Delete Files

The file(s) will be deleted after you click on the Delete button.

Files will not be removed from previously published versions of the dataset.

Compute

This dataset contains restricted files you may not compute on because you have not been granted access.

Deaccession Dataset

Are you sure you want to deaccession? This is permanent and the selected version(s) will no longer be viewable by the public.

Deaccession Dataset

Are you sure you want to deaccession this dataset? This is permanent an it will no longer be viewable by the public.

Version Differences Details

Please select two versions to view the differences.

Version Differences Details

Version:
Last Updated:

Select File(s)

Please select a file or files for access request.

Select File(s)

Embargoed files cannot be accessed. Please select an unembargoed file or files for your access request.

Edit Tags

Select existing file tags or create new tags to describe your files. Each file can have more than one tag.

Request Access

You need to Log In to request access.

Dataset Terms

Please confirm and/or complete the information needed below in order to request access to files in this dataset.

This dataset is made available under the following terms. Please confirm and/or complete the information needed below in order to continue.

License/Data Use Agreement

Our Community Norms as well as good scientific practices expect that proper credit is given via citation. Please use the data citation shown on the dataset page.

Creative Commons Attribution 4.0 International License. CC BY 4.0

Preview Guestbook

Upon downloading files the guestbook asks for the following information.

Guestbook Name

Collected Data

Account Information

Package File Download

Use the Download URL in a Wget command or a download manager to download this package file. Download via web browser is not recommended. User Guide - Downloading a Dataverse Package via URL

Download URL

https://darus.uni-stuttgart.de/api/access/datafile/

Compute Batch

Clear Batch

Dataset	Persistent Identifier	Change Compute Batch

Compute Batch

Submit for Review

Please use the publication checklist for authors ( https://www.izus.uni-stuttgart.de/en/fokus/darus/publication/ ) to ensure publication readiness. You will not be able to make changes to this dataset while it is in review.

Publish Dataset

Are you sure you want to republish this dataset?

Select if this is a minor or major version update.

Minor Release (1.1)

Major Release (2.0)

Publish Dataset

This dataset cannot be published until Scientific Computing is published by its administrator.

Publish Dataset

This dataset cannot be published until Scientific Computing and Institute for Parallel and Distributed Systems are published.

Return to Author

Return this dataset to contributor for modification. The reason for return entered below will be sent by email to the author.