hpc:numericserveruser:storage_classes

Unterschiede

Hier werden die Unterschiede zwischen zwei Versionen angezeigt.

Link zu dieser Vergleichsansicht

Beide Seiten der vorigen Revision Vorhergehende Überarbeitung
hpc:numericserveruser:storage_classes [2020/03/24 17:32] – kraushpc:numericserveruser:storage_classes [2020/03/24 17:43] (aktuell) – kraus
Zeile 1: Zeile 1:
 +====== Storage Guidelines HPC ======
 +\\
  
 +===== Storage Types =====
 +
 +| Storage type | Where/Mount | Mount type | Purpose | Quota | Capacity |
 +| HOME     | <code>/[homes|homes2]/[your_faculty|extern]/[username]</code> | network | Program application data; only small files (very precious data) | <code>~ 2.5 GiB</code> | - |
 +| WORK     | <code>/work/[username]</code> | network | **Permanent** data storage; NO scratch files| NO | - |
 +| BeeGFS  | <code>/fast/[your_directories]</code> | network | **Scratch** Storage of **(very) large** files | NO  | - |
 +| Local Scratch | For batch jobs: <code>$TMPDIR</code> or optionally: <code>/scratch/[your_directories]</code> | node-local FS \\ (shared by all jobs running on node) | Intermediary scratch storage for **small and moderate-large** files | NO | <code>~ 200 - 500 GiB</code> |
 +\\
 +
 +===== Recommendations =====
 +
 +  * ''BeeGFS'' (parallel distributed file system): preferred for **large scratch files (starting from files sizes of a few hundreds of MiB) AND (semi-)permanent storage of scratch files**
 +  * ''Node-local Scratch'': **fast, local FS storage** for small to intermediary sized scratch files up to a size of few GiBs; big amounts of small scratch files should be written here (BeeGFS shows poor performance for files with sizes smaller than a few hundreds of MiB)
 +\\
 +
 +===== General Warning =====
 +
 +  * **Please ensure, that your programs DO NOT write huge amounts of scratch files or data to your WORK (as this will block other users from getting their work done).**
 +  * **Please avoid (permanent) storage of un-needed scratch files and do not copy them to your WORK.**
 +  * **Please write your scratch files to node-local or BeeGFS (depends on size of the files; see section "Recommendations"). If needed, copy the program's checkpoint files from node-local scratch to BeeGFS.**
 +
 +\\
 +<WRAP rightalign>
 +date of revision: 03-24-2020 © kraus
 +</WRAP>