--- title: 'Use of Storage' taxonomy: category: - docs --- [toc] ## Home Directory, Large Volume Disk (LARGE) and High-Speed Storage{#home_and_large} All users can use each home directory. Users of the Personal Course, Group Course, and Private Cluster Course can use large volume disk (LARGE). If you deal with a large-scale data in the Personal Course, Group Course, and Private Cluster Course, please use large volume disk (LARGE). If you are using a program that requires high I/O performance, you can improve performance by using high-speed storage. <!-- ! We have started the trial offer of the High-speed storage from September, 2023. As a trial period, the personal course users can use 500 GB per user and group course users can use 500 GB per node free of charge during FY 2023. --> ## Home Directory{#home} The home directory path is shown below: You can also use it as a location to install software for your own use. ![](filesystem_05.png?resize=400) - The available capacity is 100 GB. (User Quota)] - The "b" part in the path is the first letter of the user number. - We save backups on a regular basis, approximately once a week. ## Large Volume Disk(LARGE){#large} Large volume disk (LARGE) is available for the Personal Course, Group Course, and Private Cluster Course. Since the system has high performance by being configured with a large number of storage drives, please use large volume disk for file access with large volume files or parallel computing. <!-- ! The storage capacity limits during the FY2022 trial operation are inherited from the limits in the first half of FY2022. --> ### For Personal Course{#personal} LARGE configuration for personal course is shown below. ![](filesystem_06.png?resize=300) - The total available capacity is 3 TB. - It is configured with two separate file systems. - By default, /LARGE1 is used as backup area. ### For Group Course and Private Cluster Course{#group} LARGE configuration for group course and private cluster course is shown below. Please make a directory for each user under the group directory. ![](filesystem_07.png?resize=300) - The available capacity will be allocated according to the storage capacity and the additional computing resources you request. - It is configured with two separate file systems. - By default, /LARGE1 is used as backup area. ### Backup Settings {#large_backup} Users of the personal course or payment manager of the group course can change the backup settings for large volume disk. If you emphasize data redundancy, enable backup; if you emphasize available capacity, disable backup. The frequency of backups is targeted to be about once a week for small-scale use and once a month for larger groups with tens of millions of files. Please refer to [Backup Settings](/group#group_backup) for the setup procedure. ### Changes from the previous system {#old_large} The previous system consisted of four LARGES of /LARGE0, /LARGE1, /LARGE2, and /LARGE3, however this system have been changed to two LARGES of /LARGE0 and /LARGE1. Data in /LARGE2 is put together in /LARGE0 and data in /LARGE3 is put together in /LARGE1, however symbolic link is also set so that you can access files with conventional paths. Please note that the symbolic links may be deleted in the future, so please update your paths accordingly. ```bash $ ls -ld /LARGE? drwxr-xr-x 201 root root 12288 Nov 18 11:46 /LARGE0 drwxr-xr-x 202 root root 12288 Nov 7 18:17 /LARGE1 lrwxrwxrwx 1 root root 7 Nov 8 10:30 /LARGE2 -> /LARGE0 lrwxrwxrwx 1 root root 7 Nov 8 10:31 /LARGE3 -> /LARGE1 ``` ## High-Speed Storage(FAST){#fast} High-speed storage is available for Personal Course, Group Course, and Private Cluster Course. High-speed storage is configured with SSDs and has high throughput, however the capacity is not that large. To take advantage of the high throughput, please use it for input/output of the calculations and avoid using it for long-term data storage. When you need to save calculation results, please move them to [Large Volume Disk(LARGE)](#large) to take advantage of the capacity and the performance of the high-speed storage. ! In FY2024, we provide 500 GB per person for personal course users and 400 GB to 1000 GB per node (depending on the type) for group course users as a trial period. Please check [quota -p](#group_quota) for details on the available capacity. High-speed storage consists of the following. ![](filesystem_08.png?resize=550) * The available capacity will be sized according to the computing resources you apply for and the additional capacity of the Large Volume Disk. ### Backup{#fast_backup} High-speed storage (FAST) does not take backups. Please make your own backups as appropriate. In addition, they are not subject to data migration at the time of the system renewal. ## Confirmation of the usage status (quota command){#quota_command} You can check the current usage status of the disk with the **quota** command. - kbytes: File capacity in use(KB) - quota: File capacity/Limit of the file number (soft limit) - limit: File capacity/Absolute limit of the file number (hard limit) - grace: Allowable period over the limit (over the soft limit) - files: Number of files in use ### Usage status of HOME directory{#user_quota} ```nohighlight $ quota -u b59999 Disk quotas for user b59999 (uid 59999): Filesystem kbytes quota limit grace files quota limit grace /home 17859980 100000000 110000000 - 69218 1000000 1100000 - ``` ### Confirmation of the usage status of large-volume disk (LARGE){#group_quota} LARGE limits the capacity with Project Quota. Specify the -p option to the quota command and execute. ```nohighlight $ quota -p Disk quotas for project gr19999 (pid 19999): Filesystem kbytes quota limit grace files quota limit grace /LARGE0 2088651052 8000000000 8050000000 - 90543 16000000 16100000 - /LARGE1 1967559464 8000000000 8050000000 - 90544 16000000 16100000 - /FAST 762012 500000000 505000000 - 4544 16000000 16100000 - ``` <!-- ### グループコースでのファイル作成について{#group_permission} グループコースでは、大容量ディスクの使用量を、グループ所有権が当該グループ名に設定されているファイルをカウントすることで算出しています。 一方、ユーザが作るファイルのグループ所有権は、デフォルトではユーザ名となっています。 この状態では、グループのファイルとしてはカウントされません。 利便性のために、この状態でも大容量ディスクにファイルを作成できますが、全ユーザに一律で容量制限(1TB)をかけております。 大容量ディスクでグループ所有権がユーザ名となっているファイルの総量を確認するには、`quota -p`コマンドを使用します。 ```nohighlight $ quota -p (中略) Disk quotas for grp b59999 (gid 59999): Filesystem blocks quota limit grace files quota limit grace /home 6758940 0 0 - 87549 0 0 - /LARGE0 1112134124 1000000000 1050000000 - 3018 1000000 1100000 - ``` この例では、w00001ユーザについて、グループ所有権がユーザ名となっているファイルがLARGE0にどれだけ置かれているかが表示されています。 blocksが既にquotaを超過しているため、これ以上グループ所有権がユーザ名のファイルは置けない状況です。 --> <!--- ## ファイルI/O{#file_io} ### Lustreストライピングの設定{#stripe} Lustreは並列のファイルシステムで、複数のディスクデバイスやI/Oサーバを論理的に束ねて、ひとつのファイルシステムを構成しています。デフォルトの状態では、ストライピングは行わない設定になっています。ストライピングの設定を行うことで、シングルプロセス―シングルファイルI/Oで、複数のディスクデバイスに分散させることで高速化できる場合があります。 **ストライプ設定** ・例)st04(ファイル名)のストライピング設定変更(4WAYストライピング) ```nohighlight $ lfs setstripe -s 1048576 -o -1 -c 4 st04 ``` |オプション名 |説明| |-|-| |-c |ストライプ数、ストライプ幅を指定します。| |-o |オフセットを指定します。| |-s |ストライピングサイズを指定します。| **ストライプ設定表示** ・例)st04(ファイル名)のストライピング設定表示 ```nohighlight $ lfs getstripe -v st04 ``` -->