--- title: 'Using File System' taxonomy: category: - docs --- [toc] ## Home Directory and Large Volume Disk (LARGE){#home_and_large} All users can use each home directory. Large volume disk (LARGE) is available in the Personal Course, Group Course, and Private Cluster Course. Use large volume disk (LARGE) when dealing with a large-scale data in the Personal Course, Group Course, and Private Cluster Course. ## Home Directory{#home} The home directory path is shown below: ![](filesystem_04.png) - A capacity of up to 100 GB is available. (User Quota) - The “b” in the path is a placeholder for the first letter of the user ID. ## Large Volume Disk(LARGE){#large} Large volume disk (LARGE) is available in the Personal Course, Group Course, and Private Cluster Course. Large volume disk consists of the pair of /LARGE0 and /LARGE1, and the pair of /LARGE2 and /LARGE3. Either pair is assinged for each personal course user and group. Which pair your course has been assigned is written in the service approval email. ### Configuration for Personal Course{#personal} LARGE configuration for personal course is shown below. ![](filesystem_02.png) - A total capacity of 2TB is available. (Group Quota) - It is divided into two areas. - In the initial setting, one area is configured for backup. - You can make backup settings. For how to set up backups, see [Backup Setting](/group#group_backup). ### Configuration for Group and Private Cluster Course{#group} LARGE configuration for group and private cluster course is shown below.You can create and use your own directory into a directory for the group. ![](filesystem_03.png) - The capacity meets the computing resources that you applied for. - LARGE restricts the capacity in Group Quota. - It is divided into two file systems. - You need to change the current group when putting a file in the LARGE storage. For how to change the current group, see [Changing Current Group](#change_current). - A group manager can change a setting to use a backup area as the normal area. For how to set up, see [information for Group Managers](/group). ## Burstbuffer{#burstbuffer} In the system A/B/C, a structure called Burstbuffer is connected for the purpose of speeding up the file access. System A has the DataWarp of Cray Inc. and system B,C has the IME of DDN Inc. Registration is needed to use the burstbuffers. Please see the details below. * [System A](/filesystem/datawarp) * [System B,C](/filesystem/ime) ## Checking Use Status (quota Command){#quota_command} The **quota** command allows to check the current disk use status. - blocks: File capacity currently in use (KB) - quota: File capacity/number limit (software limit) - limit: File capacity/absolute number limit (hardware limit) - grace: Allowable duration of limit violence (exceeding software limit) - files: A number of files currently in use ### Checking Use Status of Home Directory{#user_quota} ```nohighlight $ quota -u b59999 Disk quotas for user b59999 (uid 12345): Filesystem blocks quota limit grace files quota limit grace /home 17859980 30000000 31000000 - 69218 600000 700000 - /LARGE0 16 1 1 - 4 1 1 - /LARGE1 0 1 1 - 0 1 1 - ``` ### Checking Use Status of Large Volume Disk (LARGE){#group_quota} LARGE system restricts the capacity in Group Quota. You can execute the quota command with the -g option. ```nohighlight $ quota -g Disk quotas for group gr19999 (gid 19999): Filesystem blocks quota limit grace files quota limit grace /home 151808 0 0 - 22 0 0 - /LARGE0 2088651052 8000000000 8050000000 - 90543 16000000 16100000 - /LARGE1 1967559464 8000000000 8050000000 - 90544 16000000 16100000 - ``` ### Notes for creating files{#group_permission} In the group course and private cluster course, before using the large volume disk(LARGE) or creating a file with group authority, please change the current group by the [newgrp command](#change_current) By default, files are created with the group permission set to user name. Such files are not counted as group's files. For users' convenience, users can create files with the group permission set to user name, but such files can be created only up to 1TB on the large volume disk. ## Changing Current Group{#change_current} In the group course and private cluster course, before using the large volume disk(LARGE) or creating a file with group authority, please change the current group by the newgrp command. ```nohighlight ## Checking the current group of UID and GID. $ id uid=59999(b59999) gid=59999(b59999) groups=19999(gr19999),59999(b59999) ## Changing the current group to gr19999. $ newgrp - gr19999 $ id uid=59999(b59999) gid=19999(gr19999) groups=19999(gr19999),59999(b59999) ``` The chdefgrp command is also provided to change the default group. ```nohighlight ## Changing the default group to gr19999. $ chdefgrp gr19999 ``` ### Environment variables after newgrp{#newgrp_env} After using newgrp command, a part of the environment variables which were set by modules before using newgrp command could be disappeared. (e.g. LD_LIBRARY_PATH) `reload_module` command is available to restore the environment variables set by modules after using newgrp command. Example usage: ```nohighlight echo $LD_LIBRARY_PATH /opt/app/intel/impi/2017.2.174/intel64/lib:/opt/app/intel/compilers_and_libraries_2017.2.174/linux/tbb/lib/intel64/gcc4.4:/opt/app/intel/compilers_and_libraries_2017.2.174/linux/mkl/lib/intel64:/opt/app/intel/compilers_and_libraries_2017.2.174/linux/compiler/lib/intel64:/opt/app/intel/compilers_and_libraries_2017.2.174/linux/ipp/../compiler/lib/intel64:/opt/intel/mic/coi/host-linux-release/lib:/opt/intel/mic/myo/lib:/opt/app/intel/compilers_and_libraries_2017.2.174/linux/mpirt/lib/intel64:/opt/app/intel/compilers_and_libraries_2017.2.174/linux/ipp/lib/intel64:/opt/cray/lib64: newgrp gr19999 echo $LD_LIBRARY_PATH /opt/cray/lib64: # The content has been disappeared. reload_module echo $LD_LIBRARY_PATH /opt/app/intel/impi/2017.2.174/intel64/lib:/opt/app/intel/compilers_and_libraries_2017.2.174/linux/tbb/lib/intel64/gcc4.4:/opt/app/intel/compilers_and_libraries_2017.2.174/linux/mkl/lib/intel64:/opt/app/intel/compilers_and_libraries_2017.2.174/linux/compiler/lib/intel64:/opt/app/intel/compilers_and_libraries_2017.2.174/linux/ipp/../compiler/lib/intel64:/opt/intel/mic/coi/host-linux-release/lib:/opt/intel/mic/myo/lib:/opt/app/intel/compilers_and_libraries_2017.2.174/linux/mpirt/lib/intel64:/opt/app/intel/compilers_and_libraries_2017.2.174/linux/ipp/lib/intel64:/opt/cray/lib64: ``` ### The chdefgrp Command Notes{#chdefgrp} - This command changes all default groups of the system A, B, and C. - Changes take around 10 minutes to be reflected. ## File I/O{#file_io} ### Setting Up Lustre File Striping{#stripe} Lustre is the parallel file system which consisted of a logical unit of multiple disk devices and I/O servers. By default, the stripe setting is disabled. Setting up striping enables a single process, a single file I/O. It can distribute the load between multiple disk devices, and help provide high-speed access. **Setting up striping** ・Example: changing the stripe setting of the “st04” file (four-way stripe) ```nohighlight $ lfs setstripe -s 1048576 -o -1 -c 4 st04 ``` |Option |Purpose| |-|-| |-c |Specifies a number of stripes and a stripe width.| |-o |Specifies an offset.| |-s |Specifies a stripe size.| **Displaying a stripe information** ・Example: displaying a stripe information for the “st04” file ```nohighlight $ lfs getstripe -v st04 ```