---

 title: 'Network Storage Services'

 taxonomy:
     category:
         - docs
 external_links:
     process: true
     no_follow: true
     target: _blank
     mode: active
 media_order: 'Cyberduck_S3Connect.png,portal_afterlogin.png,WinSCP_FTPinit.png,request_passwd.png,generate_passwd.png,Cyberduck_FTPinit.png'
 ---
 
 [toc]
 

 ## 概要
 We have started offering a network storage service to expand our research data storage environment and data collection infrastructure from May 2022.

 You can start using the service by following the procedure from [User Portal](https://web.kudpc.kyoto-u.ac.jp/portal).

 Please note that the maintenance may be performed and service content may be revised without prior notice.

 
 ### Examples

 * Research data sharing by storage linkage with [GakuNin RDM](https://rdm.nii.ac.jp/)
 * Data collection for IoT devices, etc. 
 * Back up of the storage of the supercomputer

 
 ## System Configuration
 Small network storage cluster connected to a network independent from the supercomputer system.
 
 Item | Content
 -----|-------------
 Device | DELL EMC isilon A200 x 4<br>DELL EMC PowerScale F200 x 4
 Storage capacity | 260TB
 Front-end network | 5Gbps
 

 ## Services Provided
 If you have a user ID of Large-scale computer system, you can use the following services free of charge.
 **<span style="color:red">This service does not back up the user's data by the system administrator.</span>Please make backups as necessary to prepare for the loss of the registered data when using this service.**

 
 Item | Content
 -----| -------
 Supported protocols |  FTPS and S3 (Amazon S3 compatible, path format)
 Storage capacity | 100GB
 Transfer capacity | Unlimited

 Backup | None

 FTP account | 1
 S3 account | 1
 Number of S3 Buckets | 1
 Number of S3 requests | Unlimited
 Number of S3 objects | Unlimited
 Tine zone of Device | UTC
 
 ## How to Start

 1. Access the [User Portal](https://web.kudpc.kyoto-u.ac.jp/portal)and log in with your user ID of Large-scale computer system.　

 2. Click "Network Storage Service" in the left column.
 ![portal_afterlogin](portal_afterlogin.png)
 3. Select "Generate (Regenerate)" from the drop-down list surrounded by red when issuing a password used for FTP or blue when issuing a secret key for S3, and agree to the precautions. Then click the "Submit" button. 
 ![request_passwd](request_passwd.png)
 4. When the password/secret key is successfully issued, the password information will be displayed at the top of the screen. Please note that the password information is displayed only **once**.
 ![generate_passwd](generate_passwd.png)
 
 

 ## How to Connect(S3)

 #### Connection Information
 
 Item|Content
 -------| -----------------------------------

 Connection Destination | s3.kudpc.kyoto-u.ac.jp

 Access key | Please check the user portal.
 Secret key | Please check the user portal.
 Bucket name | Please check the user portal.
 
 ### When using awscli
 
 #### How to Install
 Please refer to the [Official Manual](https://docs.aws.amazon.com/ja_jp/cli/latest/userguide/getting-started-install.html).
 The supercomputer system provides module files. Please see [here](../oss#awscli) for details.

 ####  Initial Setting

 ```nohighlight
 $ aws configure --profile kudpc
 AWS Access Key ID [None]:     ## Enter the access key issued by the User Portal
 AWS Secret Access Key [None]: ## Enter the secret access key issued by the User Portal
 Default region name [None]:   ## Enter without input
 Default output format [None]: ## Enter without input
 ```
 
 #### How to Operate the File
 ```nohighlight
 # Basic command system
 $ aws --profile kudpc --endpoint-url=https://s3.kudpc.kyoto-u.ac.jp s3 [Command]
 
 ## Obtain a list of files
 $ aws --profile kudpc --endpoint-url=https://s3.kudpc.kyoto-u.ac.jp s3 ls s3://b59999
 
 ## Upload Files
 $ aws --profile kudpc --endpoint-url=https://s3.kudpc.kyoto-u.ac.jp s3 cp sample.sh s3://b59999
 

 ## Download Files

 $ aws --profile kudpc --endpoint-url=https://s3.kudpc.kyoto-u.ac.jp s3 cp s3://b59999/sample.sh ./
 
 ## Delete files from network storage
 $ aws --profile kudpc --endpoint-url=https://s3.kudpc.kyoto-u.ac.jp s3 rm s3://b59999/sample.sh
 
 ## Synchronize local folders to network storage
 $ aws --profile kudpc --endpoint-url=https://s3.kudpc.kyoto-u.ac.jp s3 sync ./ s3://b59999
 ```
 
 ### When using s3fs
 #### How to Install
 Please refer to the [Official Manual](https://github.com/s3fs-fuse/s3fs-fuse/blob/master/README.md).
 The supercomputer system can be used without loading the module.
 
 #### Initial Setting
 ```nohighlight
 ## Create directory to mount network storage
 $ mkdir ./kudpc-s3
 
 ## Create S3 authentication information

 $ echo "{accesss key}:secret key}" > ${HOME}/.passwd-s3fs

 $ chmod 600 ${HOME}/.passwd-s3fs
 ```
 
 #### How to Connect
 ```nohighlight
 ## mount
 $ s3fs b59999 ./kudpc-s3 -o use_path_request_style,url=https://s3.kudpc.kyoto-u.ac.jp,uid=$(id -u),gid=$(id -g)
 ## unmount
 $ fusermount -u ./kudpc-s3
 ```
 
 ### When using Cyberduck
 #### How to Install
 Please download the file from [official website](https://cyberduck.io/).
 #### How to Connect
 1. Start Cyberduck.
 2. Click "New Connection" at the top of the screen.
 3. Enter the session information as shown in the table below and click the "Connect" button.  
    
 Item | Content
 ---- | ---------
 Protocol Selection Screen | Amazon S3
 Server | s3.kudpc.kyoto-u.ac.jp
 Port | 443 
 Access key ID | S3 access key
 Secret access key | S3 secret key
 ![Cyberduck_S3Connect](Cyberduck_S3Connect.png)
 

 ## How to Connect(FTP)

 #### Connection Information
 Item|Content
 -------| -----------------------------------

 Connection Destination| s3.kudpc.kyoto-u.ac.jp

 Account name | Please check the user portal.
 Password |Please check the user portal.
 
 ### When using WinSCP
 #### How to Install
 Please refer to [Installing WinSCP](../install/winscp).
 
 #### How to Connect 
 1. Start WinSCP.
 2. Please enter your session information as shown in the table below and click the "Login" button.    
  

   
 Item | Content

 -----| ----------
 Transfer protocol|FTP
 Encryption|Explicit TLS/SSL encryption
 Host name|s3.kudpc.kyoto-u.ac.jp
 Port number|21
 User name| FTP account name
 Password|  FTP Password
 ![WinSCP_FTPinit](WinSCP_FTPinit.png "WinSCP_FTPinit")

 3. If successful, the WinSCP screen will be displayed and file transfer will be possible.

 
 ### When using Cyberduck
 #### How to Install
 Please download the file from [official website](https://cyberduck.io/). 
 #### How to Connect
 1. Start Cyberduck.
 2. Click "New Connection" at the top of the screen.
 3. Enter the session information as shown in the table below and click the "Connect" button.
   
  Item | Content
 -----| ----------
 Protocol Selection Screen|FTP(File Transfer Protocol)
 Server|s3.kudpc.kyoto-u.ac.jp
 Port|21
 User name| FTP account name
 Password|FTP Password
 ![Cyberduck_FTPinit](Cyberduck_FTPinit.png "Cyberduck_FTPinit")
 <!--

 ### FTPコマンドを使用する場合
 #### 接続方法

 ```nohighlight

 ## 接続

 $ ftp s3.kudpc.kyoto-u.ac.jp

 Name (s3.kudpc.kyoto-u.ac.jp:b59999): #FTPのアカウント名を入力
 Password: #FTPのパスワードを入力

 ```
 

 #### ファイルの操作方法

 ```nohighlight

 ## ファイルの一覧を取得
 #> ls

 ## 接続元端末のカレントディレクトリを表示
 #> lcd

 ## ファイルをアップロード
 #> put sample.sh ./

 ## ファイルをダウンロード
 #> get sample.sh ./

 ## ファイルをネットワークストレージから削除
 #> delete sample.sh

 ```
 -->