Table of Contents
노드 구성
모든 노드는 VMware workstation을 통해 가상머신으로 동작하고 네트워크는 Bridge 모드로 구성 되었습니다.
각 글 내용 머리부분(heading) 앞의 : 표시 이전에는 hostname표시했습니다. 어느 노드에서 작업해야될지를 정리했습니다.
역할 | Hostname | IP | CPU | RAM | DISK | 비고 |
오픈스택,swift-proxy | controller | 192.168.0.11 | 4 | 8 | 50G | |
오브젝트 스토리지 | swift1 | 192.168.0.131 | 4 | 8 | 50G | 추가 디스크 20G x 3 |
오브젝트 스토리지 | swift2 | 192.168.0.132 | 4 | 8 | 50G | 추가 디스크 20G x 3 |
오브젝트 스토리지 | swift3 | 192.168.0.133 | 4 | 8 | 50G | 추가 디스크 20G x 3 |
Cosbench Controller + Cosbench Driver | cosbench1 | 192.168.0.141 | 4 | 8 | 50G |
|
Cosbench Driver | cosbench2 | 192.168.0.142 | 4 | 8 | 50G |
|
Cosbench Driver | cosbench3 | 192.168.0.143 | 4 | 8 | 50G |
|
cosbench1-3 : Cosbench 파일 설치
사전작업
yum remove -y firewalld
yum install -y wget java nmap-ncat unzip
echo "192.168.0.11 controller" >> /etc/hosts
호스트네임은 필수는 아님
hostnamectl set-hostname cosbench1
cosbench 설치
wget https://github.com/intel-cloud/cosbench/releases/download/v0.4.2.c4/0.4.2.c4.zip
unzip 0.4.2.c4.zip
cd 0.4.2.c4
chmod +x *.sh
cosbench1 : 컨트롤러 파일 설정
<cosbench path>/conf/controller.conf 아래와 같이 내용 수정
[controller]
concurrency = 1
name = client1
url = http://192.168.0.141:19088/controller
drivers = 3
log_level = INFO
log_file = log/system.log
archive_dir = archive
[driver1]
name = cosbench1
url = http://192.168.0.141:18088/driver
[driver2]
name = cosbench2
url = http://192.168.0.142:18088/driver
[driver3]
name = cosbench3
url = http://192.168.0.143:18088/driver
cosbench1-3 : 드라이버 파일 설정
<cosbench path>/conf/driver.conf 아래와 같이 내용 수정
cosbench1
[driver]
name = cosbench1
url = http://192.168.0.141:18088/driver
log_level = INFO
cosbench2
[driver]
name = cosbench2
url = http://192.168.0.142:18088/driver
log_level = INFO
cosbench3
[driver]
name = cosbench3
url = http://192.168.0.143:18088/driver
log_level = INFO
cosbench1-3 : 서비스 시작
cosbench1 ( controller+driver )
[root@cosbensh1 0.4.2.c4]# ./start-all.sh
Launching osgi framwork ...
Successfully launched osgi framework!
Booting cosbench driver ...
Starting cosbench-log_0.4.2 [OK]
Starting cosbench-tomcat_0.4.2 [OK]
Starting cosbench-config_0.4.2 [OK]
Starting cosbench-http_0.4.2 [OK]
Starting cosbench-cdmi-util_0.4.2 [OK]
Starting cosbench-core_0.4.2 [OK]
Starting cosbench-core-web_0.4.2 [OK]
Starting cosbench-api_0.4.2 [OK]
Starting cosbench-mock_0.4.2 [OK]
Starting cosbench-ampli_0.4.2 [OK]
Starting cosbench-swift_0.4.2 [OK]
Starting cosbench-keystone_0.4.2 [OK]
Starting cosbench-httpauth_0.4.2 [OK]
Starting cosbench-s3_0.4.2 [OK]
Starting cosbench-librados_0.4.2 [OK]
Starting cosbench-scality_0.4.2 [OK]
Starting cosbench-cdmi-swift_0.4.2 [OK]
Starting cosbench-cdmi-base_0.4.2 [OK]
Starting cosbench-driver_0.4.2 [OK]
Starting cosbench-driver-web_0.4.2 [OK]
Successfully started cosbench driver!
Listening on port 0.0.0.0/0.0.0.0:18089 ...
Persistence bundle starting...
Persistence bundle started.
----------------------------------------------
!!! Service will listen on web port: 18088 !!!
----------------------------------------------
======================================================
Launching osgi framwork ...
Successfully launched osgi framework!
Booting cosbench controller ...
.
Starting cosbench-log_0.4.2 [OK]
Starting cosbench-tomcat_0.4.2 [OK]
Starting cosbench-config_0.4.2 [OK]
Starting cosbench-core_0.4.2 [OK]
Starting cosbench-core-web_0.4.2 [OK]
Starting cosbench-controller_0.4.2 [OK]
Starting cosbench-controller-web_0.4.2 [OK]
Successfully started cosbench controller!
Listening on port 0.0.0.0/0.0.0.0:19089 ...
Persistence bundle starting...
Persistence bundle started.
----------------------------------------------
!!! Service will listen on web port: 19088 !!!
----------------------------------------------
[root@cosbensh1 0.4.2.c4]#
cosbench2-3 : driver
[root@cosbensh2 0.4.2.c4]# ./start-driver.sh
Launching osgi framwork ...
Successfully launched osgi framework!
Booting cosbench driver ...
Starting cosbench-log_0.4.2 [OK]
.
Starting cosbench-tomcat_0.4.2 [OK]
Starting cosbench-config_0.4.2 [OK]
Starting cosbench-http_0.4.2 [OK]
Starting cosbench-cdmi-util_0.4.2 [OK]
Starting cosbench-core_0.4.2 [OK]
Starting cosbench-core-web_0.4.2 [OK]
Starting cosbench-api_0.4.2 [OK]
Starting cosbench-mock_0.4.2 [OK]
Starting cosbench-ampli_0.4.2 [OK]
Starting cosbench-swift_0.4.2 [OK]
Starting cosbench-keystone_0.4.2 [OK]
Starting cosbench-httpauth_0.4.2 [OK]
Starting cosbench-s3_0.4.2 [OK]
Starting cosbench-librados_0.4.2 [OK]
Starting cosbench-scality_0.4.2 [OK]
Starting cosbench-cdmi-swift_0.4.2 [OK]
Starting cosbench-cdmi-base_0.4.2 [OK]
Starting cosbench-driver_0.4.2 [OK]
Starting cosbench-driver-web_0.4.2 [OK]
Successfully started cosbench driver!
Listening on port 0.0.0.0/0.0.0.0:18089 ...
Persistence bundle starting...
Persistence bundle started.
----------------------------------------------
!!! Service will listen on web port: 18088 !!!
----------------------------------------------
[root@cosbensh2 0.4.2.c4]#
대시보드 접속
http://192.168.0.141:19088/controller/
위 화면이 나오면 config workloads 클릭해서 컨피그 생성창 으로 이동
테스트 환경 구성
현재 예시에는 둘다 사용했지만 실제 사용시에는 정확한 하나만 사요하는 것 권장.
keystoneauth를 쓰는 경우 아래와 같이 설정하고 tempauth(swauth)를 쓰는 경우 기존 swift-proxy 서버의 설정을 그대로 두어야 함.
pipeline에 tempauth OR authtoken keystoneauth 둘 중 하나를 선택해서 설정.
방법 1 : Authentication
keystone으로 설정하고 아래 값 입력
username=admin;password=ADMIN_PASS;tenant_name=admin;auth_url=http://192.168.0.11:5000/v2.0;service=swift;region=RegionOne
만약 keystone으로 하더라도 storage는 none으로 선택. 그렇지 않으면 Bandwidth 결과가 나오지 않음.
방법 2 : Storage
swift로 설정하고 아래 값 입력
token=<토큰값>;storage_url=http://192.168.0.11:8080/v1/AUTH_ccac596c068b4e1b941626e1931dfbff
토큰값 붉은색 값으로 치환하고 AUTH_뒤의 값은 노란색 값으로 치환.
[root@controller ~]# openstack token issue
+------------+--------------------------------------------------------------------------------------------------+
| Field | Value |
+------------+--------------------------------------------------------------------------------------------------+
| expires | 2018-05-15T09:37:14+0000 |
| id | gAAAAABa-pw6k7E94ZH8lCa63KtMmgEz-fI6guAfv6Catmirs7VD-eHp- |
| | vwEge5q82aRTmGEG12g5PuELyTGOTzVVJx0WpHDDmBKZj4j- |
| | jgfwUMgzP2fZK7dkX91EPE1ZkAH3tcLzyFUEdMpQhXYG1QcRorrk9qFGI2TSf5ULHPJV0QJNwAUy1E |
| project_id | ccac596c068b4e1b941626e1931dfbff |
| user_id | ccd6e34bc40b4f3fa20ba0f033ee73ab |
+------------+--------------------------------------------------------------------------------------------------+
[root@controller ~]#
여기서 만약 방법을 둘다 사용할 경우 먼저 선언된 방법으로만 시도.
그 외 아래 값들은 테스트 수치로 필요에 따라 수정.
완료되면 아래의 Generate Configuration File 을 클릭해서 다운로드
생성된 파일 첨부 : 
ryusstory3.xml
혹은 아래 내용을 수정해서 사용가능. <여기서도 storage와 auth 중 하나만 사용, 위에서 언급한것처럼 auth 사용시 공란 storage 필요 >
<?xml version="1.0" encoding="UTF-8" ?>
<workload name="swift-sample" description="sample benchmark for swift">
<storage type="swift" config="token=gAAAAABa-m5546xZ35XKCjBhjlAe3W51duBigpkuATQ3YeI5dXkEuq3UCKxlJHqgskEHO1-C8llJLdAivuxxtdlBic0ohMAM7wZfkB0ExmNT8vV9owdZKFlEpHKLk1BzuADeYWbSHCEIridJ8deFT3V1xBge5rVkV_pNXcpc-wP5ioAyLk-nWXo;storage_url=http://controller:8080/v1/AUTH_ccac596c068b4e1b941626e1931dfbff" />
<auth type="keystone" config="username=admin;password=ADMIN_PASS;tenant_name=admin;auth_url=http://controller:5000/v2.0;service=swift;region=RegionOne" />
<workflow>
<workstage name="init">
<work type="init" workers="1" config="containers=r(1,32)" />
</workstage>
<workstage name="prepare">
<work type="prepare" workers="1" config="containers=r(1,32);objects=r(1,50);sizes=c(64)KB" />
</workstage>
<workstage name="main">
<work name="main" workers="8" runtime="300">
<operation type="read" ratio="80" config="containers=u(1,32);objects=u(1,50)" />
<operation type="write" ratio="20" config="containers=u(1,32);objects=u(51,100);sizes=c(64)KB" />
</work>
</workstage>
<workstage name="cleanup">
<work type="cleanup" workers="1" config="containers=r(1,32);objects=r(1,100)" />
</workstage>
<workstage name="dispose">
<work type="dispose" workers="1" config="containers=r(1,32)" />
</workstage>
</workflow>
</workload>
다운로드 이후 최하단의 go back to index로 메인 페이지로 재접속
테스트 실행
파일 선택
Browse로 파일 선택 후 submit
submit
submit 하게되면 아래와 같이 workload가 추가됨.
view details
view details 버튼을 누르면 아래와 같이 테스트 진행사항 확인가능.
테스트 결과
테스트 결과 화면 첨부
추가 테스트
그냥 끝내면 심심할것 같아서 컨피그를 좀 바꿔서 추가로 테스트를 해봤습니다.
테스트 xml
<?xml version="1.0" encoding="UTF-8" ?>
<workload name="swift-sample" description="sample benchmark for swift">
<storage type="swift" config="token=gAAAAABa-m5546xZ35XKCjBhjlAe3W51duBigpkuATQ3YeI5dXkEuq3UCKxlJHqgskEHO1-C8llJLdAivuxxtdlBic0ohMAM7wZfkB0ExmNT8vV9owdZKFlEpHKLk1BzuADeYWbSHCEIridJ8deFT3V1xBge5rVkV_pNXcpc-wP5ioAyLk-nWXo;storage_url=http://controller:8080/v1/AUTH_ccac596c068b4e1b941626e1931dfbff" />
<auth type="keystone" config="username=admin;password=ADMIN_PASS;tenant_name=admin;auth_url=http://controller:5000/v2.0;service=swift;region=RegionOne" />
<workflow>
<workstage name="init">
<work type="init" workers="1" config="containers=r(1,8)" />
</workstage>
<workstage name="main1-prepare">
<work type="prepare" workers="1" config="containers=r(1,8);objects=r(1,50);sizes=c(64)KB" />
</workstage>
<workstage name="main1-64K_R">
<work name="main" workers="8" runtime="60">
<operation type="write" ratio="100" config="containers=u(1,8);objects=u(51,100);sizes=c(64)KB" />
</work>
</workstage>
<workstage name="main1-64K_W">
<work name="main" workers="8" runtime="60">
<operation type="read" ratio="100" config="containers=u(1,8);objects=u(1,50)" />
</work>
</workstage>
<workstage name="main1-64K_RW">
<work name="main" workers="8" runtime="60">
<operation type="read" ratio="50" config="containers=u(1,8);objects=u(1,50)" />
<operation type="write" ratio="50" config="containers=u(1,8);objects=u(51,100);sizes=c(64)KB" />
</work>
</workstage>
<workstage name="main1-cleanup">
<work type="cleanup" workers="1" config="containers=r(1,8);objects=r(1,100)" />
</workstage>
<workstage name="main2-prepare">
<work type="prepare" workers="1" config="containers=r(1,8);objects=r(1,50);sizes=c(10)MB" />
</workstage>
<workstage name="main2-10M_R">
<work name="main" workers="8" runtime="60">
<operation type="write" ratio="100" config="containers=u(1,8);objects=u(51,100);sizes=c(10)MB" />
</work>
</workstage>
<workstage name="main2-10M_W">
<work name="main" workers="8" runtime="60">
<operation type="read" ratio="100" config="containers=u(1,8);objects=u(1,50)" />
</work>
</workstage>
<workstage name="main2-10M_RW">
<work name="main" workers="8" runtime="60">
<operation type="read" ratio="50" config="containers=u(1,8);objects=u(1,50)" />
<operation type="write" ratio="50" config="containers=u(1,8);objects=u(51,100);sizes=c(10)MB" />
</work>
</workstage>
<workstage name="main2-cleanup">
<work type="cleanup" workers="1" config="containers=r(1,8);objects=r(1,100)" />
</workstage>
<workstage name="main3-FW">
<work name="main" workers="8" runtime="60">
<operation type="filewrite" ratio="100" config="containers=u(1,8);fileselection=s;files=/tmp/testfiles/" />
</work>
</workstage>
<workstage name="cleanup">
<work type="cleanup" workers="1" config="containers=r(1,8);objects=r(1,100)" />
</workstage>
<workstage name="dispose">
<work type="dispose" workers="1" config="containers=r(1,8)" />
</workstage>
</workflow>
</workload>
테스트 설명
64K Read, Write, Read-Write,
10M Read, Write, Read-Write,
file write
이렇게 총 10개 테스트를 돌려봤습니다. runtime은 기본 300초인데 60초로 줄이고 테스트 했습니다.
테스트 결과
file write 부분은 잘못 작성해서 뻑났네요 ㅋㅋ 나온 결과만 정리하면 아래와 같이 나옵니다.
General Report
| Op-Type | Op-Count | Byte-Count | Avg-ResTime | Avg-ProcTime | Throughput | Bandwidth | Succ-Ratio |
|---|---|---|---|---|---|---|---|
| op1: init -write | 0 ops | 0 B | N/A | N/A | 0 op/s | 0 B/S | N/A |
| op1: prepare -write | 400 ops | 25.6 MB | 26.52 ms | 25.51 ms | 36.97 op/s | 2.37 MB/S | 100% |
| op1: write | 6.97 kops | 446.27 MB | 68.74 ms | 67.6 ms | 116.27 op/s | 7.44 MB/S | 100% |
| op1: read | 15.13 kops | 968.19 MB | 31.7 ms | 31.45 ms | 252.19 op/s | 16.14 MB/S | 100% |
| op1: read | 5.13 kops | 328.32 MB | 27.4 ms | 27.14 ms | 85.56 op/s | 5.48 MB/S | 100% |
| op2: write | 5.2 kops | 332.99 MB | 65.1 ms | 64.14 ms | 86.77 op/s | 5.55 MB/S | 100% |
| op1: cleanup -delete | 800 ops | 0 B | 20.33 ms | 20.33 ms | 49.13 op/s | 0 B/S | 100% |
| op1: prepare -write | 400 ops | 4 GB | 216.41 ms | 118.64 ms | 4.62 op/s | 46.2 MB/S | 100% |
| op1: write | 337 ops | 3.37 GB | 1395.99 ms | 833.15 ms | 5.73 op/s | 57.32 MB/S | 100% |
| op1: read | 858 ops | 8.58 GB | 556.4 ms | 70.44 ms | 14.38 op/s | 143.78 MB/S | 100% |
| op1: read | 269 ops | 2.69 GB | 569.64 ms | 78.14 ms | 4.52 op/s | 45.24 MB/S | 100% |
| op2: write | 280 ops | 2.8 GB | 1151.01 ms | 437.64 ms | 4.71 op/s | 47.11 MB/S | 100% |
| op1: cleanup -delete | 800 ops | 0 B | 21.9 ms | 21.9 ms | 45.63 op/s | 0 B/S | 100% |
