cosbench 로 오픈스택 swift 성능 테스트

Ryu's Tech

cosbench 로 오픈스택 swift 성능 테스트

Ryusstory 2018. 5. 15. 19:46

Table of Contents

노드 구성

모든 노드는 VMware workstation을 통해 가상머신으로 동작하고 네트워크는 Bridge 모드로 구성 되었습니다.

각 글 내용 머리부분(heading) 앞의 : 표시 이전에는 hostname표시했습니다. 어느 노드에서 작업해야될지를 정리했습니다.

역할	Hostname	IP	CPU	RAM	DISK	비고
오픈스택,swift-proxy	controller	192.168.0.11	4	8	50G
오브젝트 스토리지	swift1	192.168.0.131	4	8	50G	추가 디스크 20G x 3
오브젝트 스토리지	swift2	192.168.0.132	4	8	50G	추가 디스크 20G x 3
오브젝트 스토리지	swift3	192.168.0.133	4	8	50G	추가 디스크 20G x 3
Cosbench Controller + Cosbench Driver	cosbench1	192.168.0.141	4	8	50G
Cosbench Driver	cosbench2	192.168.0.142	4	8	50G
Cosbench Driver	cosbench3	192.168.0.143	4	8	50G

cosbench1-3 : Cosbench 파일 설치

사전작업

yum remove -y firewalld
yum install -y wget java nmap-ncat unzip
echo "192.168.0.11 controller" >> /etc/hosts

호스트네임은 필수는 아님

hostnamectl set-hostname cosbench1

cosbench 설치

wget https://github.com/intel-cloud/cosbench/releases/download/v0.4.2.c4/0.4.2.c4.zip
unzip 0.4.2.c4.zip
cd 0.4.2.c4
chmod +x *.sh

cosbench1 : 컨트롤러 파일 설정

<cosbench path>/conf/controller.conf 아래와 같이 내용 수정

[controller]
concurrency = 1
name = client1
url = http://192.168.0.141:19088/controller
drivers = 3
log_level = INFO
log_file = log/system.log
archive_dir = archive

[driver1]
name = cosbench1
url = http://192.168.0.141:18088/driver

[driver2]
name = cosbench2
url = http://192.168.0.142:18088/driver

[driver3]
name = cosbench3
url = http://192.168.0.143:18088/driver

cosbench1-3 : 드라이버 파일 설정

<cosbench path>/conf/driver.conf 아래와 같이 내용 수정

cosbench1

[driver]
name = cosbench1
url = http://192.168.0.141:18088/driver
log_level = INFO

cosbench2

[driver]
name = cosbench2
url = http://192.168.0.142:18088/driver
log_level = INFO

cosbench3

[driver]
name = cosbench3
url = http://192.168.0.143:18088/driver
log_level = INFO

cosbench1-3 : 서비스 시작

cosbench1 ( controller+driver )

[root@cosbensh1 0.4.2.c4]# ./start-all.sh
Launching osgi framwork ...
Successfully launched osgi framework!
Booting cosbench driver ...
Starting    cosbench-log_0.4.2    [OK]
Starting    cosbench-tomcat_0.4.2    [OK]
Starting    cosbench-config_0.4.2    [OK]
Starting    cosbench-http_0.4.2    [OK]
Starting    cosbench-cdmi-util_0.4.2    [OK]
Starting    cosbench-core_0.4.2    [OK]
Starting    cosbench-core-web_0.4.2    [OK]
Starting    cosbench-api_0.4.2    [OK]
Starting    cosbench-mock_0.4.2    [OK]
Starting    cosbench-ampli_0.4.2    [OK]
Starting    cosbench-swift_0.4.2    [OK]
Starting    cosbench-keystone_0.4.2    [OK]
Starting    cosbench-httpauth_0.4.2    [OK]
Starting    cosbench-s3_0.4.2    [OK]
Starting    cosbench-librados_0.4.2    [OK]
Starting    cosbench-scality_0.4.2    [OK]
Starting    cosbench-cdmi-swift_0.4.2    [OK]
Starting    cosbench-cdmi-base_0.4.2    [OK]
Starting    cosbench-driver_0.4.2    [OK]
Starting    cosbench-driver-web_0.4.2    [OK]
Successfully started cosbench driver!
Listening on port 0.0.0.0/0.0.0.0:18089 ...
Persistence bundle starting...
Persistence bundle started.
----------------------------------------------
!!! Service will listen on web port: 18088 !!!
----------------------------------------------

======================================================

Launching osgi framwork ...
Successfully launched osgi framework!
Booting cosbench controller ...
.
Starting    cosbench-log_0.4.2    [OK]
Starting    cosbench-tomcat_0.4.2    [OK]
Starting    cosbench-config_0.4.2    [OK]
Starting    cosbench-core_0.4.2    [OK]
Starting    cosbench-core-web_0.4.2    [OK]
Starting    cosbench-controller_0.4.2    [OK]
Starting    cosbench-controller-web_0.4.2    [OK]
Successfully started cosbench controller!
Listening on port 0.0.0.0/0.0.0.0:19089 ...
Persistence bundle starting...
Persistence bundle started.
----------------------------------------------
!!! Service will listen on web port: 19088 !!!
----------------------------------------------
[root@cosbensh1 0.4.2.c4]#

cosbench2-3 : driver

[root@cosbensh2 0.4.2.c4]# ./start-driver.sh
Launching osgi framwork ...
Successfully launched osgi framework!
Booting cosbench driver ...
Starting    cosbench-log_0.4.2    [OK]
.
Starting    cosbench-tomcat_0.4.2    [OK]
Starting    cosbench-config_0.4.2    [OK]
Starting    cosbench-http_0.4.2    [OK]
Starting    cosbench-cdmi-util_0.4.2    [OK]
Starting    cosbench-core_0.4.2    [OK]
Starting    cosbench-core-web_0.4.2    [OK]
Starting    cosbench-api_0.4.2    [OK]
Starting    cosbench-mock_0.4.2    [OK]
Starting    cosbench-ampli_0.4.2    [OK]
Starting    cosbench-swift_0.4.2    [OK]
Starting    cosbench-keystone_0.4.2    [OK]
Starting    cosbench-httpauth_0.4.2    [OK]
Starting    cosbench-s3_0.4.2    [OK]
Starting    cosbench-librados_0.4.2    [OK]
Starting    cosbench-scality_0.4.2    [OK]
Starting    cosbench-cdmi-swift_0.4.2    [OK]
Starting    cosbench-cdmi-base_0.4.2    [OK]
Starting    cosbench-driver_0.4.2    [OK]
Starting    cosbench-driver-web_0.4.2    [OK]
Successfully started cosbench driver!
Listening on port 0.0.0.0/0.0.0.0:18089 ...
Persistence bundle starting...
Persistence bundle started.
----------------------------------------------
!!! Service will listen on web port: 18088 !!!
----------------------------------------------
[root@cosbensh2 0.4.2.c4]#

대시보드 접속

http://192.168.0.141:19088/controller/

위 화면이 나오면 config workloads 클릭해서 컨피그 생성창 으로 이동

테스트 환경 구성

현재 예시에는 둘다 사용했지만 실제 사용시에는 정확한 하나만 사요하는 것 권장.

keystoneauth를 쓰는 경우 아래와 같이 설정하고 tempauth(swauth)를 쓰는 경우 기존 swift-proxy 서버의 설정을 그대로 두어야 함.

pipeline에 tempauth OR authtoken keystoneauth 둘 중 하나를 선택해서 설정.

방법 1 : Authentication

keystone으로 설정하고 아래 값 입력

username=admin;password=ADMIN_PASS;tenant_name=admin;auth_url=http://192.168.0.11:5000/v2.0;service=swift;region=RegionOne

만약 keystone으로 하더라도 storage는 none으로 선택. 그렇지 않으면 Bandwidth 결과가 나오지 않음.

방법 2 : Storage

swift로 설정하고 아래 값 입력

token=<토큰값>;storage_url=http://192.168.0.11:8080/v1/AUTH_ccac596c068b4e1b941626e1931dfbff

토큰값 붉은색 값으로 치환하고 AUTH_뒤의 값은 노란색 값으로 치환.

[root@controller ~]# openstack token issue
+------------+--------------------------------------------------------------------------------------------------+
| Field      | Value                                                                                            |
+------------+--------------------------------------------------------------------------------------------------+
| expires    | 2018-05-15T09:37:14+0000                                                                         |
| id         | gAAAAABa-pw6k7E94ZH8lCa63KtMmgEz-fI6guAfv6Catmirs7VD-eHp-                                        |
|            | vwEge5q82aRTmGEG12g5PuELyTGOTzVVJx0WpHDDmBKZj4j-                                                 |
|            | jgfwUMgzP2fZK7dkX91EPE1ZkAH3tcLzyFUEdMpQhXYG1QcRorrk9qFGI2TSf5ULHPJV0QJNwAUy1E                   |
| project_id | ccac596c068b4e1b941626e1931dfbff                                                                 |
| user_id    | ccd6e34bc40b4f3fa20ba0f033ee73ab                                                                 |
+------------+--------------------------------------------------------------------------------------------------+
[root@controller ~]# 

여기서 만약 방법을 둘다 사용할 경우 먼저 선언된 방법으로만 시도.

그 외 아래 값들은 테스트 수치로 필요에 따라 수정.

완료되면 아래의 Generate Configuration File 을 클릭해서 다운로드

생성된 파일 첨부 : ryusstory3.xml

혹은 아래 내용을 수정해서 사용가능. <여기서도 storage와 auth 중 하나만 사용, 위에서 언급한것처럼 auth 사용시 공란 storage 필요 >

<?xml version="1.0" encoding="UTF-8" ?>
<workload name="swift-sample" description="sample benchmark for swift">
<storage type="swift" config="token=gAAAAABa-m5546xZ35XKCjBhjlAe3W51duBigpkuATQ3YeI5dXkEuq3UCKxlJHqgskEHO1-C8llJLdAivuxxtdlBic0ohMAM7wZfkB0ExmNT8vV9owdZKFlEpHKLk1BzuADeYWbSHCEIridJ8deFT3V1xBge5rVkV_pNXcpc-wP5ioAyLk-nWXo;storage_url=http://controller:8080/v1/AUTH_ccac596c068b4e1b941626e1931dfbff" />
<auth type="keystone" config="username=admin;password=ADMIN_PASS;tenant_name=admin;auth_url=http://controller:5000/v2.0;service=swift;region=RegionOne" />
  <workflow>
    <workstage name="init">
      <work type="init" workers="1" config="containers=r(1,32)" />
    </workstage>
    <workstage name="prepare">
      <work type="prepare" workers="1" config="containers=r(1,32);objects=r(1,50);sizes=c(64)KB" />
    </workstage>
    <workstage name="main">
      <work name="main" workers="8" runtime="300">
        <operation type="read" ratio="80" config="containers=u(1,32);objects=u(1,50)" />
        <operation type="write" ratio="20" config="containers=u(1,32);objects=u(51,100);sizes=c(64)KB" />
      </work>
    </workstage>
    <workstage name="cleanup">
      <work type="cleanup" workers="1" config="containers=r(1,32);objects=r(1,100)" />
    </workstage>
    <workstage name="dispose">
      <work type="dispose" workers="1" config="containers=r(1,32)" />
    </workstage>
  </workflow>
</workload>

다운로드 이후 최하단의 go back to index로 메인 페이지로 재접속

테스트 실행

파일 선택

Browse로 파일 선택 후 submit

submit

submit 하게되면 아래와 같이 workload가 추가됨.

view details

view details 버튼을 누르면 아래와 같이 테스트 진행사항 확인가능.

테스트 결과

테스트 결과 화면 첨부

추가 테스트

그냥 끝내면 심심할것 같아서 컨피그를 좀 바꿔서 추가로 테스트를 해봤습니다.

테스트 xml

<?xml version="1.0" encoding="UTF-8" ?>
<workload name="swift-sample" description="sample benchmark for swift">
<storage type="swift" config="token=gAAAAABa-m5546xZ35XKCjBhjlAe3W51duBigpkuATQ3YeI5dXkEuq3UCKxlJHqgskEHO1-C8llJLdAivuxxtdlBic0ohMAM7wZfkB0ExmNT8vV9owdZKFlEpHKLk1BzuADeYWbSHCEIridJ8deFT3V1xBge5rVkV_pNXcpc-wP5ioAyLk-nWXo;storage_url=http://controller:8080/v1/AUTH_ccac596c068b4e1b941626e1931dfbff" />
<auth type="keystone" config="username=admin;password=ADMIN_PASS;tenant_name=admin;auth_url=http://controller:5000/v2.0;service=swift;region=RegionOne" />
  <workflow>
    <workstage name="init">
      <work type="init" workers="1" config="containers=r(1,8)" />
    </workstage>

    <workstage name="main1-prepare">
      <work type="prepare" workers="1" config="containers=r(1,8);objects=r(1,50);sizes=c(64)KB" />
    </workstage>
  <workstage name="main1-64K_R">
      <work name="main" workers="8" runtime="60">
        <operation type="write" ratio="100" config="containers=u(1,8);objects=u(51,100);sizes=c(64)KB" />
      </work>
    </workstage>
    <workstage name="main1-64K_W">
      <work name="main" workers="8" runtime="60">
        <operation type="read" ratio="100" config="containers=u(1,8);objects=u(1,50)" />
      </work>
    </workstage>
    <workstage name="main1-64K_RW">
      <work name="main" workers="8" runtime="60">
        <operation type="read" ratio="50" config="containers=u(1,8);objects=u(1,50)" />
    <operation type="write" ratio="50" config="containers=u(1,8);objects=u(51,100);sizes=c(64)KB" />
      </work>
    </workstage>
    <workstage name="main1-cleanup">
      <work type="cleanup" workers="1" config="containers=r(1,8);objects=r(1,100)" />
    </workstage>

    <workstage name="main2-prepare">
      <work type="prepare" workers="1" config="containers=r(1,8);objects=r(1,50);sizes=c(10)MB" />
    </workstage>
  <workstage name="main2-10M_R">
      <work name="main" workers="8" runtime="60">
        <operation type="write" ratio="100" config="containers=u(1,8);objects=u(51,100);sizes=c(10)MB" />
      </work>
    </workstage>
    <workstage name="main2-10M_W">
      <work name="main" workers="8" runtime="60">
        <operation type="read" ratio="100" config="containers=u(1,8);objects=u(1,50)" />
      </work>
    </workstage>
    <workstage name="main2-10M_RW">
      <work name="main" workers="8" runtime="60">
        <operation type="read" ratio="50" config="containers=u(1,8);objects=u(1,50)" />
    <operation type="write" ratio="50" config="containers=u(1,8);objects=u(51,100);sizes=c(10)MB" />
      </work>
    </workstage>
    <workstage name="main2-cleanup">
      <work type="cleanup" workers="1" config="containers=r(1,8);objects=r(1,100)" />
    </workstage>
  
  <workstage name="main3-FW">
      <work name="main" workers="8" runtime="60">
        <operation type="filewrite" ratio="100" config="containers=u(1,8);fileselection=s;files=/tmp/testfiles/" />
      </work>
    </workstage>
  
    <workstage name="cleanup">
      <work type="cleanup" workers="1" config="containers=r(1,8);objects=r(1,100)" />
    </workstage>
    <workstage name="dispose">
      <work type="dispose" workers="1" config="containers=r(1,8)" />
    </workstage>
  </workflow>
</workload>

테스트 설명

64K Read, Write, Read-Write,

10M Read, Write, Read-Write,

file write

이렇게 총 10개 테스트를 돌려봤습니다. runtime은 기본 300초인데 60초로 줄이고 테스트 했습니다.

테스트 결과

file write 부분은 잘못 작성해서 뻑났네요 ㅋㅋ 나온 결과만 정리하면 아래와 같이 나옵니다.

General Report

Op-Type	Op-Count	Byte-Count	Avg-ResTime	Avg-ProcTime	Throughput	Bandwidth	Succ-Ratio
op1: init -write	0 ops	0 B	N/A	N/A	0 op/s	0 B/S	N/A
op1: prepare -write	400 ops	25.6 MB	26.52 ms	25.51 ms	36.97 op/s	2.37 MB/S	100%
op1: write	6.97 kops	446.27 MB	68.74 ms	67.6 ms	116.27 op/s	7.44 MB/S	100%
op1: read	15.13 kops	968.19 MB	31.7 ms	31.45 ms	252.19 op/s	16.14 MB/S	100%
op1: read	5.13 kops	328.32 MB	27.4 ms	27.14 ms	85.56 op/s	5.48 MB/S	100%
op2: write	5.2 kops	332.99 MB	65.1 ms	64.14 ms	86.77 op/s	5.55 MB/S	100%
op1: cleanup -delete	800 ops	0 B	20.33 ms	20.33 ms	49.13 op/s	0 B/S	100%
op1: prepare -write	400 ops	4 GB	216.41 ms	118.64 ms	4.62 op/s	46.2 MB/S	100%
op1: write	337 ops	3.37 GB	1395.99 ms	833.15 ms	5.73 op/s	57.32 MB/S	100%
op1: read	858 ops	8.58 GB	556.4 ms	70.44 ms	14.38 op/s	143.78 MB/S	100%
op1: read	269 ops	2.69 GB	569.64 ms	78.14 ms	4.52 op/s	45.24 MB/S	100%
op2: write	280 ops	2.8 GB	1151.01 ms	437.64 ms	4.71 op/s	47.11 MB/S	100%
op1: cleanup -delete	800 ops	0 B	21.9 ms	21.9 ms	45.63 op/s	0 B/S	100%

저작자표시 비영리 변경금지