clickhouse 集群安装配置

张映 发表于 2021-08-17

分类目录: clickhouse

标签:,

随着数据量的不断增加,又不想增加开发成本,我们选择了tidb数据库,在olap和oltp方面表现都非常的抢眼。为了数据分层,更加方便的提供数据,还需要一个在olap方面有优秀表现的数据存储工具,es我们也在用,不过不能跨索引查询,这会导致单索引会越来越大,不好维护。而clickhouse在olap方面表现优异,并且支持联合查询。

一,zookeeper安装配置

1,安装java

yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel

2,下载zookeeper

wget https://downloads.apache.org/zookeeper/stable/apache-zookeeper-3.6.3-bin.tar.gz
tar zxvf apache-zookeeper-3.6.3-bin.tar.gz -C /home/zookeeper

3,配置zookeeper

mkdir -p /home/zookeeper/{data,logs}  

# cat /home/zookeeper/conf/zoo.cfg
ticketTime=2000
clientPort=2181
dataDir=/home/zookeeper/data
dataLogDir=/home/zookeeper/logs 

initLimit=10
syncLimit=5
server.1=10.0.55.17:2888:3888
server.2=10.0.10.23:2888:3888
server.3=10.0.55.16:2888:3888
server.4=10.0.10.24:2888:3888

echo 1 > /home/zookeeper/data/myid   #10.0.55.17机器上
echo 2 > /home/zookeeper/data/myid   #10.0.10.23机器上
echo 3 > /home/zookeeper/data/myid   #10.0.55.16机器上
echo 4 > /home/zookeeper/data/myid   #10.0.10.24机器上

4,配置自启动脚本

# cat /usr/lib/systemd/system/zookeeper.service
[Unit]
Description=zookeeper
After=syslog.target network.target

[Service]
Type=forking
User=root
Group=root
ExecStart=/home/zookeeper/bin/zkServer.sh start
ExecStop=/home/zookeeper/bin/zkServer.sh stop
Restart=on-failure

[Install]
WantedBy=multi-user.target

5,启动zookeeper

# systemctl start zookeeper && systemctl enable zookeeper
# jps
8459 QuorumPeerMain   //有该进程说明启动成功
13131 Jps

二,clickhouse安装配置

1,安装clickhouse

curl -s https://packagecloud.io/install/repositories/altinity/clickhouse/script.rpm.sh | sudo bash
yum install -y clickhouse-server-common.x86_64 clickhouse-server.x86_64 clickhouse-client.x86_64

2,创建目录

mkdir -p /home/clickhouse/tmp /home/clickhouse/logs /home/clickhouse/lock/
chown clickhouse.clickhouse -R /etc/clickhouse-server
chown clickhouse.clickhouse -R /home/clickhouse

3,配置config.xml

# cat /etc/clickhouse-server/config.xml
<?xml version="1.0"?>
<yandex>
 <!--日志-->
 <logger>
     <level>error</level>
     <log>/home/clickhouse/logs/clickhouse.log</log>
     <errorlog>/home/clickhouse/logs/error.log</errorlog>
     <size>500M</size>
     <count>5</count>
 </logger>
 <!--本地节点信息-->
 <http_port>8123</http_port>
 <tcp_port>9000</tcp_port>
 <interserver_http_port>9009</interserver_http_port>
 <interserver_http_host>clickhouse.test.com</interserver_http_host>
 <!--本机域名或IP-->
 <!--本地配置-->
 <listen_host>0.0.0.0</listen_host>
 <max_connections>4096</max_connections>
 <keep_alive_timeout>300</keep_alive_timeout>
 <max_concurrent_queries>1000</max_concurrent_queries>
 <uncompressed_cache_size>8589934592</uncompressed_cache_size>
 <mark_cache_size>5368709120</mark_cache_size>
 <path>/home/clickhouse/</path>
 <tmp_path>/home/clickhouse/tmp/</tmp_path>
 <users_config>users.xml</users_config>
 <default_profile>default</default_profile>
 <default_database>default</default_database>
 <!--集群相关配置-->
 <remote_servers incl="clickhouse_remote_servers"/>
 <zookeeper incl="zookeeper-servers" optional="true"/>
 <macros incl="macros" optional="true"/>
 <builtin_dictionaries_reload_interval>3600</builtin_dictionaries_reload_interval>
 <max_session_timeout>3600</max_session_timeout>
 <default_session_timeout>300</default_session_timeout>
 <max_table_size_to_drop>0</max_table_size_to_drop>
 <merge_tree>
     <parts_to_delay_insert>300</parts_to_delay_insert>
     <parts_to_throw_insert>600</parts_to_throw_insert>
     <max_delay_to_insert>2</max_delay_to_insert>
 </merge_tree>
 <max_table_size_to_drop>0</max_table_size_to_drop>
 <max_partition_size_to_drop>0</max_partition_size_to_drop>
 <distributed_ddl>
     <!-- Path in ZooKeeper to queue with DDL queries -->
     <path>/clickhouse/task_queue/ddl</path>
     <cleanup_delay_period>60</cleanup_delay_period>
     <task_max_lifetime>86400</task_max_lifetime>
     <max_tasks_in_queue>200</max_tasks_in_queue>
 </distributed_ddl>
 <include_from>/etc/clickhouse-server/metrika.xml</include_from>
</yandex>

4,配置users.xml

# cat /etc/clickhouse-server/users.xml
<?xml version="1.0"?>
<yandex>
 <profiles>
 <default>
 <max_memory_usage>10000000000</max_memory_usage>
 <use_uncompressed_cache>0</use_uncompressed_cache>
 <load_balancing>random</load_balancing>
 </default>
 <readonly>
 <max_memory_usage>10000000000</max_memory_usage>
 <use_uncompressed_cache>0</use_uncompressed_cache>
 <load_balancing>random</load_balancing>
 <readonly>1</readonly>
 </readonly>
 </profiles>
 <quotas>
 <!-- Name of quota. -->
 <default>
 <interval>
 <queries>0</queries>
 <errors>0</errors>
 <result_rows>0</result_rows>
 <read_rows>0</read_rows>
 <execution_time>0</execution_time>
 </interval>
 </default>
 </quotas>
 <users>
 <default>
 <!-- PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha256sum | tr -d '-' -->
 <!-- password UHXQQmhb -->
 <password>test1234</password>
 <networks>
 <ip>::/0</ip>
 </networks>
 <profile>default</profile>
 <quota>default</quota>
 </default>
 <ck>
 <password>test1234</password>
 <networks>
 <ip>::/0</ip>
 </networks>
 <profile>readonly</profile>
 <quota>default</quota>
 </ck>
 </users>
</yandex>

5,配置metrika.xml

# cat /etc/clickhouse-server/metrika.xml
<yandex>
 <!--ck集群节点-->
 <clickhouse_remote_servers>
 <clickhouse_test_netjoy>
 <!--分片1-->
 <shard>
 <internal_replication>true</internal_replication>
 <replica>
 <host>10.0.55.17</host>
 <port>9000</port>
 <user>default</user>
 <password>test1234</password>
 </replica>
 <!--复制集1-->
 <replica>
 <host>10.0.10.23</host>
 <port>9000</port>
 <user>default</user>
 <password>test1234</password>
 </replica>
 </shard>
 <!--分片2-->
 <shard>
 <internal_replication>true</internal_replication>
 <replica>
 <host>10.0.55.16</host>
 <port>9000</port>
 <user>default</user>
 <password>test1234</password>
 </replica>
 <!--复制集2-->
 <replica>
 <host>10.0.10.24</host>
 <port>9000</port>
 <user>default</user>
 <password>test1234</password>
 </replica>
 </shard>
 </clickhouse_test_netjoy>
 </clickhouse_remote_servers>
 <!--zookeeper相关配置-->
 <zookeeper-servers>
 <node index="1">
 <host>10.0.55.17</host>
 <port>2181</port>
 </node>
 <node index="2">
 <host>10.0.10.23</host>
 <port>2181</port>
 </node>
 <node index="3">
 <host>10.0.55.16</host>
 <port>2181</port>
 </node>
 <node index="4">
 <host>10.0.10.24</host>
 <port>2181</port>
 </node>
 </zookeeper-servers>

 <macros>
 <layer>test_netjoy1</layer>
 <shard>2</shard>
 <!--分片号-->
 <replica>10.0.10.23</replica>
 <!--当前节点IP-->
 </macros>
 <networks>
 <ip>::/0</ip>
 </networks>
 <!--压缩相关配置-->
 <clickhouse_compression>
 <case>
 <min_part_size>10000000000</min_part_size>
 <min_part_size_ratio>0.01</min_part_size_ratio>
 <method>lz4</method>
 <!--压缩算法lz4压缩比zstd快, 更占磁盘-->
 </case>
 </clickhouse_compression>
</yandex>

在这里采用了2个分片,2个副本,macros的配置,同一分片的shard号,一定要一样。

三,启动与集群检查

1,启动

# sudo -u clickhouse clickhouse-server --config-file=/etc/clickhouse-server/config.xml
Processing configuration file '/etc/clickhouse-server/config.xml'.
Including configuration file '/etc/clickhouse-server/metrika.xml'.
Logging error to /home/clickhouse/logs/clickhouse.log
Logging errors to /home/clickhouse/logs/error.log
Logging error to console
2021.08.17 10:37:28.281597 [ 12288 ] {} <Error> Application: DB::Exception: A setting 'receive_timeout' appeared at top level in config /etc/clickhouse-server/config.xml. But it is user-level setting that should be located in users.xml inside <profiles> section for specific profile. You can add it to <profiles><default> if you want to change default value of this setting. You can also disable the check - specify <skip_check_for_incorrect_settings>1</skip_check_for_incorrect_settings> in the main configuration file.

如果有配置错误,可以很好的体现出来。

# /etc/init.d/clickhouse-server start  //启动

2,集群测试

# netstat -tpnl |grep click
tcp        0      0 0.0.0.0:9009            0.0.0.0:*               LISTEN      12418/clickhouse-se
tcp        0      0 0.0.0.0:8123            0.0.0.0:*               LISTEN      12418/clickhouse-se
tcp        0      0 0.0.0.0:9000            0.0.0.0:*               LISTEN      12418/clickhouse-se

有些进程说明已启动成功

# clickhouse-client -u default --password test1234 -h127.0.0.1
ClickHouse client version 20.8.3.18.
Connecting to 127.0.0.1:9000 as user default.
Connected to ClickHouse server version 20.8.3 revision 54438.

testjian :)  show databases;

SHOW DATABASES

┌─name───────────────────────────┐
│ _temporary_and_external_tables │
│ default                        │
│ system                         │
└────────────────────────────────┘

clickhouse集群安装配置 clickhouse集群安装配置



转载请注明
作者:海底苍鹰
地址:http://blog.51yip.com/clickhouse/2516.html