16.高可用高并发集群配置

Posted on Edited on In

 

高可用高并发集群配置

高可用: 24小时对外提供服务

高并发: 同一时间段能处理的请求数

1，中心化和去中心化

1.1，中心化

意思是所有的节点都要有一个主节点

缺点：

  • 中心挂了，服务就挂了
  • 中心处理数据的能力有限，不能把节点性能发挥到最大

特点：就是一个路由作用

1.2，去中心化

特点：去掉路由，我自己来路由

以上通俗的就是

中心化：几个经过认证的嘉宾在‘讲话’，所有其他人在听。

去中心化：每个人都可以‘讲话’，每个人都可以选择听或者讲。

2，Redis集群的执行流程分析

2.1，哈希槽说明

Redis 集群中内置了 16384 个哈希槽，当需要在 Redis 集群中放置一个 key-value时，redis 先对 key 使用 crc16 算法算出一个结果，然后把结果对 16384 求余数，这样每个 key 都会对应一个编号在 0-16383 之间的哈希槽，redis 会根据节点数量大致均等的将哈希槽映射到不同的节点。

当你往Redis Cluster中加入一个Key时，会根据crc16(key) mod 16384计算这个key应该分布到哪个hash slot中，一个hash slot中会有很多key和value。你可以理解成表的分区，使用单节点时的redis时只有一个表，所有的key都放在这个表里；改用Redis Cluster以后会自动为你生成16384个分区表，你insert数据时会根据上面的简单算法来决定你的key应该存在哪个分区，每个分区里有很多key。

2.2，执行流程分析

假如redis集群里面能存放90个key，那么redis集群把90key平分到3个主机

redis对每个主机里面30个存储位置都编号，当应用连接到主机1上面时，应该发送一个写的命令

主机使用crc16算出槽号

如果槽号在1-30 可以直接操作主机1

如果槽号在31-60那么redis会转发到主机2

如果应该再发一个命令set age 22

那么主机2使用crc16再算槽号再转发

3，Redis集群的搭建

Redis集群最少6个节点，3个master，3个slave

3.1，文档

http://redis.cn/topics/cluster-tutorial.html

3.2，原理：去中心化

3.3，集群规则

机器编号 ip port
1 192.168.168.130 7001
2 192.168.168.130 7002
3 192.168.168.130 7003
4 192.168.168.130 7004
5 192.168.168.130 7005
6 192.168.168.130 7006

3.4，搭建过程

3.4.1 新建文件夹

1
2
3
4
[root@localhost redis-6.0.9]# mkdir redis_cluster
[root@localhost redis-6.0.9]# cd redis_cluster/
[root@localhost redis_cluster]# pwd
/usr/local/redis-6.0.9/redis_cluster

3.4.2 准备服务端程序

复制redis-server和redis-cli文件

1
2
3
4
5
6
7
[root@localhost redis_cluster]# cp /usr/local/redis-6.0.9/bin/redis-server .
[root@localhost redis_cluster]# cp /usr/local/redis-6.0.9/bin/redis-cli .
[root@localhost redis_cluster]# ls -lh
total 14M
-rwxr-xr-x. 1 root root 4.9M Dec  9 21:00 redis-cli
-rwxr-xr-x. 1 root root 8.7M Dec  9 20:59 redis-server
[root@localhost redis_cluster]#

3.4.3 准备6个redis的配置文件

创建文件夹，配置redis.conf文件

1
2
3
4
5
6
7
8
9
10
11
12
13
[root@localhost redis_cluster]# mkdir redis-1
[root@localhost redis_cluster]# mkdir redis-2
[root@localhost redis_cluster]# mkdir redis-3
[root@localhost redis_cluster]# mkdir redis-4
[root@localhost redis_cluster]# mkdir redis-5
[root@localhost redis_cluster]# mkdir redis-6

[root@localhost redis_cluster]# cp /usr/local/src/redis-6.0.9/redis.conf redis-1/redis-7001.conf
[root@localhost redis_cluster]# cp /usr/local/src/redis-6.0.9/redis.conf redis-2/redis-7002.conf
[root@localhost redis_cluster]# cp /usr/local/src/redis-6.0.9/redis.conf redis-3/redis-7003.conf
[root@localhost redis_cluster]# cp /usr/local/src/redis-6.0.9/redis.conf redis-4/redis-7004.conf
[root@localhost redis_cluster]# cp /usr/local/src/redis-6.0.9/redis.conf redis-5/redis-7005.conf
[root@localhost redis_cluster]# cp /usr/local/src/redis-6.0.9/redis.conf redis-6/redis-7006.conf
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

bind 0.0.0.0

port 7001
# port 7002 ...

pidfile /var/run/redis_7001.pid
# pidfile /var/run/redis_7002.pid ...

logfile "/usr/local/redis-6.0.9/redis_cluster/redis-1/redis-7001.log"
# logfile "/usr/local/redis-6.0.9/redis_cluster/redis-2/redis-7002.log" ...

dbfilename dump-7001.rdb
# dbfilename dump-7002.rdb

dir /usr/local/redis-6.0.9/redis_cluster/redis-1/
# dir /usr/local/redis-6.0.9/redis_cluster/redis-2/ ...

daemonize yes

appendonly yes

################################ REDIS CLUSTER  ###############################

# Normal Redis instances can't be part of a Redis Cluster; only nodes that are
# started as cluster nodes can. In order to start a Redis instance as a
# cluster node enable the cluster support uncommenting the following:
#
cluster-enabled yes

# Every cluster node has a cluster configuration file. This file is not
# intended to be edited by hand. It is created and updated by Redis nodes.
# Every Redis Cluster node requires a different cluster configuration file.
# Make sure that instances running in the same system do not have
# overlapping cluster configuration file names.
#
# 集群的配置文件,该文件自动生成
cluster-config-file nodes-7001.conf
# cluster-config-file nodes-7002.conf ...

# Cluster node timeout is the amount of milliseconds a node must be unreachable
# for it to be considered in failure state.
# Most other internal time limits are a multiple of the node timeout.
#
cluster-node-timeout 5000

3.4.4 同时启动所有的redis

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
[root@localhost redis_cluster]# ./redis-server redis-1/redis-7001.conf
[root@localhost redis_cluster]# ./redis-server redis-2/redis-7002.conf
[root@localhost redis_cluster]# ./redis-server redis-3/redis-7003.conf
[root@localhost redis_cluster]# ./redis-server redis-4/redis-7004.conf
[root@localhost redis_cluster]# ./redis-server redis-5/redis-7005.conf
[root@localhost redis_cluster]# ./redis-server redis-6/redis-7006.conf
[root@localhost redis_cluster]#
[root@localhost redis_cluster]# ps -ef | grep redis
root       1890      1  0 22:00 ?        00:00:00 ./redis-server 0.0.0.0:7001 [cluster]
root       1896      1  0 22:00 ?        00:00:00 ./redis-server 0.0.0.0:7002 [cluster]
root       1902      1  0 22:00 ?        00:00:00 ./redis-server 0.0.0.0:7003 [cluster]
root       1921      1  0 22:01 ?        00:00:00 ./redis-server 0.0.0.0:7004 [cluster]
root       1927      1  0 22:01 ?        00:00:00 ./redis-server 0.0.0.0:7005 [cluster]
root       1933      1  0 22:01 ?        00:00:00 ./redis-server 0.0.0.0:7006 [cluster]
root       1940   1774  0 22:01 pts/1    00:00:00 grep --color=auto redis
[root@localhost redis_cluster]#
1
2
3
4
5
6
7
8
9
10
11
[root@localhost redis-7001]# pwd
/usr/local/redis-6.0.9/redis-cluster/redis-7001
[root@localhost redis-7001]# 
[root@localhost redis-7001]# ll
total 100
-rw-r--r--. 1 root root     0 Dec 10 20:46 appendonly.aof
-rw-r--r--. 1 root root   176 Dec 10 22:17 dump-7001.rdb
-rw-r--r--. 1 root root   841 Dec 10 22:17 nodes-7001.conf
-rw-r--r--. 1 root root 85043 Dec 10 22:18 redis-7001.conf
-rw-r--r--. 1 root root  7605 Dec 10 22:22 redis-7001.log
[root@localhost redis-7001]#

nodes-7001.conf文件是./redis-server redis.conf之后就生成了，不是./redis-cli分配hash slots之后才生成的。

3.4.5 创建集群（分配槽）

未验证：redis3.x/4.x创建集群是使用redis-trib.rb来创建，不过5.0.x以后的redis-cli包括了原redis-trib.rb的全部功能，可用来创建cluster。

redis 6.0以后不再使用redis-trib.rb来创建集群，而是使用redis-cli创建集群

(1) 使用redis-cli创建集群

1
bind 127.0.0.1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
[root@localhost redis-cluster]# ps -ef | grep redis
root      11202      1  0 20:46 ?        00:00:00 ./redis-server 127.0.0.1:7001 [cluster]
root      11213      1  0 20:46 ?        00:00:00 ./redis-server 127.0.0.1:7002 [cluster]
root      11244      1  0 20:47 ?        00:00:00 ./redis-server 127.0.0.1:7003 [cluster]
root      11254      1  0 20:47 ?        00:00:00 ./redis-server 127.0.0.1:7004 [cluster]
root      11268      1  0 20:47 ?        00:00:00 ./redis-server 127.0.0.1:7005 [cluster]
root      11280      1  0 20:47 ?        00:00:00 ./redis-server 127.0.0.1:7006 [cluster]
root      11338   1749  0 20:48 pts/0    00:00:00 grep --color=auto redis
[root@localhost redis-cluster]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:39:24:e4 brd ff:ff:ff:ff:ff:ff
    inet 192.168.88.128/24 brd 192.168.88.255 scope global noprefixroute dynamic ens33
       valid_lft 1641sec preferred_lft 1641sec
    inet6 fe80::aff0:e07e:612b:e7e3/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:87:d4:66:93 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 scope global docker0
       valid_lft forever preferred_lft forever
[root@localhost redis-cluster]# 

[root@localhost redis-cluster]# ./redis-cli --cluster create 192.168.88.128:7001 192.168.88.128:7002 192.168.88.128:7003 192.168.88.128:7004 192.168.88.128:7005 192.168.88.128:7006 --cluster-replicas 1
Could not connect to Redis at 192.168.88.128:7001: Connection refused
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
[root@localhost redis-cluster]# ./redis-cli --cluster create 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 127.0.0.1:7006 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:7005 to 127.0.0.1:7001
Adding replica 127.0.0.1:7006 to 127.0.0.1:7002
Adding replica 127.0.0.1:7004 to 127.0.0.1:7003
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 5c523bbdaafb06d0c04e8ad99c17780e29ec3ecf 127.0.0.1:7001
   slots:[0-5460] (5461 slots) master
M: bd2d26e54655c97f3c30621914e9c0ab63131cab 127.0.0.1:7002
   slots:[5461-10922] (5462 slots) master
M: ca9186f686953474039919f2c188cdfbd6c2e87b 127.0.0.1:7003
   slots:[10923-16383] (5461 slots) master
S: a1393679b96b166190d79dd2169206fea08a331d 127.0.0.1:7004
   replicates 5c523bbdaafb06d0c04e8ad99c17780e29ec3ecf
S: c9b6f5e54e9dfb123b48445cda938d902e5d6bae 127.0.0.1:7005
   replicates bd2d26e54655c97f3c30621914e9c0ab63131cab
S: ea0a983e8d26da5d4997b4bfd5567b05d5130cc1 127.0.0.1:7006
   replicates ca9186f686953474039919f2c188cdfbd6c2e87b
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 127.0.0.1:7001)
M: 5c523bbdaafb06d0c04e8ad99c17780e29ec3ecf 127.0.0.1:7001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: ca9186f686953474039919f2c188cdfbd6c2e87b 127.0.0.1:7003
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: ea0a983e8d26da5d4997b4bfd5567b05d5130cc1 127.0.0.1:7006
   slots: (0 slots) slave
   replicates ca9186f686953474039919f2c188cdfbd6c2e87b
M: bd2d26e54655c97f3c30621914e9c0ab63131cab 127.0.0.1:7002
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: a1393679b96b166190d79dd2169206fea08a331d 127.0.0.1:7004
   slots: (0 slots) slave
   replicates 5c523bbdaafb06d0c04e8ad99c17780e29ec3ecf
S: c9b6f5e54e9dfb123b48445cda938d902e5d6bae 127.0.0.1:7005
   slots: (0 slots) slave
   replicates bd2d26e54655c97f3c30621914e9c0ab63131cab
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@localhost redis-cluster]# 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
[root@localhost redis-cluster]# ./redis-cli -c -p 7001
127.0.0.1:7001> 
127.0.0.1:7001> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:163
cluster_stats_messages_pong_sent:157
cluster_stats_messages_sent:320
cluster_stats_messages_ping_received:152
cluster_stats_messages_pong_received:163
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:320
127.0.0.1:7001> 
127.0.0.1:7001> set k1 v1
-> Redirected to slot [12706] located at 127.0.0.1:7003
OK
127.0.0.1:7003> 
127.0.0.1:7003> set number 100
-> Redirected to slot [7743] located at 127.0.0.1:7002
OK
127.0.0.1:7002> get number
"100"
127.0.0.1:7002>

(2) 使用ruby脚本redis-trib.rb创建集群(在redis 6.0版本已经被废弃不可用了)

找到集群脚本，在/usr/local/src/redis-6.0.9/src/redis-trib.rb，需要安装Ruby的环境

安装ruby，准备redis-trib.rb文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
[root@localhost redis-cluster]# yum install ruby
[root@localhost redis-cluster]# 
[root@localhost redis-cluster]# ruby --version
ruby 2.0.0p648 (2015-12-16) [x86_64-linux]
[root@localhost redis-cluster]# 
[root@localhost redis-cluster]# cp /usr/local/src/redis-6.0.9/src/redis-
redis-benchmark    redis-check-aof    redis-check-rdb    redis-cli          redis-sentinel
redis-benchmark.c  redis-check-aof.c  redis-check-rdb.c  redis-cli.c        redis-server
redis-benchmark.d  redis-check-aof.d  redis-check-rdb.d  redis-cli.d        redis-trib.rb
redis-benchmark.o  redis-check-aof.o  redis-check-rdb.o  redis-cli.o        
[root@localhost redis-cluster]# cp /usr/local/src/redis-6.0.9/src/redis-trib.rb .
[root@localhost redis-cluster]# ll
total 13820
drwxr-xr-x. 2 root root     117 Dec 10 20:58 redis-7001
drwxr-xr-x. 2 root root     117 Dec 10 21:13 redis-7002
drwxr-xr-x. 2 root root     117 Dec 10 21:13 redis-7003
drwxr-xr-x. 2 root root     117 Dec 10 20:58 redis-7004
drwxr-xr-x. 2 root root     117 Dec 10 21:02 redis-7005
drwxr-xr-x. 2 root root     117 Dec 10 21:02 redis-7006
-rwxr-xr-x. 1 root root 5049408 Dec 10 17:49 redis-cli
-rwxr-xr-x. 1 root root 9094264 Dec 10 17:49 redis-server
-rwxr-xr-x. 1 root root    3600 Dec 10 22:13 redis-trib.rb
[root@localhost redis-cluster]#

redis.conf配置文件

1
bind 192.168.88.128

启动6个redis实例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[root@localhost redis-cluster]# ./redis-server redis-7001/redis-7001.conf 
[root@localhost redis-cluster]# ./redis-server redis-7002/redis-7002.conf 
[root@localhost redis-cluster]# ./redis-server redis-7003/redis-7003.conf 
[root@localhost redis-cluster]# ./redis-server redis-7004/redis-7004.conf 
[root@localhost redis-cluster]# ./redis-server redis-7005/redis-7005.conf 
[root@localhost redis-cluster]# ./redis-server redis-7006/redis-7006.conf 
[root@localhost redis-cluster]# 
[root@localhost redis-cluster]# ps -ef | grep redis
root      65362      1  0 22:22 ?        00:00:01 ./redis-server 192.168.88.128:7001 [cluster]
root      65368      1  0 22:22 ?        00:00:01 ./redis-server 192.168.88.128:7002 [cluster]
root      65374      1  0 22:22 ?        00:00:01 ./redis-server 192.168.88.128:7003 [cluster]
root      65380      1  0 22:22 ?        00:00:01 ./redis-server 192.168.88.128:7004 [cluster]
root      65386      1  0 22:22 ?        00:00:01 ./redis-server 192.168.88.128:7005 [cluster]
root      65392      1  0 22:22 ?        00:00:01 ./redis-server 192.168.88.128:7006 [cluster]
root      65399   1749  0 22:25 pts/0    00:00:00 grep --color=auto redis

用redis-trib.rb启动，发现redis 6.0已经不可用了，redis-trib.rb中所有的功能已经被转移到了redis-cli中。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
[root@localhost redis-cluster]# 
[root@localhost redis-cluster]# ./redis-trib.rb create --replicas 1 192.168.88.128:7001 192.168.88.128:7002 192.168.88.128:7003 192.168.88.128:7004 192.168.88.128:7005 192.168.88.128:7006
WARNING: redis-trib.rb is not longer available!
You should use redis-cli instead.

All commands and features belonging to redis-trib.rb have been moved
to redis-cli.
In order to use them you should call redis-cli with the --cluster
option followed by the subcommand name, arguments and options.

Use the following syntax:
redis-cli --cluster SUBCOMMAND [ARGUMENTS] [OPTIONS]

Example:
redis-cli --cluster create 192.168.88.128:7001 192.168.88.128:7002 192.168.88.128:7003 192.168.88.128:7004 192.168.88.128:7005 192.168.88.128:7006 --cluster-replicas 1

To get help about all subcommands, type:
redis-cli --cluster help

[root@localhost redis-cluster]# 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
[root@localhost redis_cluster]# ./redis-cli --cluster create 192.168.168.130:7001 192.168.168.130:7002 192.168.168.130:7003 192.168.168.130:7004 192.168.168.130:7005 192.168.168.130:7006 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.168.130:7005 to 192.168.168.130:7001
Adding replica 192.168.168.130:7006 to 192.168.168.130:7002
Adding replica 192.168.168.130:7004 to 192.168.168.130:7003
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 99abc1f93ec1832530f32c2040149f10fd6341b4 192.168.168.130:7001
   slots:[0-5460] (5461 slots) master
M: 9ba7a3caa97ae0accfc21d38a88c2f986f46e06a 192.168.168.130:7002
   slots:[5461-10922] (5462 slots) master
M: feedb1e4cde43310045d00de50c77c067686e1e3 192.168.168.130:7003
   slots:[10923-16383] (5461 slots) master
S: 923c9615d2dd90cdf07f5043acf2892ec8fbd46f 192.168.168.130:7004
   replicates feedb1e4cde43310045d00de50c77c067686e1e3
S: 0e2148ede5c79e4c9a275d16aa67860a3eabec59 192.168.168.130:7005
   replicates 99abc1f93ec1832530f32c2040149f10fd6341b4
S: 8651ce57669506d66cdb747226dd0d0dbb524d60 192.168.168.130:7006
   replicates 9ba7a3caa97ae0accfc21d38a88c2f986f46e06a
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 192.168.168.130:7001)
M: 99abc1f93ec1832530f32c2040149f10fd6341b4 192.168.168.130:7001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: feedb1e4cde43310045d00de50c77c067686e1e3 192.168.168.130:7003
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 8651ce57669506d66cdb747226dd0d0dbb524d60 192.168.168.130:7006
   slots: (0 slots) slave
   replicates 9ba7a3caa97ae0accfc21d38a88c2f986f46e06a
S: 0e2148ede5c79e4c9a275d16aa67860a3eabec59 192.168.168.130:7005
   slots: (0 slots) slave
   replicates 99abc1f93ec1832530f32c2040149f10fd6341b4
M: 9ba7a3caa97ae0accfc21d38a88c2f986f46e06a 192.168.168.130:7002
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 923c9615d2dd90cdf07f5043acf2892ec8fbd46f 192.168.168.130:7004
   slots: (0 slots) slave
   replicates feedb1e4cde43310045d00de50c77c067686e1e3
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@localhost redis_cluster]#
1
2
3
4
5
6
7
8
9
10
11
12
[root@localhost redis_cluster]#
[root@localhost redis_cluster]# ./redis-cli -c -p 7001
Could not connect to Redis at 127.0.0.1:7001: Connection refused
not connected> quit
[root@localhost redis_cluster]#
[root@localhost redis_cluster]# ./redis-cli -c -h 192.168.168.130 -p 7001
192.168.168.130:7001> ping
PONG
192.168.168.130:7001> get k1
-> Redirected to slot [12706] located at 192.168.168.130:7003
(nil)
192.168.168.130:7003>

(3) 使用docker创建集群，下载redis-trib的镜像运行

安装Docker

yum install docker

启动docker

systemctl start docker

A: 下载镜像
docker pull inem0o/redis-trib

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[root@localhost ~]# docker pull inem0o/redis-trib
Using default tag: latest
Trying to pull repository docker.io/inem0o/redis-trib ...
latest: Pulling from docker.io/inem0o/redis-trib
a2b2998a36ab: Pull complete
a3ed95caeb02: Pull complete
46ab6b64c08e: Pull complete
3d82c3ac2025: Pull complete
Digest: sha256:0b89d25b387f70ef1c54605bdf061dd86e0833dbc0e2149390570b8b372278f8
Status: Downloaded newer image for docker.io/inem0o/redis-trib:latest
[root@localhost ~]#
[root@localhost ~]# docker images
REPOSITORY                    TAG                 IMAGE ID            CREATED             SIZE
docker.io/inem0o/redis-trib   latest              0f7b910114d5        3 years ago         32 MB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
[root@localhost ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:1f:b2:aa brd ff:ff:ff:ff:ff:ff
    inet 192.168.168.130/24 brd 192.168.168.255 scope global noprefixroute dynamic ens33
       valid_lft 1596sec preferred_lft 1596sec
    inet6 fe80::8c0b:4db3:b3f3:c20b/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:fe:33:13:bf brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 scope global docker0
       valid_lft forever preferred_lft forever
[root@localhost ~]#
  • -it是为了可以输入
  • –net host 是为了上docker容器能连接上本地的宿主机
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    [root@localhost ~]# docker run -it --net host inem0o/redis-trib create --replicas 1 192.168.168.130:7001 192.168.168.130:7002 192.168.168.130:7003 192.168.168.130:7004 192.168.168.130:7005 192.168.168.130:7006
    >>> Creating cluster
    >>> Performing hash slots allocation on 6 nodes...
    Using 3 masters:
    192.168.168.130:7001
    192.168.168.130:7002
    192.168.168.130:7003
    Adding replica 192.168.168.130:7004 to 192.168.168.130:7001
    Adding replica 192.168.168.130:7005 to 192.168.168.130:7002
    Adding replica 192.168.168.130:7006 to 192.168.168.130:7003
    M: d131d3a216afc2330c34eae03d652261098b8cf0 192.168.168.130:7001
       slots:0-5460 (5461 slots) master
    M: 900862148b90c9318e594f987d26e53b3524afbb 192.168.168.130:7002
       slots:5461-10922 (5462 slots) master
    M: 57ecc520ffdd5b41527a9f8d4ff511574b169c19 192.168.168.130:7003
       slots:10923-16383 (5461 slots) master
    S: bfe2fd23f51fe3893e286c499acee797b497cc1c 192.168.168.130:7004
       replicates d131d3a216afc2330c34eae03d652261098b8cf0
    S: a194063118c49e7aa62131bd8bfa9790a3d44214 192.168.168.130:7005
       replicates 900862148b90c9318e594f987d26e53b3524afbb
    S: 115540c6c8e06784f31700f05abba6f0b572b48e 192.168.168.130:7006
       replicates 57ecc520ffdd5b41527a9f8d4ff511574b169c19
    Can I set the above configuration? (type 'yes' to accept): yes
    >>> Nodes configuration updated
    >>> Assign a different config epoch to each node
    >>> Sending CLUSTER MEET messages to join the cluster
    Waiting for the cluster to join.
    >>> Performing Cluster Check (using node 192.168.168.130:7001)
    M: d131d3a216afc2330c34eae03d652261098b8cf0 192.168.168.130:7001
       slots:0-5460 (5461 slots) master
       1 additional replica(s)
    M: 57ecc520ffdd5b41527a9f8d4ff511574b169c19 192.168.168.130:7003@17003
       slots:10923-16383 (5461 slots) master
       1 additional replica(s)
    S: a194063118c49e7aa62131bd8bfa9790a3d44214 192.168.168.130:7005@17005
       slots: (0 slots) slave
       replicates 900862148b90c9318e594f987d26e53b3524afbb
    S: bfe2fd23f51fe3893e286c499acee797b497cc1c 192.168.168.130:7004@17004
       slots: (0 slots) slave
       replicates d131d3a216afc2330c34eae03d652261098b8cf0
    M: 900862148b90c9318e594f987d26e53b3524afbb 192.168.168.130:7002@17002
       slots:5461-10922 (5462 slots) master
       1 additional replica(s)
    S: 115540c6c8e06784f31700f05abba6f0b572b48e 192.168.168.130:7006@17006
       slots: (0 slots) slave
       replicates 57ecc520ffdd5b41527a9f8d4ff511574b169c19
    [OK] All nodes agree about slots configuration.
    >>> Check for open slots...
    >>> Check slots coverage...
    [OK] All 16384 slots covered.
    [root@localhost ~]#

一定要启动redis实例，否则会出现以下错误信息

1
2
3
4
[root@localhost ~]# docker run -it --net host inem0o/redis-trib create --replicas 1 192.168.168.130:7001 192.168.168.130:7002 192.168.168.130:7003 192.168.168.130:7004 192.168.168.130:7005 192.168.168.130:7006
>>> Creating cluster
[ERR] Sorry, can't connect to node 192.168.168.130:7001
[root@localhost ~]#

3.4.6 测试集群环境

-c 表示连接集群

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[root@localhost redis_cluster]# ./redis-cli -c -p 7001
127.0.0.1:7001>
127.0.0.1:7001> set name Jim
-> Redirected to slot [5798] located at 192.168.168.130:7002
OK
192.168.168.130:7002> set number 112200
OK
192.168.168.130:7002> get number
"112200"
192.168.168.130:7002>
192.168.168.130:7002> set k1 v1
-> Redirected to slot [12706] located at 192.168.168.130:7003
OK
192.168.168.130:7003> set level 1
-> Redirected to slot [1085] located at 192.168.168.130:7001
OK
192.168.168.130:7001>

到此集群搭建完成

连接其他master

1
2
3
4
5
6
7
8
9
[root@localhost redis_cluster]# ./redis-cli -c -p 7002
127.0.0.1:7002> get k1
-> Redirected to slot [12706] located at 192.168.168.130:7003
"v1"
192.168.168.130:7003> get k1
"v1"
192.168.168.130:7003> get name
-> Redirected to slot [5798] located at 192.168.168.130:7002
"Jim"

连接slave，slave也可以写入数据

1
2
3
4
5
6
[root@localhost redis_cluster]# ./redis-cli -c -p 7005
127.0.0.1:7005>
127.0.0.1:7005> get level
-> Redirected to slot [1085] located at 192.168.168.130:7001
"1"
192.168.168.130:7001>
1
2
3
4
127.0.0.1:7005> set k2 v2
-> Redirected to slot [449] located at 192.168.168.130:7001
OK
192.168.168.130:7001>

host ip

redis.conf配置文件

1
2
# 标明redis的运行所在的ip
bind 192.168.88.128
1
2
3
4
5
6
7
[root@localhost redis-cluster]# ps -ef | grep redis
root      65362      1  0 22:22 ?        00:00:01 ./redis-server 192.168.88.128:7001 [cluster]
root      65368      1  0 22:22 ?        00:00:01 ./redis-server 192.168.88.128:7002 [cluster]
root      65374      1  0 22:22 ?        00:00:01 ./redis-server 192.168.88.128:7003 [cluster]
root      65380      1  0 22:22 ?        00:00:01 ./redis-server 192.168.88.128:7004 [cluster]
root      65386      1  0 22:22 ?        00:00:01 ./redis-server 192.168.88.128:7005 [cluster]
root      65392      1  0 22:22 ?        00:00:01 ./redis-server 192.168.88.128:7006 [cluster]

./redis-server 192.168.88.128:7001 [cluster]与redis.conf中的bond一致

1
[root@localhost redis_cluster]# ./redis-cli --cluster create 192.168.168.130:7001 192.168.168.130:7002 192.168.168.130:7003 192.168.168.130:7004 192.168.168.130:7005 192.168.168.130:7006 --cluster-replicas 1

192.168.168.130:7001: 表示想要搭建集群的redis节点的ip

1
[root@localhost redis_cluster]# ./redis-cli -c -h 192.168.168.130 -p 7001

-h 192.168.168.130: 表示想要连接的redis的ip