Cluster Egaes

De e-Ciencia

Load balancing in Egaes cluster:

Structure

        O
       -|- ----->----
        |\          |
                    |
  Outside   ------------------
   World    |                |   
----------- |    egeeui01    | ---------
  Inside    |                |
   World    ------------------
                    |
                    |
            ------------------
            |                | 10.10.0.X
----------- |     egaes      | ---------
   Egaes    |                | 10.10.5.X
  Cluster   ------------------
                  / | \ 
                 /  |  \ 
                /   |   \
               /    |    \
              /     |     \
       -----------     -----------
       |         |     |         |
       | egaes01 | ... | egaes20 |
       |         |     |         |
       -----------     -----------

Technical details

• The tool used to have load balancing is Linux Virtual Server.
• Director:
  • It is configured on the same network as the Egaes cluster.
  • A Virtual Interface is created in the same network as the egeeui01 machine. This interface will be the one used to access the Egaes Cluster and distribute the traffic to the realservers.
  • Must suport NAT on the kernel.
• Realservers:
  • Use the director as the GW.
  • No other routing rules must enable a path to the director.
• Having the machines configured with the 255.255.0.0 netmask enables direct access to the Egaes cluster.

Configuration

Master (Director)

• egaes19.ifca.es -> egaes.ifca.es
• Install a kernel with LVS support compiled in. In this case 2.6.20.21 will be used.
• Compile & install ipvsadm
• Configure the Virtual Server director.sh:

#!/bin/sh

#set ip_forward ON for vs-nat director (1 on, 0 off).
cat /proc/sys/net/ipv4/ip_forward
echo "1" >/proc/sys/net/ipv4/ip_forward

#director is gw for realservers
#turn OFF icmp redirects (1 on, 0 off)
echo "0" >/proc/sys/net/ipv4/conf/all/send_redirects
cat       /proc/sys/net/ipv4/conf/all/send_redirects
echo "0" >/proc/sys/net/ipv4/conf/default/send_redirects
cat       /proc/sys/net/ipv4/conf/default/send_redirects
echo "0" >/proc/sys/net/ipv4/conf/eth0/send_redirects
cat       /proc/sys/net/ipv4/conf/eth0/send_redirects

#setup VIP
/sbin/ifconfig eth0:110 10.10.0.70 broadcast 10.10.0.255 netmask 255.255.255.0

#set default gateway
/sbin/route add default gw 10.10.0.254 netmask 0.0.0.0 metric 1

#clear ipvsadm tables
/sbin/ipvsadm -C

#install LVS services with ipvsadm
/sbin/ipvsadm -A -t 10.10.0.70:22 -s rr

# Install realservers
/sbin/ipvsadm -a -t 10.10.0.70:22 -r 10.10.5.1:22 -m -w 1
/sbin/ipvsadm -a -t 10.10.0.70:22 -r 10.10.5.2:22 -m -w 1
..
/sbin/ipvsadm -a -t 10.10.0.70:22 -r 10.10.5.18:22 -m -w 1

#list ipvsadm table
/sbin/ipvsadm

Slaves (Real Servers)

• Install a kernel with LVS support compiled in. In this case 2.6.20.21 will be used.
• Run realservers.sh:

#!/bin/sh

# Add the director as a GW
/sbin/route add default gw 10.10.5.70

# Remove the other routing tables.
/sbin/route del -net 10.10.0.0 netmask 255.255.0.0
/sbin/route del default gw torque00.ifca.es

#set_realserver_ip_forwarding to OFF (1 on, 0 off).
echo "0" >/proc/sys/net/ipv4/ip_forward
cat       /proc/sys/net/ipv4/ip_forward

Monitoring

TODO