As I am sure you already know, IPv4 addresses are in limited supply right now. The solution to this is IPv6 which greatly enlarges the available address space. The problem is that IPv6 is not yet deployed everywhere, so there is still a need to figure out how to maximize the usage of your existing IPv4 addresses.
I have a VPS on the Internet which only provides 1 IPv4 address. Of course, I want to run multiple services on this VPS. I also want to use well-known ports to decrease the chance of being blocked from accessing my VPS.
There are several tools that can handle port multi-plexing. Probably among the most widely used are
haproxy and
sslh. Both of these tools are probably available in your Linux package manager.
SSLH is very easy to use but it only multiplexes SSL and SSH sessions. If you want more than 2 services on the same port then this tool is not for you.
HAPROXY is a bit more complicated to set up but it is also a lot more configurable. This post will describe the way that I have haproxy configured to host multiple services. I will post the full configuration file at the bottom of this post for easy copying and pasting.
NOTE: When you are reading the code below, any text that is
underlined needs to be replaced with values that are appropriate to your installation.
The first step in configuring haproxy is to set up the "frontend" This is the portion of haproxy that listens for incoming connections. Your "frontend" might look like this:
frontend ssl
mode tcp
bind <ipaddress>:<port>
tcp-request inspect-delay 3s
tcp-request content accept if { req.ssl_hello_type 1 }
This basically tells haproxy which IP address and port to listen on for incoming connections. You can also use the IP address 0.0.0.0 for every available IP address, if you have multiple.
The "inspect-delay" tells haproxy how long it should wait to receive data from the client before making a decision about what to do with the incoming connection. This is required due to the difference in the way that HTTPS and SSH sessions are negotiated. This is also the way that we distinguish the traffic type.
Once you have this front-end configured, you next need to configure your access control lists which connect your front-end to your backend(s).
The ACL for an SSH session looks like this:
acl <ssh label> payload(0,7) -m bin 5353482d322e30
This will detect SSH sessions and mark them with <ssh_label> This is an arbitrary label and you can pick any name you want. The only requirement is that it matches the rules that connect to the SSH backend.
Your "use_backend" statement for SSH would then look like:
use_backend <ssh backend name> if <ssh label>
As before, the <ssh backend name> is an arbitrary label you can pick. The only requirement again is that the backend name must match the backend definition.
Since we are now talking about the backend, here is what an SSH backend would look like:
backend openssh
mode tcp
timeout server 3h
server openssh <ip address>:<port>
Typically you would use an IP address of 127.0.0.1 to mean localhost or the local machine. The default port for SSH is 22. It is possible to use any IP address and port you want in this definition. That would be useful if the SSH server is on a different machine on a network behind your haproxy system.
Now we can add additional services. It is common for a single web-server to host multiple web-sites. These web-sites are identified by their DNS name. On the server side this is called SNI or Server Name Indication.
Let's start by setting up an ACL for server1.visideas.com
acl <server one> req.ssl_sni -i server1.visideas.com
Then the matching use_backend rule would look like:
use_backend <server 1 backend> if <server one acl> { req.ssl_hello_type 1 }
Finally, your matching backend might look like:
backend <server 1 backend>
mode tcp
server webserver <server 1 IP>:<server 1 port>
There are also some powerful matching criteria that you can use in your ACL's. For example, both of these are valid:
acl <some acl> req.ssl_sni -m end .visideas.com
acl <different acl> req.ssl_sni -m found
The first line matches any domain name that ends in .visideas.com and marks it with <some acl>. The second line matches any name and tags it with <different acl>. These lines will
not mark any requests received that were directed directly to an IP address.
Another use_backend that is useful is:
use_backend <another backend> if { req.ssl_hello_type 1 }
The ssl_hello_type of 1 indicates the presence of an HTTPS request. Since there is no tag name after the "if" this ACL would catch requests which were sent to this haproxy server by IP address. This means that you can route traffic which came in by specifying IP address to an alternate service.
The final ACL that I will discuss is:
use_backend <shadowsocks> if !{ req.ssl_hello_type 1 } !{ req.len 0 }
This ACL can detect traffic that is meant to be sent to a Shadowsocks server. This traffic is identified because it does not contain an ssl_hello_type of 1 and it sends traffic immediately without waiting - i.e. the request length is not 0.
There are probably other protocols that this statement would match as well but I am using it for Shadowsocks.
Now, as promised, here is my complete haproxy.conf. Again, please remember to change everything that is underlined to match your specific settings.
This configuration allows me to access the following services on port 443:
- An nginx server when accessed as https://s.visideas.com/
- An Apache2 server when access as https://k.visideas.com/ or https://*.visideas.com/ or https://<any DNS name>
- A Monit server when accessed as https://monit.visideas.com/
- An OpenConnect SSL VPN server when accessed as https://<ip address>/
- A Shadowsocks server when accessed using a Shadowsocks client
- An SSH server
global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
user haproxy
group haproxy
daemon
defaults
log global
mode tcp
option tcplog
option dontlognull
maxconn 2000
timeout connect 5000
timeout client 500000
timeout server 500000
frontend ssl
mode tcp
bind <host IP>:443
tcp-request inspect-delay 3s
tcp-request content accept if { req.ssl_hello_type 1 }
acl ssh_payload payload(0,7) -m bin 5353482d322e30
acl www-monit req.ssl_sni -i monit.visideas.com
acl www-s req.ssl_sni -i s.visideas.com
acl www-r req.ssl_sni -i r.visideas.com
acl www-k req.ssl_sni -m end .visideas.com
acl www-k req.ssl_sni -m found
use_backend www-monit if www-monit { req.ssl_hello_type 1 }
use_backend nginx-s if www-s { req.ssl_hello_type 1 }
use_backend apache2-k if www-k { req.ssl_hello_type 1 }
use_backend ocserv if { req.ssl_hello_type 1 }
use_backend openssh if ssh_payload
use_backend openssh if !{ req.ssl_hello_type 1 } { req.len 0 }
use_backend shadowsocks if !{ req.ssl_hello_type 1 } !{ req.len 0 }
backend openssh
mode tcp
timeout server 3h
server openssh 127.0.0.1:22
backend ocserv
mode tcp
timeout server 24h
server sslvpn 127.0.0.1:4443
backend nginx-s
mode tcp
server webserver 127.0.0.1:8443
backend apache2-k
mode tcp
server webserver 127.0.0.1:10443
backend www-monit
mode tcp
server webserver 127.0.0.1:2812
backend shadowsocks
mode tcp
server socks 127.0.0.1:8530
I hope this helps you with maximizing the value of your shared IPv4 addresses with haproxy.
