How To Install Ansible Tower with Customer Provided Database
Ansible Tower can be installed with an external database that is provided by the customer, whether on bare metal, virtual machine, container, or cloud-hosted service. I have simulated this in my lab before the actual deployment. Let see.
My Scenario:
Prepared DB1:
Step:1 Verify and install required RPMS
# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.122.11 at1.lab.munshibari.biz at1
192.168.122.12 db1.lab.munshibari.biz db1
192.168.122.13 db2.lab.munshibari.biz db2
# yum repolist
Loaded plugins: product-id, search-disabled-repos, subscription-manager
repo id repo name status
!rhel-7-server-ansible-2.9-rpms/x86_64 Red Hat Ansible Engine 2.9 RPMs for Red Hat Enterprise Linux 7 Server 21
!rhel-7-server-extras-rpms/x86_64 Red Hat Enterprise Linux 7 Server - Extras (RPMs) 1,303
!rhel-7-server-rpms/x86_64 Red Hat Enterprise Linux 7 Server (RPMs) 29,413
!rhel-server-rhscl-7-rpms/x86_64 Red Hat Software Collections RPMs for Red Hat Enterprise Linux 7 Server 12,735
repolist: 43,472
# yum install rh-postgresql10-postgresql-server -y
# rpm -qa|grep postgresql
rh-postgresql10-postgresql-libs-10.12-2.el7.x86_64
rh-postgresql10-postgresql-server-10.12-2.el7.x86_64
rh-postgresql10-runtime-3.1-1.el7.x86_64
rh-postgresql10-postgresql-10.12-2.el7.x86_64
postgresql-libs-9.2.24-4.el7_8.x86_64Step:2 Initialize the new PostgreSQL database server installation.
# /opt/rh/rh-postgresql10/root/usr/bin/postgresql-setup --initdb
* Initializing database in '/var/opt/rh/rh-postgresql10/lib/pgsql/data'
* Initialized, logs are in /var/lib/pgsql/initdb_rh-postgresql10-postgresql.logStep:3 Enable the PostgreSQL database port in the firewall service.
# firewall-cmd --add-port=5432/tcp --permanent
success
# firewall-cmd --add-port=5432/tcp
successStep:4 To set the Environment variable. And read and execute a shell environment.
# cat /etc/profile.d/python.sh
export PATH="/opt/rh/python27/root/usr/bin:/opt/rh/rh-postgresql10/root/usr/bin:$PATH"
export LD_LIBRARY_PATH="/opt/rh/python27/root/usr/lib64:/opt/rh/rh-postgresql10/root/usr/lib64:$LD_LIBRARY_PATH"
export MANPATH="/opt/rh/python27/root/usr/share/man:$MANPATH"
export XDG_DATA_DIRS="/opt/rh/python27/root/usr/share:$XDG_DATA_DIRS"
export PKG_CONFIG_PATH="/opt/rh/python27/root/usr/lib64/pkgconfig:$PKG_CONFIG_PATH"
# source /etc/profile.d/python.shStep:5 To start the postgresql10 service.
# systemctl start rh-postgresql10-postgresql.service
# systemctl enable rh-postgresql10-postgresql.serviceStep:6 To create the database and users as required for the Ansible Tower.
# su - postgres
$ ls -la .pgpass
-rw-------. 1 postgres postgres 34 Aug 13 00:18 .pgpass
$ cat .pgpass
192.168.122.13:5432:*:replica:redhat123
$ psql
psql (10.12)
Type "help" for help.
postgres=# CREATE ROLE awxdbuser1 WITH LOGIN encrypted PASSWORD 'redhat123' VALID UNTIL 'infinity';
CREATE ROLE
postgres=# CREATE USER replica REPLICATION LOGIN ENCRYPTED PASSWORD 'redhat123';
CREATE ROLE
postgres=# \du
List of roles
Role name | Attributes | Member of
------------+------------------------------------------------------------+-----------
awxdbuser1 | Password valid until infinity | {}
postgres | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
replica | Replication | {}
postgres=# CREATE DATABASE tower1 WITH ENCODING='UTF8' OWNER=awxdbuser1 CONNECTION LIMIT=-1;
CREATE DATABASE
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+------------+----------+-------------+-------------+-----------------------
postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
template0 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
tower1 | awxdbuser1 | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
(4 rows)Step:7 To change PostgreSQL listening IP Address and the PostgreSQL Client Authentication Configuration.
# grep listen_addresses /var/opt/rh/rh-postgresql10/lib/pgsql/data/postgresql.conf
listen_addresses = '*' # what IP address(es) to listen on;
# vi /var/opt/rh/rh-postgresql10/lib/pgsql/data/pg_hba.conf
::::::::::::: CUT SOME OUTPUT :::::::::::::
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all peer
# IPv4 local connections:
host all all 127.0.0.1/32 ident
host all awxdbuser1 192.168.122.11/32 md5
host all awxdbuser1 192.168.122.12/32 md5
host all awxdbuser1 192.168.122.13/32 md5
# IPv6 local connections:
host all all ::1/128 ident
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all peer
host replication all 127.0.0.1/32 ident
host replication all ::1/128 ident
host replication replica 192.168.122.12/32 md5
host replication replica 192.168.122.13/32 md5Step:8 To restart the PostgreSQL 10 service to effect the change.
# systemctl restart rh-postgresql10-postgresql.servicePrepared DB2:
Step:1 Verify and install required RPMS
# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.122.11 at1.lab.munshibari.biz at1
192.168.122.12 db1.lab.munshibari.biz db1
192.168.122.13 db2.lab.munshibari.biz db2
# yum repolist
Loaded plugins: product-id, search-disabled-repos, subscription-manager
repo id repo name status
!rhel-7-server-ansible-2.9-rpms/x86_64 Red Hat Ansible Engine 2.9 RPMs for Red Hat Enterprise Linux 7 Server 21
!rhel-7-server-extras-rpms/x86_64 Red Hat Enterprise Linux 7 Server - Extras (RPMs) 1,303
!rhel-7-server-rpms/x86_64 Red Hat Enterprise Linux 7 Server (RPMs) 29,413
!rhel-server-rhscl-7-rpms/x86_64 Red Hat Software Collections RPMs for Red Hat Enterprise Linux 7 Server 12,735
repolist: 43,472
# yum install rh-postgresql10-postgresql-server -y
# rpm -qa|grep postgresql
rh-postgresql10-postgresql-libs-10.12-2.el7.x86_64
rh-postgresql10-postgresql-server-10.12-2.el7.x86_64
rh-postgresql10-runtime-3.1-1.el7.x86_64
rh-postgresql10-postgresql-10.12-2.el7.x86_64
postgresql-libs-9.2.24-4.el7_8.x86_64Step:2 Enable the PostgreSQL database port in the firewall service.
# firewall-cmd --add-port=5432/tcp --permanent
success
# firewall-cmd --add-port=5432/tcp
successInstall the Ansible Tower:
I have customized the role and playbooks atower-pgsql to automate some of the tasks for me which is based on automate-tower-ha-dr. both role are available in the GitHub.
Step:1 To download the atower-pgsql role.
# git clone git@github.com:mh2ict/atower-pgsql.git
Cloning into 'atower-pgsql'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 67 (delta 0), reused 3 (delta 0), pack-reused 64
Receiving objects: 100% (67/67), 26.46 KiB | 0 bytes/s, done.
Resolving deltas: 100% (3/3), done.
# cd atower-pgsql/
# ls
ansible.cfg hosts README.md setup-pgsql-replication.yml tower-setup.yml tower-stop-services.yml
check-pgsql-replication.yml pgsql-promote.yml roles tower-set-inventory-ha.yml tower-start-services.ymlStep:2 To download the Ansible Tower Installer under the atower-pgsql, as below.
# wget https://releases.ansible.com/ansible-tower/setup-bundle/ansible-tower-setup-bundle-latest.el7.tar.gz
# tar xzf ansible-tower-setup-bundle-latest.el7.tar.gzStep:3 To verify and change two inventory file, as below.
# pwd
atower-pgsql
# grep ^inventory ansible.cfg
inventory = hosts
# cat hosts
[tower]
at1.lab.munshibari.biz ansible_host=192.168.122.11
[database]
db1.lab.munshibari.biz pgsqlrep_role=master
[database_replica]
db2.lab.munshibari.biz pgsqlrep_role=replica
[all:vars]
pg_host='db1.lab.munshibari.biz'
# cat ansible-tower-setup-bundle-3.7.1-1/inventory
[tower]
at1.lab.munshibari.biz ansible_host=192.168.122.11
[database]
[all:vars]
admin_password='redhat123'
pg_host='db1.lab.munshibari.biz'
pg_port='5432'
pg_database='tower1'
pg_username='awxdbuser1'
pg_password='redhat123'
rabbitmq_port=5672
rabbitmq_username=tower
rabbitmq_password='redhat123'
rabbitmq_cookie=secretvalue
rabbitmq_vhost=at1.lab.munshibari.biz
rabbitmq_use_long_name=true
# 80/443 4369/TCP 5672/TCP 25672/TCP 15672/TCP 5432/TCP
# Isolated Tower nodes automatically generate an RSA key for authentication;
# To disable this behavior, set this value to false
# isolated_key_generation=trueStep:4 To create Passwordless Login Using SSH Keygen.
# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:YIjyODF7jlsLOkRYRsynncVPFJftXKdMf3gTarMfgk0 root@at1.lab.munshibari.biz
The key's randomart image is:
+---[RSA 2048]----+
| +. . .o..o |
| =...o ... . o..|
|=o.+.ooo o +.=.|
|.O. o. .. oE+.+|
|= o S = o.o|
| * . + . |
|+ o o .|
|o+ . . |
|o.. |
+----[SHA256]-----+
# ssh-copy-id root@at1
# ssh-copy-id root@db1
# ssh-copy-id root@db2
# ssh root@db2 date
# ssh root@db1 date
# ssh root@at1 dateNote: I'm using the root user for the installation, but you can you any non-root account as well.
Step:5 To verify the required RPMS repository.
# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.122.11 at1.lab.munshibari.biz at1
192.168.122.12 db1.lab.munshibari.biz db1
192.168.122.13 db2.lab.munshibari.biz db2
# yum repolist
Loaded plugins: product-id, search-disabled-repos, subscription-manager
repo id repo name status
!rhel-7-server-ansible-2.9-rpms/x86_64 Red Hat Ansible Engine 2.9 RPMs for Red Hat Enterprise Linux 7 Server 21
!rhel-7-server-extras-rpms/x86_64 Red Hat Enterprise Linux 7 Server - Extras (RPMs) 1,303
!rhel-7-server-rpms/x86_64 Red Hat Enterprise Linux 7 Server (RPMs) 29,413
!rhel-server-rhscl-7-rpms/x86_64 Red Hat Software Collections RPMs for Red Hat Enterprise Linux 7 Server 12,735
repolist: 43,472Step:6 To start the Ansible Tower installation.
# pwd
atower-pgsql
# cd ansible-tower-setup-bundle-3.7.1-1/
# ./setup.shonce the installation is successful, we will have the installation summary as below.
Setup PostgreSQL replication:
Step:1 To execute the setup-pgsql-replication.yml playbook, as below.
# pwd
atower-pgsql
# ansible-playbook -i hosts setup-pgsql-replication.ymlonce the replication setup is successful, we will have the replication setup summary as below.
Step:2 To execute the check-pgsql-replication.yml playbook to verify the replication setup, as below.
# pwd
atower-pgsql
# ansible-playbook -i hosts check-pgsql-replication.ymlonce the check-pgsql-replication.yml playbook is successful, we will have the summary as below.
Note: db2 has recovery mode enabled and db1 does not have. and the ansible tower configures with db1.
We can also use the "netstat -atnp" command to verify the connection, as below.
# netstat -atnp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
::::::::::::: CUT SOME OUTPUT :::::::::::::
tcp 0 0 192.168.122.13:38474 192.168.122.12:5432 TIME_WAIT -
tcp 0 0 192.168.122.13:38482 192.168.122.12:5432 ESTABLISHED 25783/postgres: wal
tcp 0 0 192.168.122.13:22 192.168.122.1:42364 ESTABLISHED 11210/sshd: root@pt
::::::::::::: CUT SOME OUTPUT :::::::::::::
# netstat -atnp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
::::::::::::: CUT SOME OUTPUT :::::::::::::tcp 0 0 192.168.122.12:5432 192.168.122.11:49642 ESTABLISHED 23285/postgres: awx
tcp 0 0 192.168.122.12:5432 192.168.122.11:49644 ESTABLISHED 23286/postgres: awx
tcp 0 0 192.168.122.12:5432 192.168.122.13:38482 ESTABLISHED 23283/postgres: wal
tcp 0 0 192.168.122.12:5432 192.168.122.11:49664 ESTABLISHED 23361/postgres: awx
tcp 0 0 192.168.122.12:5432 192.168.122.11:49674 ESTABLISHED 23366/postgres: awx
tcp 0 0 192.168.122.12:5432 192.168.122.11:49682 ESTABLISHED 23395/postgres: awx
tcp 0 0 192.168.122.12:5432 192.168.122.11:49646 ESTABLISHED 23291/postgres: awx
tcp 0 0 192.168.122.12:5432 192.168.122.11:49686 ESTABLISHED 23405/postgres: awx
::::::::::::: CUT SOME OUTPUT :::::::::::::HA failover to DB2 Server:
Step:1 To stop the PostgreSQL 10 service in DB1 Server.
# systemctl stop rh-postgresql10-postgresql.serviceStep:2 To execute the pgsql-promote.yml playbook from the Ansible Tower Server (at1) to promote standby database server as primary.
# pwd
atower-pgsql
# ansible-playbook -i hosts pgsql-promote.ymlonce the pgsql-promote.yml playbook is successful, we will have the summary as below.
Now, the db2 database server is the primary.
Repair Ansible Tower with New PostgreSQL:
Method 1: To change PostgreSQL Database server IP address in the "/etc/tower/conf.d/postgres.py" file and restart the Ansible Tower Services.
# vi /etc/tower/conf.d/postgres.py
# Ansible Tower database settings.
DATABASES = {
'default': {
'ATOMIC_REQUESTS': True,
'ENGINE': 'awx.main.db.profiled_pg',
'NAME': 'tower1',
'USER': 'awxdbuser1',
'PASSWORD': """redhat123""",
'HOST': 'db2.lab.munshibari.biz',
'PORT': '5432',
'OPTIONS': { 'sslmode': 'prefer',
'sslrootcert': '/etc/pki/tls/certs/ca-bundle.crt',
},
}
}
# ansible-tower-service restart
# ansible-tower-service statusMethod 2: To modify our setup inventory file according to changed Database IP addresses/hostnames and run setup.sh
# pwd
ansible-tower-setup-bundle-3.7.1-1
# cat inventory
[tower]
at1.lab.munshibari.biz ansible_host=192.168.122.11
[database]
[all:vars]
admin_password='redhat123'
pg_host='db2.lab.munshibari.biz'
::::::::::::: CUT SOME OUTPUT :::::::::::::
# ./setup.shYou can verify the Ansible Tower User Interface (GUI), after every step as we have mentioned above from the browser as well.
We can also configure the database replication from the db2 server to db1 as will by swaping database swapping database host information in the hosts inventory file.
Hope this post will help.