Pre-Installation
Operating System Settings
Stardust is supported to be installed on the following operating systems.
Redhat Enterprise Linux 6.6 x86_64
Redhat Enterprise Linux 6.7 x86_64
Redhat Enterprise Linux 6.8 x86_64
Oracle Enterprise Linux 6.6 x86_64
Oracle Enterprise Linux 6.7 x86_64
Oracle Enterprise Linux 6.8 x86_64
SELinux Settings
It is recommended to disable SELinux. To disable SELinux on linux follow the following instructions.
With the root user;
1- Open /etc/selinux/config file with vi
2- Change SELINUX variable to disabled.
3- Reboot the operating system.
bash#> vi /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
:wq
bash#> reboot
IPTABLES settings
Following ports need to be opened by iptables to be able to access Hadoop and Stardust application server.
Application Server
80 Default application server port
8080 xennlake Portal port
8081 xennlake Portal port
8082 xennlake Portal port
4848 Web server management port
Administration Access
22 SSH port
21 FTP port
Hadoop
50010 HDFS Datanode data transfer port
50020 HDFS Datanode IPC port
50070 HDFS Namenode http port
50470 HDFS Namenode https port
50075 HDFS Datanode http port
50475 . HDFS Datanode https port
50090 HDFS secondarynamenode http port
50091 HDFS secondarynamenode https port
50100 HDFS namenode backup port
50105 HDFS namenode backup http port
9000 HDFS filesystem port
8020 HDFS Namenode default filesystem port
--
50030 Mapreduce jobtracker http port
50060 Mapreduce tasktracker http port
13562 Mapreduce shufflehandler port
10020 Mapreduce jobhistory server
19888 Mapreduce Job History Http Server
--
8088 Yarn resource manager http port
8090 Yarn resource manager https port
8030 Yarn resourcemanager scheduler port
8031 Yarn resource tracker port
8032 Yarn resource manager port
8033 Yarn resourcemanager admin port
8042 Yarn resourcemanager node manager http port
Hive
10000 Hive JDBC port
10002 Hve management Port
Postgresql
5432 Postgresql listener port
Xenn.lake Streaming Data Transfer Ports
20001
20002
20003
20004
20005
Kafka
2181 Kafka Zookeeper port
9092 Kafka Server port
Linux 6.x IPtables config
To enable IPtables for the necessary ports, following steps can be executed with root privileges;
Execute iptables commands to enable ports.
Restart iptables
Alternatively you can update the /etc/sysconfig/iptables file to make the configuration persistent
bash#>
iptables -A INPUT -p tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp --dport 8080 -j ACCEPT
iptables -A INPUT -p tcp --dport 8081 -j ACCEPT
iptables -A INPUT -p tcp --dport 8082 -j ACCEPT
iptables -A INPUT -p tcp --dport 4848 -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -A INPUT -p tcp --dport 21 -j ACCEPT
iptables -A INPUT -p tcp --dport 50010 -j ACCEPT
iptables -A INPUT -p tcp --dport 50020 -j ACCEPT
iptables -A INPUT -p tcp --dport 50030 -j ACCEPT
iptables -A INPUT -p tcp --dport 50060 -j ACCEPT
iptables -A INPUT -p tcp --dport 50070 -j ACCEPT
iptables -A INPUT -p tcp --dport 50470 -j ACCEPT
iptables -A INPUT -p tcp --dport 50075 -j ACCEPT
iptables -A INPUT -p tcp --dport 50475 -j ACCEPT
iptables -A INPUT -p tcp --dport 50090 -j ACCEPT
iptables -A INPUT -p tcp --dport 50091 -j ACCEPT
iptables -A INPUT -p tcp --dport 50100 -j ACCEPT
iptables -A INPUT -p tcp --dport 50105 -j ACCEPT
iptables -A INPUT -p tcp --dport 9000 -j ACCEPT
iptables -A INPUT -p tcp --dport 8020 -j ACCEPT
iptables -A INPUT -p tcp --dport 8088 -j ACCEPT
iptables -A INPUT -p tcp --dport 8090 -j ACCEPT
iptables -A INPUT -p tcp --dport 8030 -j ACCEPT
iptables -A INPUT -p tcp --dport 8031 -j ACCEPT
iptables -A INPUT -p tcp --dport 8032 -j ACCEPT
iptables -A INPUT -p tcp --dport 8033 -j ACCEPT
iptables -A INPUT -p tcp --dport 8042 -j ACCEPT
iptables -A INPUT -p tcp --dport 10020 -j ACCEPT
iptables -A INPUT -p tcp --dport 13562 -j ACCEPT
iptables -A INPUT -p tcp --dport 19888 -j ACCEPT
iptables -A INPUT -p tcp --dport 10000 -j ACCEPT
iptables -A INPUT -p tcp --dport 10002 -j ACCEPT
iptables -A INPUT -p tcp --dport 5432 -j ACCEPT
iptables -A INPUT -p tcp --dport 20001 -j ACCEPT
iptables -A INPUT -p tcp --dport 20002 -j ACCEPT
iptables -A INPUT -p tcp --dport 20003 -j ACCEPT
iptables -A INPUT -p tcp --dport 20004 -j ACCEPT
iptables -A INPUT -p tcp --dport 20005 -j ACCEPT
iptables -A INPUT -p tcp --dport 2181 -j ACCEPT
iptables -A INPUT -p tcp --dport 9092 -j ACCEPT>
bash#> service iptables restart
linux 7.x Firewalld config
firewall-cmd --zone=public --add-port=80/tcp --permanent
firewall-cmd --zone=public --add-port=8080/tcp --permanent
firewall-cmd --zone=public --add-port=8081/tcp --permanent
firewall-cmd --zone=public --add-port=8082/tcp --permanent
firewall-cmd --zone=public --add-port=4848/tcp --permanent
firewall-cmd --zone=public --add-port=22/tcp --permanent
firewall-cmd --zone=public --add-port=21/tcp --permanent
firewall-cmd --zone=public --add-port=50010/tcp --permanent
firewall-cmd --zone=public --add-port=50020/tcp --permanent
firewall-cmd --zone=public --add-port=50030/tcp --permanent
firewall-cmd --zone=public --add-port=50060/tcp --permanent
firewall-cmd --zone=public --add-port=50070/tcp --permanent
firewall-cmd --zone=public --add-port=50470/tcp --permanent
firewall-cmd --zone=public --add-port=50075/tcp --permanent
firewall-cmd --zone=public --add-port=50475/tcp --permanent
firewall-cmd --zone=public --add-port=50090/tcp --permanent
firewall-cmd --zone=public --add-port=50091/tcp --permanent
firewall-cmd --zone=public --add-port=50100/tcp --permanent
firewall-cmd --zone=public --add-port=50105/tcp --permanent
firewall-cmd --zone=public --add-port=9000/tcp --permanent
firewall-cmd --zone=public --add-port=8020/tcp --permanent
firewall-cmd --zone=public --add-port=8088/tcp --permanent
firewall-cmd --zone=public --add-port=8090/tcp --permanent
firewall-cmd --zone=public --add-port=8030/tcp --permanent
firewall-cmd --zone=public --add-port=8031/tcp --permanent
firewall-cmd --zone=public --add-port=8032/tcp --permanent
firewall-cmd --zone=public --add-port=8033/tcp --permanent
firewall-cmd --zone=public --add-port=8042/tcp --permanent
firewall-cmd --zone=public --add-port=10020/tcp --permanent
firewall-cmd --zone=public --add-port=13562/tcp --permanent
firewall-cmd --zone=public --add-port=19888/tcp --permanent
firewall-cmd --zone=public --add-port=10000/tcp --permanent
firewall-cmd --zone=public --add-port=10002/tcp --permanent
firewall-cmd --zone=public --add-port=5432/tcp --permanent
firewall-cmd --zone=public --add-port=20001/tcp --permanent
firewall-cmd --zone=public --add-port=20002/tcp --permanent
firewall-cmd --zone=public --add-port=20003/tcp --permanent
firewall-cmd --zone=public --add-port=20004/tcp --permanent
firewall-cmd --zone=public --add-port=20005/tcp --permanent
firewall-cmd --zone=public --add-port=2181/tcp --permanent
firewall-cmd --zone=public --add-port=9092/tcp --permanent
[root@tt-dpgs-linfr01 sysconfig]# firewall-cmd --reload
success
[root@tt-dpgs-linfr01 sysconfig]# firewall-cmd --list-ports
80/tcp 8080/tcp 8081/tcp 8082/tcp 4848/tcp
22/tcp 21/tcp 50010/tcp 50020/tcp 50030/tcp
50060/tcp 50070/tcp 50470/tcp 50075/tcp 50475/tcp
50090/tcp 50091/tcp 50100/tcp 50105/tcp 9000/tcp
8020/tcp 8088/tcp 8090/tcp 8030/tcp 8031/tcp
8032/tcp 8033/tcp 8042/tcp 10020/tcp 13562/tcp
19888/tcp 10000/tcp 10002/tcp 5432/tcp 20001/tcp
20002/tcp 20003/tcp 20004/tcp 20005/tcp 2181/tcp
9092/tcp
[root@tt-dpgs-linfr01 sysconfig]#
OS Limits
Append the following lines to the /etc/security/limits.conf
hadoop soft nproc 32000
hadoop hard nproc 32000
hadoop soft nofile 32000
hadoop hard nofile 32000
hadoop soft nproc 4096
hadoop hard nproc 4096
Append the following lines to the /etc/sysctl.conf
fs.file-max = 32000
kernel.pid_max = 32000
Now you are good to go with the do_install.sh which is described in detailed on the "Installing xenn.lake" document.
Last updated