Hadoop

Go to Resource > Infrastructure > Hadoop Backup and click on Add button to add a Hadoop Cluster. You can view, add, modify, delete, refresh and other operations on Hadoop clusters.

Add

Simple verification type

Click on Add, set the Hadoop cluster name, select the verification method as Simple, fill in the host IP, user information, and REST API port information, and click OK to complete the creation. Host: The IP of Hadoop Cluster NameNode. SSL Certificate: Use SSL secure connection, this option requires Hadoop cluster configuration and enabling HTTPS service to use, otherwise there is no need to check this option. User: Users with read and write permissions for HDFS in Hadoop Clusters. REST API Port: The port number of the HTTP service for the hdfs-site.xml configuration file in the NameNode. Add NameNode: Can add multiple NameNode (HA). Proxy: You can choose a proxy that only works during the scanning phase when creating jobs and connecting to the cluster. The proxy is deployed on a physical server in the virtualization platform or data center, used to improve task efficiency and save network bandwidth. To enable the proxy, you can added in Resource > Infrastructure > Agent & Client.

Kerberos verification type

Click on Add, set the Hadoop cluster name, select the authentication method as Kerberos, fill in the relevant information, and click OK to complete the creation. Realm Name: Configure the Realm name when creating Kerberos. Realm KDC Server:Realm KDC server IP or hostname. Realm Management Server:Realm Management Server IP or Host Name. WebHDFS API Principal:Enter the WebHDFS API Principal name. You can view it in the keytab file, such as klist-k-ttest.keytab. UDP Preference Limit:This parameter specifies the maximum value for UDP transmission packets, and data packets larger than this value are transmitted using TCP. The default value is 1, and TCP transmission is used by default. It should be adjusted according to the parameters in/etc/krb5. conf of the KDC service. krb5.keytab File:Retrieve the keytab file and copy it to a secure location on the host accessing the Vinchin system. Host:The IP of Hadoop Cluster NameNode. User:Users with read and write permissions for HDFS in Hadoop Clusters. REST API Port:The port number of the HTTP service for the hdfs-site.xml configuration file in the NameNode. Add NameNode: Can add multiple NameNode (HA). Proxy: You can choose a proxy that only works during the scanning phase when creating jobs and connecting to the cluster. The proxy is deployed on a physical server in the virtualization platform or data center, used to improve task efficiency and save network bandwidth. To enable the proxy, you can added in Resource > Infrastructure > Agent & Client.

Modify

Check the Hadoop cluster you want to modify and click the 'Modify' icon. Other information except for the host IP can be modified, and after modification, click OK to submit.

Delete

Check the Hadoop cluster to be deleted and click the Delete button. Click OK to delete the selected Hadoop cluster. If the Hadoop cluster has associated jobs, deletion operations will not be allowed.

Refresh

After adding a Hadoop cluster, the platform's automatic refresh time can be configured, with a default of once every 60 minutes. The refresh interval can be increased or decreased according to the actual situation.