Falcon provides feature to replicate Hive metadata and data events from source cluster to destination cluster. This is supported for both secure and unsecure cluster through Falcon extensions.
Following is the prerequisites to use Hive Mirrroring
Note: Set following properties in hive-site.xml for replicating the Hive events on source and destination Hive cluster:
<property> <name>hive.metastore.event.listeners</name> <value>org.apache.hive.hcatalog.listener.DbNotificationListener</value> <description>event listeners that are notified of any metastore changes</description> </property> <property> <name>hive.metastore.dml.events</name> <value>true</value> </property>
* Currently Hive doesn't support create database, roles, views, offline tables, direct HDFS writes without registering with metadata and Database/Table name mapping replication events. Hence Hive mirroring extension cannot be used to replicate above mentioned events between warehouses.
Perform initial bootstrap of Table and Database from source cluster to destination cluster
$FALCON_HOME/bin/falcon entity -submit -type cluster -file /cluster/definition.xml
Extension artifacts are expected to be installed on HDFS at the path specified by "extension.store.uri" in startup properties. hive-mirroring-properties.json file located at "<extension.store.uri>/hive-mirroring/META/hive-mirroring-properties.json" lists all the required and optional parameters/arguments for scheduling Hive mirroring job.