Hive is a data warehousing tool based on Hadoop. As we know Hadoop provides massive scale out on distributed infrastructure with high degree of fault tolerance for data storage and processing. Hadoop uses Map Reduce algorithm to process huge amount of data with minimal cost as it does not require high end machines to process such amount of data. Hive processor converts most of its queries into a Map Reduce job which runs on Hadoop cluster. Hive is designed for easy and effective data aggregation, ad-hoc querying and analysis of huge volumes of data.
Hive Is Not
Even though Hive gives SQL dialect it does not give SQL like latency as it ultimately runs Map Reduce programs underneath. As we all know, Map Reduce framework is built for batch processing jobs it has high latency, even the fastest hive query would take several minutes to get executed on relatively smaller set of data in few megabytes. We cannot simply compare the performance of traditional SQL systems like Oracle, MySQL or SQL Server as these systems are meant to do something and Hive is meant to do else. Hive aims to provide acceptable (but not optimal) latency for interactive querying over small data sets for sample queries.
hive is not an OLTP (Online transaction Processing) application and not meant to be connected with systems which needs interactive processing. It is meant to be used to process batch jobs on huge data which is immutable. A good example of such kind of data would be Web logs, Application Logs, call data records (CDR) etc.
Features of Hive
- Stores schema in database and processed data into HDFS.
- Designed for OLAP
- Provides SQL type language for querying called HiveQL
- Fast, Scalable and extensible
Hive supports Data definition Language(DDL), Data Manipulation Language(DML) and user defined functions.
- DDL: create table, create index, create views.
- DML: Select, Where, group by, Join, Order By
- Pluggable Functions:
- UDF: User Defined Function
- UDAF: User Defined Aggregate Function
- UDTF: User Defined Table Function