sqoop

mirror of https://github.com/apache/sqoop.git synced 2025-05-02 12:49:54 +08:00

Go to file

Andrew Bayer 7e0ccb4008 SQOOP-14. Allow Sqoop to import data into HBase. Added FieldMappable and FieldMapProcessor interfaces. Added ProcessingException class. Added NullOutputCommitter class. SqoopRecord now has delegate() method which calls a FieldMapProcessor. ClassWriter now generates getFieldMap() method for SqoopRecords. Added HBasePutProcessor to transform SqoopRecords into Put commands, implementing FieldMapProcessor. Added PutTransformer interface class and ToStringPutTransformer implementation. Added DelegatingOutputFormat that uses a FieldMapProcessor. Added HBase deps to build.xml via hbase.home property. Added HBase, ZooKeeper to the dependency net added by configure-sqoop. Added HBaseImportJob, HBaseImportMapper. ImportJobBase now has jobSetup() step executed just before job submission. ImportJobContext now holds a reference to the ConnManager. DataDrivenImportJob retrieves ConnManager from ImportJobContext, it no longer creates a new one. Added HBase table import configuration parameters to SqoopOptions, ImportTool. SqlManager.importQuery() needs to set ConnManager in ImportJobContext. Added HBase import user documentation. Described PutTransformer API in developer docs. Added HBase unit tests. Added ANT_ARGUMENTS env variable to Hudson test scripts to allow freeform parameters. Added HBASE_HOME and ZOOKEEPER_HOME variables to hudson scripts. From: Aaron Kimball <aaron@cloudera.com> git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149935 13f79535-47bb-0310-9956-ffa450edef68		2011-07-22 20:04:03 +00:00
bin	SQOOP-14. Allow Sqoop to import data into HBase.	2011-07-22 20:04:03 +00:00
conf	Move org.apache.hadoop.sqoop to com.cloudera.sqoop	2011-07-22 20:03:52 +00:00
ivy	SQOOP-14. Allow Sqoop to import data into HBase.	2011-07-22 20:04:03 +00:00
lib	SQOOP-16. Generate Eclipse project with ant.	2011-07-22 20:04:01 +00:00
src	SQOOP-14. Allow Sqoop to import data into HBase.	2011-07-22 20:04:03 +00:00
testdata/hive	Changed license headers to reference Cloudera instead of the ASF.	2011-07-22 20:03:36 +00:00
.gitignore	SIP-1. Refactor operations into distinct tools.	2011-07-22 20:03:45 +00:00
build.xml	SQOOP-14. Allow Sqoop to import data into HBase.	2011-07-22 20:04:03 +00:00
COMPILING.txt	SQOOP-16. Generate Eclipse project with ant.	2011-07-22 20:04:01 +00:00
ivy.xml	SQOOP-14. Allow Sqoop to import data into HBase.	2011-07-22 20:04:03 +00:00
LICENSE.txt	Changed license headers to reference Cloudera instead of the ASF.	2011-07-22 20:03:36 +00:00
NOTICE.txt	Changed license headers to reference Cloudera instead of the ASF.	2011-07-22 20:03:36 +00:00
README.txt	Add ability to compile against Cloudera or Apache Hadoop.	2011-07-22 20:03:41 +00:00
svn-placeholder	Placeholder commit	2011-07-22 19:59:57 +00:00

README.txt

= Welcome to Sqoop!

This is the Sqoop (SQL-to-Hadoop) tool. Sqoop allows easy imports and
exports of data sets between databases and HDFS.


== More Documentation

Sqoop ships with additional documentation: a user guide and a manual page.

Asciidoc sources for both of these are in +src/docs/+. Run +ant docs+ to build
the documentation. It will be created in +build/docs/+.

If you got Sqoop in release form, documentation will already be built and
available in the +docs/+ directory.


== Compiling Sqoop

Compiling Sqoop requires the following tools:

* Apache ant (1.7.1)
* Java JDK 1.6

Additionally, building the documentation requires these tools:

* asciidoc
* make
* python 2.5+
* xmlto
* tar
* gzip

To compile Sqoop, run +ant package+. There will be a fully self-hosted build
provided in the +build/sqoop-(version)/+ directory. 

You can build just the jar by running +ant jar+.

See the COMPILING.txt document for for information.

== This is also an Asciidoc file!

* Try running +asciidoc README.txt+
* For more information about asciidoc, see http://www.methods.co.nz/asciidoc/