This change removes the ShimLoader and various Shim classes such as CDH3Shim
etc. It introduces a couple of new classes - ConfigurationConstants and
ConfigurationHelper - that provide a unique place for articulating interface
related details such as configuration keys that can likely change from version
to version of Hadoop.
From: Arvind Prabhakar <arvind@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149994 13f79535-47bb-0310-9956-ffa450edef68
Users currently need to specify hbase.home and zk.home in build.properties.
This change helps automatically resolve these dependencies by downloading
release tarballs. Would be best to do this via SQOOP-101 but the hbase and
zk maven layout currently has some issues that are painful to workaround
in ivy.
Reason: Improvement
Author: Eli Collins via Arvind Prabhakar
From: Arvind Prabhakar <arvind@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149986 13f79535-47bb-0310-9956-ffa450edef68
Sqoop now builds a sqoop-test-(version).jar file.
Added "jar-all" target to build.xml to build all artifacts.
Default ant target switched to "jar-all".
mvn-install, mvn-deploy updated deploy to deploy all main, test, and shim jars.
Added source jar artifacts to maven installation/deploy process.
Fixed build dependency bugs in generated poms.
Added Sqoop pom template; generating from ivy.xml was over-broad.
write-version-info.sh generates correct directory hierarchy in build/src/.
bin/sqoop and bin/configure-sqoop tweaked to account for multiple sqoop-* jars
in the $SQOOP_HOME directory.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149946 13f79535-47bb-0310-9956-ffa450edef68
Added FieldMappable and FieldMapProcessor interfaces.
Added ProcessingException class.
Added NullOutputCommitter class.
SqoopRecord now has delegate() method which calls a FieldMapProcessor.
ClassWriter now generates getFieldMap() method for SqoopRecords.
Added HBasePutProcessor to transform SqoopRecords into Put commands,
implementing FieldMapProcessor.
Added PutTransformer interface class and ToStringPutTransformer implementation.
Added DelegatingOutputFormat that uses a FieldMapProcessor.
Added HBase deps to build.xml via hbase.home property.
Added HBase, ZooKeeper to the dependency net added by configure-sqoop.
Added HBaseImportJob, HBaseImportMapper.
ImportJobBase now has jobSetup() step executed just before job submission.
ImportJobContext now holds a reference to the ConnManager.
DataDrivenImportJob retrieves ConnManager from ImportJobContext, it no longer
creates a new one.
Added HBase table import configuration parameters to SqoopOptions, ImportTool.
SqlManager.importQuery() needs to set ConnManager in ImportJobContext.
Added HBase import user documentation.
Described PutTransformer API in developer docs.
Added HBase unit tests.
Added ANT_ARGUMENTS env variable to Hudson test scripts to allow freeform parameters.
Added HBASE_HOME and ZOOKEEPER_HOME variables to hudson scripts.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149935 13f79535-47bb-0310-9956-ffa450edef68
Introduce LobFile format for storing large objects.
Implemented LobFile.Reader, LobFile.Writer classes.
Added a performance test of LobFile reading/writing speed.
Build system: fix cobertura build deps.
Remove unused utility classes from o.a.h.s.io.
Use LobFile for external storage in {B,C}lobRef.
Added LobReaderCache.
Converted BlobRef to read from LobFiles (through LobReaderCache).
LargeObjectLoader writes to LobFiles.
Common code from BlobRef and ClobRef factored out into LobRef abstract
base class.
Updated Test{B,C}lobRef and TestLargeObjectLoader for new external LOB storage.
Updated *ImportMappers to close LargeObjectLoaders when they're done.
Added performance tests to build.
Added script to run perf tests; factored out common logic into config script.
Fixed ivy dependency resolution to use multiple configuration inheritance.
Added LobFileStressTest.
Added readme with instructions to src/perftest directory.
Added CodecMap that abstracts compression codec classes to names.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149897 13f79535-47bb-0310-9956-ffa450edef68
Introduced SqoopTool interface.
Added cli package for option parsing:
includes RelatedOptions, ToolOptions, SqoopParser.
'Sqoop' is now a wrapper that invokes a SqoopTool.
Added setter methods for all fields of SqoopOptions.
Added commons-cli 1.2 build dependency.
Argument parsing is removed from SqoopOptions and pushed into individual tools.
Added HelpTool to display basic usage information for Sqoop and
usage for subcommands.
Added ImportTool to perform imports.
Added EvalSqlTool.
Added ExportTool.
Added ImportAllTablesTool.
Added ListDatabasesTool, ListTablesTool.
Added CodeGenTool.
Added CreateHiveTableTool.
Small changes to orm.ClassWriter.
Auto-generate bin scripts for all tools; include in release package.
Allow user to provide build properties in a file.
Shim use of GenericOptionsParser to allow cross-compilation.
Fix Hive testcases to pass under CDH.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149894 13f79535-47bb-0310-9956-ffa450edef68
