Added FieldMappable and FieldMapProcessor interfaces.
Added ProcessingException class.
Added NullOutputCommitter class.
SqoopRecord now has delegate() method which calls a FieldMapProcessor.
ClassWriter now generates getFieldMap() method for SqoopRecords.
Added HBasePutProcessor to transform SqoopRecords into Put commands,
implementing FieldMapProcessor.
Added PutTransformer interface class and ToStringPutTransformer implementation.
Added DelegatingOutputFormat that uses a FieldMapProcessor.
Added HBase deps to build.xml via hbase.home property.
Added HBase, ZooKeeper to the dependency net added by configure-sqoop.
Added HBaseImportJob, HBaseImportMapper.
ImportJobBase now has jobSetup() step executed just before job submission.
ImportJobContext now holds a reference to the ConnManager.
DataDrivenImportJob retrieves ConnManager from ImportJobContext, it no longer
creates a new one.
Added HBase table import configuration parameters to SqoopOptions, ImportTool.
SqlManager.importQuery() needs to set ConnManager in ImportJobContext.
Added HBase import user documentation.
Described PutTransformer API in developer docs.
Added HBase unit tests.
Added ANT_ARGUMENTS env variable to Hudson test scripts to allow freeform parameters.
Added HBASE_HOME and ZOOKEEPER_HOME variables to hudson scripts.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149935 13f79535-47bb-0310-9956-ffa450edef68
Ensure that tests involving dates/times use proper ANSI SQL Date/Time escape
formatting (yyyy-mm-dd or hh:mm:ss). After Java 1.6u17, dates of the form
yyyy-m-dd or hhⓂ️ss are not parsed by java.sql.Date/Time and throw
IllegalArgumentException.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149934 13f79535-47bb-0310-9956-ffa450edef68
Add --update-key argument to sqoop-export tool.
Refactor ExportOutputFormat into AsyncSqlOutputFormat.
Added UpdateOutputFormat.
ClassWriter now allows alternate serialization order in database write() method.
SqoopOptions holds column list for alternate db serialization order.
Added TestExportUpdate unit test battery.
AsyncSqlRecordWriter now allows "batch" execution mode.
Updated documentation for export updates.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149933 13f79535-47bb-0310-9956-ffa450edef68
Add ConnManager.importQuery() API.
Change BaseSqoopTool.DEBUG_SQL_CMD_ARG to SQL_QUERY_ARG to reflect
the broader applicability of the argument.
Change 'debugSqlCmd' member of SqoopOptions to 'sqlQuery'.
CompilationManager now sets jar name based on specified class name.
Add tests for query support.
Add documentation for query-based import.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149932 13f79535-47bb-0310-9956-ffa450edef68
Maven pushes sqoop jar and shim jars.
Renamed shim jar files to conform to maven artifact-version naming.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149925 13f79535-47bb-0310-9956-ffa450edef68
bin/configure-sqoop now puts the 'conf' directory on the classpath.
Users can override this by setting $SQOOP_CONF_DIR.
Added src/perftest/ExtConnFactoryTest and ExtFactory, that demonstrates
the configuration is being correctly read.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149921 13f79535-47bb-0310-9956-ffa450edef68
If the Hive CliDriver is present on the classpath, Sqoop will
call it directly in the same JVM, rather than invoke a separate
JVM instance for Hive.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149916 13f79535-47bb-0310-9956-ffa450edef68
Comment blocks (for the license header) must appear at the end
of a man page asciidoc file; they cannot appear near the top.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149910 13f79535-47bb-0310-9956-ffa450edef68
Moves TaskId to com.cloudera.sqoop.util.
Add com.cloudera.sqoop.lib.DelimiterSet.
Rewrite FieldFormatter, RecordParser, to use DelimiterSet.
Add generated class version id to SqoopRecord.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149907 13f79535-47bb-0310-9956-ffa450edef68
Add documentation for all SqoopTool implementations.
Add database compatibility notes.
Separate user guide from the develpment guide.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149902 13f79535-47bb-0310-9956-ffa450edef68
Introduce LobFile format for storing large objects.
Implemented LobFile.Reader, LobFile.Writer classes.
Added a performance test of LobFile reading/writing speed.
Build system: fix cobertura build deps.
Remove unused utility classes from o.a.h.s.io.
Use LobFile for external storage in {B,C}lobRef.
Added LobReaderCache.
Converted BlobRef to read from LobFiles (through LobReaderCache).
LargeObjectLoader writes to LobFiles.
Common code from BlobRef and ClobRef factored out into LobRef abstract
base class.
Updated Test{B,C}lobRef and TestLargeObjectLoader for new external LOB storage.
Updated *ImportMappers to close LargeObjectLoaders when they're done.
Added performance tests to build.
Added script to run perf tests; factored out common logic into config script.
Fixed ivy dependency resolution to use multiple configuration inheritance.
Added LobFileStressTest.
Added readme with instructions to src/perftest directory.
Added CodecMap that abstracts compression codec classes to names.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149897 13f79535-47bb-0310-9956-ffa450edef68
Introduced SqoopTool interface.
Added cli package for option parsing:
includes RelatedOptions, ToolOptions, SqoopParser.
'Sqoop' is now a wrapper that invokes a SqoopTool.
Added setter methods for all fields of SqoopOptions.
Added commons-cli 1.2 build dependency.
Argument parsing is removed from SqoopOptions and pushed into individual tools.
Added HelpTool to display basic usage information for Sqoop and
usage for subcommands.
Added ImportTool to perform imports.
Added EvalSqlTool.
Added ExportTool.
Added ImportAllTablesTool.
Added ListDatabasesTool, ListTablesTool.
Added CodeGenTool.
Added CreateHiveTableTool.
Small changes to orm.ClassWriter.
Auto-generate bin scripts for all tools; include in release package.
Allow user to provide build properties in a file.
Shim use of GenericOptionsParser to allow cross-compilation.
Fix Hive testcases to pass under CDH.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149894 13f79535-47bb-0310-9956-ffa450edef68
OracleConnManager now nullifies references to connection instances
after returning them to the connection cache or closing them.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149891 13f79535-47bb-0310-9956-ffa450edef68
