5
0
mirror of https://github.com/apache/sqoop.git synced 2025-05-18 09:51:11 +08:00
Commit Graph

14 Commits

Author SHA1 Message Date
Andrew Bayer
3509b7941e SQOOP-90. Tool to merge datasets imported via incremental import.
Adds 'merge' tool.
Adds MergeJob, Merge*Mapper, MergeReducer.
Merge-specific arguments added to SqoopOptions, BaseSqoopTool.
Add TestMerge to test that this tool functions as expected.

From: Aaron Kimball <aaron@cloudera.com>

git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149980 13f79535-47bb-0310-9956-ffa450edef68
2011-07-22 20:04:17 +00:00
Andrew Bayer
5cc48f3051 SQOOP-51. Document driver jar installation process.
Updated the "supported databases" section of the user guide to reflect the
current driver installation process.
Updated "support" section of the user guide to mention current issue tracker.

From: Aaron Kimball <aaron@cloudera.com>

git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149969 13f79535-47bb-0310-9956-ffa450edef68
2011-07-22 20:04:14 +00:00
Andrew Bayer
d656663a14 SQOOP-42. Document saved jobs, metastore, and incremental imports.
Added manual pages and user guide sections for sqoop-job and sqoop-metastore.
Updated sqoop-import documentation to describe incremental imports.

From: Aaron Kimball <aaron@cloudera.com>

git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149968 13f79535-47bb-0310-9956-ffa450edef68
2011-07-22 20:04:14 +00:00
Andrew Bayer
66c753c2e7 SQOOP-72. sqoop-eval should use PreparedStatement.execute().
Modify sqoop-eval to use generic execute() instead of
executeQuery()/executeUpdate().
The --update argument to sqoop-eval is unnecessary and has been removed.
Modify ResultSetPrinter to properly display left border on output.

From: Aaron Kimball <aaron@cloudera.com>

git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149963 13f79535-47bb-0310-9956-ffa450edef68
2011-07-22 20:04:12 +00:00
Andrew Bayer
692ee04f14 SQOOP-59. 'sqoop eval' should support DDL/DML operations.
Added --update argument to sqoop-eval tool, which runs
the user's statement in update mode rather than select mode.

From: Aaron Kimball <aaron@cloudera.com>

git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149952 13f79535-47bb-0310-9956-ffa450edef68
2011-07-22 20:04:09 +00:00
Andrew Bayer
f4eba3dd8c SQOOP-32. sqoop-export documentation in SqoopUserGuide refers to 'import-args'.
From: Aaron Kimball <aaron@cloudera.com>

git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149936 13f79535-47bb-0310-9956-ffa450edef68
2011-07-22 20:04:04 +00:00
Andrew Bayer
7e0ccb4008 SQOOP-14. Allow Sqoop to import data into HBase.
Added FieldMappable and FieldMapProcessor interfaces.
Added ProcessingException class.
Added NullOutputCommitter class.
SqoopRecord now has delegate() method which calls a FieldMapProcessor.
ClassWriter now generates getFieldMap() method for SqoopRecords.
Added HBasePutProcessor to transform SqoopRecords into Put commands,
implementing FieldMapProcessor.
Added PutTransformer interface class and ToStringPutTransformer implementation.
Added DelegatingOutputFormat that uses a FieldMapProcessor.
Added HBase deps to build.xml via hbase.home property.
Added HBase, ZooKeeper to the dependency net added by configure-sqoop.
Added HBaseImportJob, HBaseImportMapper.
ImportJobBase now has jobSetup() step executed just before job submission.
ImportJobContext now holds a reference to the ConnManager.
DataDrivenImportJob retrieves ConnManager from ImportJobContext, it no longer
creates a new one.
Added HBase table import configuration parameters to SqoopOptions, ImportTool.
SqlManager.importQuery() needs to set ConnManager in ImportJobContext.
Added HBase import user documentation.
Described PutTransformer API in developer docs.
Added HBase unit tests.
Added ANT_ARGUMENTS env variable to Hudson test scripts to allow freeform parameters.
Added HBASE_HOME and ZOOKEEPER_HOME variables to hudson scripts.

From: Aaron Kimball <aaron@cloudera.com>

git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149935 13f79535-47bb-0310-9956-ffa450edef68
2011-07-22 20:04:03 +00:00
Andrew Bayer
6ea1665716 SQOOP-4. Add update-mode export process.
Add --update-key argument to sqoop-export tool.
Refactor ExportOutputFormat into AsyncSqlOutputFormat.
Added UpdateOutputFormat.
ClassWriter now allows alternate serialization order in database write() method.
SqoopOptions holds column list for alternate db serialization order.
Added TestExportUpdate unit test battery.
AsyncSqlRecordWriter now allows "batch" execution mode.
Updated documentation for export updates.

From: Aaron Kimball <aaron@cloudera.com>

git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149933 13f79535-47bb-0310-9956-ffa450edef68
2011-07-22 20:04:02 +00:00
Andrew Bayer
b0cd632b58 SQOOP-3. Add ability to import arbitrary queries.
Add ConnManager.importQuery() API.
Change BaseSqoopTool.DEBUG_SQL_CMD_ARG to SQL_QUERY_ARG to reflect
the broader applicability of the argument.
Change 'debugSqlCmd' member of SqoopOptions to 'sqlQuery'.
CompilationManager now sets jar name based on specified class name.
Add tests for query support.
Add documentation for query-based import.

From: Aaron Kimball <aaron@cloudera.com>

git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149932 13f79535-47bb-0310-9956-ffa450edef68
2011-07-22 20:04:02 +00:00
Andrew Bayer
3edcf20d33 Add webdocs target to build release docs website.
From: Aaron Kimball <aaron@cloudera.com>

git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149922 13f79535-47bb-0310-9956-ffa450edef68
2011-07-22 20:03:59 +00:00
Andrew Bayer
1d80c4ded7 Minor manpage fixes.
From: Aaron Kimball <aaron@cloudera.com>

git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149915 13f79535-47bb-0310-9956-ffa450edef68
2011-07-22 20:03:57 +00:00
Andrew Bayer
1f9ca86a2f Add documentation for --append and --target-dir.
From: Aaron Kimball <aaron@cloudera.com>

git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149914 13f79535-47bb-0310-9956-ffa450edef68
2011-07-22 20:03:57 +00:00
Andrew Bayer
0abcdb0af9 Add new manual pages for all Sqoop tools.
From: Aaron Kimball <aaron@cloudera.com>

git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149903 13f79535-47bb-0310-9956-ffa450edef68
2011-07-22 20:03:50 +00:00
Andrew Bayer
faabc51a90 Rewrite documentation for updated Sqoop API
Add documentation for all SqoopTool implementations.
Add database compatibility notes.
Separate user guide from the develpment guide.

From: Aaron Kimball <aaron@cloudera.com>

git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149902 13f79535-47bb-0310-9956-ffa450edef68
2011-07-22 20:03:50 +00:00