All classes which depend on MapReduce APIs which change from
interfaces to classes between 0.20 and 0.22 are moved to distribution-
specific shim jars.
"Common" shim classes are now compiled multiple times against different
Hadoop distributions.
Shim classes are broken out into separate jars; ShimLoader now picks
the appropriate jar to load at runtime.
Configuration constants moved into HadoopShim.
BlobRef/ClobRef methods changed to use Mapper.Context for binary compatibility.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149884 13f79535-47bb-0310-9956-ffa450edef68
Version-incompatible code now moved to HadoopShim subclasses.
HadoopShim singleton instance dynamically loaded based on VersionInfo.
Separate MRUnit builds from Apache and CDH placed in /lib subdirs.
Modified 'ant package' target to properly include all shims.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149880 13f79535-47bb-0310-9956-ffa450edef68
Using --direct in conjunction with --export-dir on a MySQL database will use
mysqlimport to emit the data to the database.
DirectMySQLManager now creates instances of MySQLExportJob.
src/test/.../MySQLUtils is renamed to MySQLTestUtils to avoid conflict with
src/java/.../MySQLUtils added by this patch.
MySQLUtils contains methods factored out of import-specific code for sharing
with exports.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149877 13f79535-47bb-0310-9956-ffa450edef68
To use this, run:
ant cobertura -Dcobertura.home=/path/to/cobertura
That will run the standard smoke tests. To get a full report, you'll
need to re-run it on the thirdparty tests:
ant cobertura -Dcobertura.home=/path/to/cobertura -Dthirdparty=true \
-Dsqoop.thirdparty.lib.dir=/path/to/jdbc/drivers
The results of both runs will be combined.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149876 13f79535-47bb-0310-9956-ffa450edef68
Add 'JobBase' which attempts to unify ExportJob with ImportJobBase.
ImportJobBase and ExportJobBase override job-type specific behavior.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149875 13f79535-47bb-0310-9956-ffa450edef68
Some spurious warnings (and inconsequential warnings in test code)
have been disabled by src/test/findbugsExcludeFile.xml.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149874 13f79535-47bb-0310-9956-ffa450edef68
OracleManager now caches Connection instances for subsequent OracleManager
instances.
Refactored uses of ConnManager to call close() before discarding them.
This allows the Oracle JUnit tests to sleep less frequently to wait for Oracle
to reap closed server-side connection resources, improving Oracle test speed
by 50%.
Sleeping cannot be fully eliminated because MapReduce-side Connections are not
governed by this caching mechanism.
Also added some debugging advice re. this topic to OracleManagerTest's comment.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149872 13f79535-47bb-0310-9956-ffa450edef68
Uses CombineFileInputFormat to run exports over a target number
of mappers independent of the number of input files.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149869 13f79535-47bb-0310-9956-ffa450edef68
CLOB/BLOB data may now be stored in additional files in HDFS which are
accessible through streams if the data cannot be fully materialized in RAM.
Adds tests for external large objects.
Refactored large object loading into the map() method from readFields().
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149866 13f79535-47bb-0310-9956-ffa450edef68
Major refactoring of DataDrivenImportJob to support mysqldump in mappers.
ImportJobBase added below DataDrivenImportJob.
MySQLDumpImportJob added on top of ImportJobBase.
LocalMySQLManager -> renamed to -> DirectMySQLManager now just runs MysqldumpIJ.
MySQLDumpImportJob configures MySQLDumpMapper to run mysqldump instances on
multiple nodes and is split-aware (via MySQLDumpInputFormat).
TestImportJob works with new ImportJobBase framework.
Added test that imports a subset of columns in mysql imports.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149865 13f79535-47bb-0310-9956-ffa450edef68
Modified build.xml to run without Hadoop's build-contrib wrapper.
Added MRUnit jar from Hadoop MapReduce (not exposed via mvn).
Added 'package' and 'tar' targets for redistribution.
Added ivy settings files for direct dependencies.
Added gitignores where appropriate.
Move documentation from /doc to /src/docs.
Add LICENSE.txt.
Move readme.txt to README.txt.
Provide more fine-grained control of third-party redistributables
via 'redist' ivy configuration.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149861 13f79535-47bb-0310-9956-ffa450edef68
