This change removes the ShimLoader and various Shim classes such as CDH3Shim
etc. It introduces a couple of new classes - ConfigurationConstants and
ConfigurationHelper - that provide a unique place for articulating interface
related details such as configuration keys that can likely change from version
to version of Hadoop.
From: Arvind Prabhakar <arvind@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149994 13f79535-47bb-0310-9956-ffa450edef68
This fix allows the user to optionally specify the connection
manager class to be used, instead of inferring it from the
connection string.
From: Ahmed Radwan <ahmed@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149993 13f79535-47bb-0310-9956-ffa450edef68
This change fixes a problem that prevented users from importing into Hive
data extracted using a free form query.
From: Ahmed Radwan <ahmed@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149992 13f79535-47bb-0310-9956-ffa450edef68
This change allows customized ExportRecordWriters the opportunity to execute
code after the last commit is performed and before the database connection
is closed.
(Guy le Mar via Arvind Prabhakar)
From: Arvind Prabhakar <arvind@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149991 13f79535-47bb-0310-9956-ffa450edef68
This is a fix for a case of malformed command line arguments,
where the tool name is correct, the option is also correct,
but the value of the option is missing.
From: Ahmed Radwan <ahmed@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149990 13f79535-47bb-0310-9956-ffa450edef68
Hive allows the use of keywords as column and table names as long as they are
escaped using back-ticks. This change makes Sqoop always escape table and
column names using back-ticks thereby allowing Sqoop to work with Hive tables
that use keywords for either the table name or column names.
(Lars Francke via arvind)
From: Arvind Prabhakar <arvind@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149989 13f79535-47bb-0310-9956-ffa450edef68
The TextSplitter implementation used when creating splits on top
of String based columns, has a bug in its logic which causes
the bounds for splits to be created incorrectly. This results
in the import of duplicate data. This change fixes the TextSplitter
in order to ensure that the bound checks are created correctly.
From: Arvind Prabhakar <arvind@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149988 13f79535-47bb-0310-9956-ffa450edef68
This change removes the MySQL JDBC driver distribution that was
bundled with Sqoop previously. This is done to make sure that the
Sqoop distribution is completely Apache 2.0 compliant.
From: Arvind Prabhakar <arvind@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149987 13f79535-47bb-0310-9956-ffa450edef68
Users currently need to specify hbase.home and zk.home in build.properties.
This change helps automatically resolve these dependencies by downloading
release tarballs. Would be best to do this via SQOOP-101 but the hbase and
zk maven layout currently has some issues that are painful to workaround
in ivy.
Reason: Improvement
Author: Eli Collins via Arvind Prabhakar
From: Arvind Prabhakar <arvind@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149986 13f79535-47bb-0310-9956-ffa450edef68
This change modifies the ClassWriter implementation to provide
support for interpeting empty strings as NULL values for datatypes
other than String. For String datatype, an explicit string 'null'
is interpreted as NULL value and empty string is not. This is
because certain databases distinguish between NULL and empty
strings.
The clone implementation generated by ClassWriter has also been
modified to make it more defensive against the presence of NULL
values.
From: Arvind Prabhakar <arvind@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149985 13f79535-47bb-0310-9956-ffa450edef68
Current CDH3 build includes version 0.20.3 of Hadoop which is now
mapped to CDH3Shim loader. Apart from that, this change includes
a change in build.xml and OracleUtils test class that allows the
ability to override connect string for Oracle tests.
From: Arvind Prabhakar <arvind@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149984 13f79535-47bb-0310-9956-ffa450edef68
Adds 'merge' tool.
Adds MergeJob, Merge*Mapper, MergeReducer.
Merge-specific arguments added to SqoopOptions, BaseSqoopTool.
Add TestMerge to test that this tool functions as expected.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149980 13f79535-47bb-0310-9956-ffa450edef68
Create nonce dirs as /tmp/sqoop-${user.name}/compile.
Remove unused compilation directories at end of execution.
Do not memoize nonce compilation directory names in metastore.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149975 13f79535-47bb-0310-9956-ffa450edef68
Added start-metastore.sh and stop-metastore.sh scripts, which start and stop
the metastore silently.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149972 13f79535-47bb-0310-9956-ffa450edef68
Eliminates "webdocs" target from build, merged with "docs" target.
Web-based format for document delivery is used for local documentation as well.
Links to release notes, API documentation added to documentation homepage.
Release notes are generated for non-SNAPSHOT builds as part of "package" step.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149971 13f79535-47bb-0310-9956-ffa450edef68
Updated the "supported databases" section of the user guide to reflect the
current driver installation process.
Updated "support" section of the user guide to mention current issue tracker.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149969 13f79535-47bb-0310-9956-ffa450edef68
Added manual pages and user guide sections for sqoop-job and sqoop-metastore.
Updated sqoop-import documentation to describe incremental imports.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149968 13f79535-47bb-0310-9956-ffa450edef68
Files in conf/managers.d/ are treated as configuration files that specify
classes for sqoop.connection.factories. If this property is unset, these
files are processed in order.
ClassLoaderStack no longer attempts to load a jar if the test class is
already available with the current set of ClassLoaders.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149966 13f79535-47bb-0310-9956-ffa450edef68
Modify sqoop-eval to use generic execute() instead of
executeQuery()/executeUpdate().
The --update argument to sqoop-eval is unnecessary and has been removed.
Modify ResultSetPrinter to properly display left border on output.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149963 13f79535-47bb-0310-9956-ffa450edef68
Added ManagerFactory.accept(SessionData) API to allow ManagerFactory
to inspect the chosen SqoopTool.
Deprecated ManagerFactory.accept(SqoopOptions).
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149958 13f79535-47bb-0310-9956-ffa450edef68
Add com.cloudera.sqoop.io.NamedFifo class to represent named FIFO objects.
Added TestNamedFifo as unit test.
MySQLExportMapper now uses this utility.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149954 13f79535-47bb-0310-9956-ffa450edef68
ImportJobContext and ExportJobContext now contain references to the
already-instantiated ConnManager. JobBase subclasses should no
longer use a ConnFactory, but should use the existing ConnManager.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149953 13f79535-47bb-0310-9956-ffa450edef68
Added --update argument to sqoop-eval tool, which runs
the user's statement in update mode rather than select mode.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149952 13f79535-47bb-0310-9956-ffa450edef68
Restart MiniHBaseCluster between each test to prevent triggering
livelock at end of test battery.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149950 13f79535-47bb-0310-9956-ffa450edef68
.class and .jar generation occurs in a per-process anonymous
directory. Generation of .java files now occurs in the bin directory;
.java files are moved to the src directory after compilation.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149947 13f79535-47bb-0310-9956-ffa450edef68
Sqoop now builds a sqoop-test-(version).jar file.
Added "jar-all" target to build.xml to build all artifacts.
Default ant target switched to "jar-all".
mvn-install, mvn-deploy updated deploy to deploy all main, test, and shim jars.
Added source jar artifacts to maven installation/deploy process.
Fixed build dependency bugs in generated poms.
Added Sqoop pom template; generating from ivy.xml was over-broad.
write-version-info.sh generates correct directory hierarchy in build/src/.
bin/sqoop and bin/configure-sqoop tweaked to account for multiple sqoop-* jars
in the $SQOOP_HOME directory.
From: Aaron Kimball <aaron@cloudera.com>
git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149946 13f79535-47bb-0310-9956-ffa450edef68
