github/sqoop
5
0
Fork 0
mirror of https://github.com/apache/sqoop.git synced 2025-05-05 07:01:02 +08:00
Code Issues Projects Releases Wiki Activity

SQOOP-1556: Sqoop2: Add documentation clarifying connectors vs. engines

Browse Source 
(Gwen Shapira via Jarek Jarcec Cecho)
This commit is contained in:
Jarek Jarcec Cecho 2014-09-26 16:12:22 -07:00 committed by Abraham Elmahrek
parent 35a060e048
commit 196346d5cc
1 changed files with 14 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
docs/src/site/sphinx/ConnectorDevelopment.rst
View File

@ -31,6 +31,20 @@ Connector reads data from databases for import,
and write data to databases for export. and write data to databases for export.
Interaction with Hadoop is taken cared by common modules of Sqoop 2 framework. Interaction with Hadoop is taken cared by common modules of Sqoop 2 framework.
When do we add a new connector?
===============================
You add a new connector when you need to extract data from a new data source, or load
data to a new target.
In addition to the connector API, Sqoop 2 also has an engine interface.
At the moment the only engine is MapReduce,but we may support additional engines in the future.
Since many parallel execution engines are capable of reading/writing data
there may be a question of whether support for specific data stores should be done
through a new connector or new engine.
**Our guideline is:** Connectors should manage all data extract/load. Engines manage job
life cycles. If you need to support a new data store and don't care how jobs run -
you are looking to add a connector.
Connector Implementation Connector Implementation
++++++++++++++++++++++++ ++++++++++++++++++++++++