github/sqoop
5
0
Fork 0
mirror of https://github.com/apache/sqoop.git synced 2025-05-04 19:09:42 +08:00
Code Issues Projects Releases Wiki Activity

SQOOP-1556: Sqoop2: Add documentation clarifying connectors vs. engines

Browse Source 
(Gwen Shapira via Jarek Jarcec Cecho)
This commit is contained in:
Jarek Jarcec Cecho 2014-09-26 16:12:22 -07:00 committed by Abraham Elmahrek
parent 35a060e048
commit 196346d5cc
1 changed files with 14 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
docs/src/site/sphinx/ConnectorDevelopment.rst
View File

@ -31,6 +31,20 @@ Connector reads data from databases for import,
and write data to databases for export.
Interaction with Hadoop is taken cared by common modules of Sqoop 2 framework.
When do we add a new connector?
===============================
You add a new connector when you need to extract data from a new data source, or load
data to a new target.
In addition to the connector API, Sqoop 2 also has an engine interface.
At the moment the only engine is MapReduce,but we may support additional engines in the future.
Since many parallel execution engines are capable of reading/writing data
there may be a question of whether support for specific data stores should be done
through a new connector or new engine.
**Our guideline is:** Connectors should manage all data extract/load. Engines manage job
life cycles. If you need to support a new data store and don't care how jobs run -
you are looking to add a connector.
Connector Implementation
++++++++++++++++++++++++