github/sqoop
5
0
Fork 0
mirror of https://github.com/apache/sqoop.git synced 2025-05-21 11:21:39 +08:00
Code Issues Projects Releases Wiki Activity

SQOOP-2287: Sqoop2: Create TO direction documentation

Browse Source 
(Abraham Elmahrek via Jarek Jarcec Cecho)
This commit is contained in:
Jarek Jarcec Cecho 2015-04-08 18:10:16 -07:00
parent 06f79093f5
commit 7d9f6c840e
1 changed files with 46 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

46
docs/src/site/sphinx/Connectors.rst
View File

@ -340,3 +340,49 @@ Destroyers
----------
The HDFS TO destroyer moves all created files to the proper output directory.
+++++++++++++++
Kafka Connector
+++++++++++++++
Currently, only the TO direction is supported.
-----
Usage
-----
To use the Kafka Connector, create a link for the connector and a job that uses the link.
**Link Configuration**
++++++++++++++++++++++
Inputs associated with the link configuration include:
+----------------------+---------+-----------------------------------------------------------+-------------------------------------+
| Input | Type | Description | Example |
+======================+=========+===========================================================+=====================================+
| Broker list | String | Comma separated list of kafka brokers. | example.com:10000,example.com:11000 |
| | | *Required*. | |
+----------------------+---------+-----------------------------------------------------------+-------------------------------------+
| Zookeeper connection | String | Comma separated list of zookeeper servers in your quorum. | /etc/conf/hadoop |
| | | *Required*. | |
+----------------------+---------+-----------------------------------------------------------+-------------------------------------+
**TO Job Configuration**
++++++++++++++++++++++++
Inputs associated with the Job configuration for the FROM direction include:
+-------+---------+---------------------------------+----------+
| Input | Type | Description | Example |
+=======+=========+=================================+==========+
| topic | String | The Kafka topic to transfer to. | my topic |
| | | *Required*. | |
+-------+---------+---------------------------------+----------+
------
Loader
------
During the *loading* phase, Kafka is written to directly from each loader. The order in which data is loaded into Kafka is not guaranteed.