mirror of
https://github.com/apache/sqoop.git
synced 2025-05-11 22:41:50 +08:00
SQOOP-1824: Sqoop2: Create a common RDBMS table structure doc
(Banmeet Singh via Abraham Elmahrek)
This commit is contained in:
Abraham Elmahrek
parent
bf4ae0b3c7
commit
788fd54a3d
335
docs/src/site/sphinx/Repository.rst
Normal file
335
docs/src/site/sphinx/Repository.rst
Normal file
@ -0,0 +1,335 @@
|
||||
.. Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
contributor license agreements. See the NOTICE file distributed with
|
||||
this work for additional information regarding copyright ownership.
|
||||
The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
(the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
|
||||
==========
|
||||
Repository
|
||||
==========
|
||||
|
||||
This repository contains additional information regarding Sqoop.
|
||||
|
||||
|
||||
Sqoop Schema
|
||||
------------
|
||||
|
||||
The DDL queries that create the Sqoop repository schema in Derby database create the following tables:
|
||||
|
||||
|
||||
|
||||
SQ_SYSTEM
|
||||
+++++++++
|
||||
Store for various state information
|
||||
|
||||
+----------------------------+
|
||||
| SQ_SYSTEM |
|
||||
+============================+
|
||||
| SQM_ID: BIGINT PK |
|
||||
+----------------------------+
|
||||
| SQM_KEY: VARCHAR(64) |
|
||||
+----------------------------+
|
||||
| SQM_VALUE: VARCHAR(64) |
|
||||
+----------------------------+
|
||||
|
||||
|
||||
|
||||
|
||||
SQ_DIRECTION
|
||||
++++++++++++
|
||||
Directions
|
||||
|
||||
+---------------------------------------+-------------+
|
||||
| SQ_DIRECTION | |
|
||||
+=======================================+=============+
|
||||
| SQD_ID: BIGINT PK AUTO-GEN | |
|
||||
+---------------------------------------+-------------+
|
||||
| SQD_NAME: VARCHAR(64) | "FROM"|"TO" |
|
||||
+---------------------------------------+-------------+
|
||||
|
||||
|
||||
|
||||
|
||||
SQ_CONFIGURABLE
|
||||
+++++++++++++++
|
||||
Configurable registration
|
||||
|
||||
+-----------------------------+----------------------+
|
||||
| SQ_CONFIGURABLE | |
|
||||
+=============================+======================+
|
||||
| SQC_ID: BIGINT PK AUTO-GEN | |
|
||||
+-----------------------------+----------------------+
|
||||
| SQC_NAME: VARCHAR(64) | |
|
||||
+-----------------------------+----------------------+
|
||||
| SQC_CLASS: VARCHAR(255) | |
|
||||
+-----------------------------+----------------------+
|
||||
| SQC_TYPE: VARCHAR(32) | "CONNECTOR"|"DRIVER" |
|
||||
+-----------------------------+----------------------+
|
||||
| SQC_VERSION: VARCHAR(64) | |
|
||||
+-----------------------------+----------------------+
|
||||
|
||||
|
||||
|
||||
|
||||
SQ_CONNECTOR_DIRECTIONS
|
||||
+++++++++++++++++++++++
|
||||
Connector directions
|
||||
|
||||
+------------------------------+------------------------------+
|
||||
| SQ_CONNECTOR_DIRECTIONS | |
|
||||
+==============================+==============================+
|
||||
| SQCD_ID: BIGINT PK AUTO-GEN | |
|
||||
+------------------------------+------------------------------+
|
||||
| SQCD_CONNECTOR: BIGINT | FK SQCD_CONNECTOR(SQC_ID) |
|
||||
+------------------------------+------------------------------+
|
||||
| SQCD_DIRECTION: BIGINT | FK SQCD_DIRECTION(SQD_ID) |
|
||||
+------------------------------+------------------------------+
|
||||
|
||||
|
||||
|
||||
|
||||
SQ_CONFIG
|
||||
+++++++++
|
||||
Config details
|
||||
|
||||
+-------------------------------------+------------------------------------------------------+
|
||||
| SQ_CONFIG | |
|
||||
+=====================================+======================================================+
|
||||
| SQ_CFG_ID: BIGINT PK AUTO-GEN | |
|
||||
+-------------------------------------+------------------------------------------------------+
|
||||
| SQ_CFG_CONNECTOR: BIGINT | FK SQ_CFG_CONNECTOR(SQC_ID), NULL for driver |
|
||||
+-------------------------------------+------------------------------------------------------+
|
||||
| SQ_CFG_NAME: VARCHAR(64) | |
|
||||
+-------------------------------------+------------------------------------------------------+
|
||||
| SQ_CFG_TYPE: VARCHAR(32) | "LINK"|"JOB" |
|
||||
+-------------------------------------+------------------------------------------------------+
|
||||
| SQ_CFG_INDEX: SMALLINT | |
|
||||
+-------------------------------------+------------------------------------------------------+
|
||||
|
||||
|
||||
|
||||
|
||||
SQ_CONFIG_DIRECTIONS
|
||||
++++++++++++++++++++
|
||||
Connector directions
|
||||
|
||||
+------------------------------+------------------------------+
|
||||
| SQ_CONNECTOR_DIRECTIONS | |
|
||||
+==============================+==============================+
|
||||
| SQCD_ID: BIGINT PK AUTO-GEN | |
|
||||
+------------------------------+------------------------------+
|
||||
| SQCD_CONFIG: BIGINT | FK SQCD_CONFIG(SQ_CFG_ID) |
|
||||
+------------------------------+------------------------------+
|
||||
| SQCD_DIRECTION: BIGINT | FK SQCD_DIRECTION(SQD_ID) |
|
||||
+------------------------------+------------------------------+
|
||||
|
||||
|
||||
|
||||
|
||||
SQ_INPUT
|
||||
++++++++
|
||||
Input details
|
||||
|
||||
+----------------------------+--------------------------+
|
||||
| SQ_INPUT | |
|
||||
+============================+==========================+
|
||||
| SQI_ID: BIGINT PK AUTO-GEN | |
|
||||
+----------------------------+--------------------------+
|
||||
| SQI_NAME: VARCHAR(64) | |
|
||||
+----------------------------+--------------------------+
|
||||
| SQI_CONFIG: BIGINT | FK SQ_CONFIG(SQ_CFG_ID) |
|
||||
+----------------------------+--------------------------+
|
||||
| SQI_INDEX: SMALLINT | |
|
||||
+----------------------------+--------------------------+
|
||||
| SQI_TYPE: VARCHAR(32) | "STRING"|"MAP" |
|
||||
+----------------------------+--------------------------+
|
||||
| SQI_STRMASK: BOOLEAN | |
|
||||
+----------------------------+--------------------------+
|
||||
| SQI_STRLENGTH: SMALLINT | |
|
||||
+----------------------------+--------------------------+
|
||||
| SQI_ENUMVALS: VARCHAR(100) | |
|
||||
+----------------------------+--------------------------+
|
||||
|
||||
|
||||
|
||||
|
||||
SQ_LINK
|
||||
+++++++
|
||||
Stored links
|
||||
|
||||
+-----------------------------------+--------------------------+
|
||||
| SQ_LINK | |
|
||||
+===================================+==========================+
|
||||
| SQ_LNK_ID: BIGINT PK AUTO-GEN | |
|
||||
+-----------------------------------+--------------------------+
|
||||
| SQ_LNK_NAME: VARCHAR(64) | |
|
||||
+-----------------------------------+--------------------------+
|
||||
| SQ_LNK_CONNECTOR: BIGINT | FK SQ_CONNECTOR(SQC_ID) |
|
||||
+-----------------------------------+--------------------------+
|
||||
| SQ_LNK_CREATION_USER: VARCHAR(32) | |
|
||||
+-----------------------------------+--------------------------+
|
||||
| SQ_LNK_CREATION_DATE: TIMESTAMP | |
|
||||
+-----------------------------------+--------------------------+
|
||||
| SQ_LNK_UPDATE_USER: VARCHAR(32) | |
|
||||
+-----------------------------------+--------------------------+
|
||||
| SQ_LNK_UPDATE_DATE: TIMESTAMP | |
|
||||
+-----------------------------------+--------------------------+
|
||||
| SQ_LNK_ENABLED: BOOLEAN | |
|
||||
+-----------------------------------+--------------------------+
|
||||
|
||||
|
||||
|
||||
|
||||
SQ_JOB
|
||||
++++++
|
||||
Stored jobs
|
||||
|
||||
+--------------------------------+-----------------------+
|
||||
| SQ_JOB | |
|
||||
+================================+=======================+
|
||||
| SQB_ID: BIGINT PK AUTO-GEN | |
|
||||
+--------------------------------+-----------------------+
|
||||
| SQB_NAME: VARCHAR(64) | |
|
||||
+--------------------------------+-----------------------+
|
||||
| SQB_FROM_LINK: BIGINT | FK SQ_LINK(SQ_LNK_ID) |
|
||||
+--------------------------------+-----------------------+
|
||||
| SQB_TO_LINK: BIGINT | FK SQ_LINK(SQ_LNK_ID) |
|
||||
+--------------------------------+-----------------------+
|
||||
| SQB_CREATION_USER: VARCHAR(32) | |
|
||||
+--------------------------------+-----------------------+
|
||||
| SQB_CREATION_DATE: TIMESTAMP | |
|
||||
+--------------------------------+-----------------------+
|
||||
| SQB_UPDATE_USER: VARCHAR(32) | |
|
||||
+--------------------------------+-----------------------+
|
||||
| SQB_UPDATE_DATE: TIMESTAMP | |
|
||||
+--------------------------------+-----------------------+
|
||||
| SQB_ENABLED: BOOLEAN | |
|
||||
+--------------------------------+-----------------------+
|
||||
|
||||
|
||||
|
||||
|
||||
SQ_LINK_INPUT
|
||||
+++++++++++++
|
||||
N:M relationship link and input
|
||||
|
||||
+----------------------------+-----------------------+
|
||||
| SQ_LINK_INPUT | |
|
||||
+============================+=======================+
|
||||
| SQ_LNKI_LINK: BIGINT PK | FK SQ_LINK(SQ_LNK_ID) |
|
||||
+----------------------------+-----------------------+
|
||||
| SQ_LNKI_INPUT: BIGINT PK | FK SQ_INPUT(SQI_ID) |
|
||||
+----------------------------+-----------------------+
|
||||
| SQ_LNKI_VALUE: LONG VARCHAR| |
|
||||
+----------------------------+-----------------------+
|
||||
|
||||
|
||||
|
||||
|
||||
SQ_JOB_INPUT
|
||||
++++++++++++
|
||||
N:M relationship job and input
|
||||
|
||||
+----------------------------+---------------------+
|
||||
| SQ_JOB_INPUT | |
|
||||
+============================+=====================+
|
||||
| SQBI_JOB: BIGINT PK | FK SQ_JOB(SQB_ID) |
|
||||
+----------------------------+---------------------+
|
||||
| SQBI_INPUT: BIGINT PK | FK SQ_INPUT(SQI_ID) |
|
||||
+----------------------------+---------------------+
|
||||
| SQBI_VALUE: LONG VARCHAR | |
|
||||
+----------------------------+---------------------+
|
||||
|
||||
|
||||
|
||||
|
||||
SQ_SUBMISSION
|
||||
+++++++++++++
|
||||
List of submissions
|
||||
|
||||
+-----------------------------------+-------------------+
|
||||
| SQ_JOB_SUBMISSION | |
|
||||
+===================================+===================+
|
||||
| SQS_ID: BIGINT PK | |
|
||||
+-----------------------------------+-------------------+
|
||||
| SQS_JOB: BIGINT | FK SQ_JOB(SQB_ID) |
|
||||
+-----------------------------------+-------------------+
|
||||
| SQS_STATUS: VARCHAR(20) | |
|
||||
+-----------------------------------+-------------------+
|
||||
| SQS_CREATION_USER: VARCHAR(32) | |
|
||||
+-----------------------------------+-------------------+
|
||||
| SQS_CREATION_DATE: TIMESTAMP | |
|
||||
+-----------------------------------+-------------------+
|
||||
| SQS_UPDATE_USER: VARCHAR(32) | |
|
||||
+-----------------------------------+-------------------+
|
||||
| SQS_UPDATE_DATE: TIMESTAMP | |
|
||||
+-----------------------------------+-------------------+
|
||||
| SQS_EXTERNAL_ID: VARCHAR(50) | |
|
||||
+-----------------------------------+-------------------+
|
||||
| SQS_EXTERNAL_LINK: VARCHAR(150) | |
|
||||
+-----------------------------------+-------------------+
|
||||
| SQS_EXCEPTION: VARCHAR(150) | |
|
||||
+-----------------------------------+-------------------+
|
||||
| SQS_EXCEPTION_TRACE: VARCHAR(750) | |
|
||||
+-----------------------------------+-------------------+
|
||||
|
||||
|
||||
|
||||
|
||||
SQ_COUNTER_GROUP
|
||||
++++++++++++++++
|
||||
List of counter groups
|
||||
|
||||
+----------------------------+
|
||||
| SQ_COUNTER_GROUP |
|
||||
+============================+
|
||||
| SQG_ID: BIGINT PK |
|
||||
+----------------------------+
|
||||
| SQG_NAME: VARCHAR(75) |
|
||||
+----------------------------+
|
||||
|
||||
|
||||
|
||||
|
||||
SQ_COUNTER
|
||||
++++++++++
|
||||
List of counters
|
||||
|
||||
+----------------------------+
|
||||
| SQ_COUNTER |
|
||||
+============================+
|
||||
| SQR_ID: BIGINT PK |
|
||||
+----------------------------+
|
||||
| SQR_NAME: VARCHAR(75) |
|
||||
+----------------------------+
|
||||
|
||||
|
||||
|
||||
|
||||
SQ_COUNTER_SUBMISSION
|
||||
+++++++++++++++++++++
|
||||
N:M Relationship
|
||||
|
||||
+----------------------------+--------------------------------+
|
||||
| SQ_COUNTER_SUBMISSION | |
|
||||
+============================+================================+
|
||||
| SQRS_GROUP: BIGINT PK | FK SQ_COUNTER_GROUP(SQR_ID) |
|
||||
+----------------------------+--------------------------------+
|
||||
| SQRS_COUNTER: BIGINT PK | FK SQ_COUNTER(SQR_ID) |
|
||||
+----------------------------+--------------------------------+
|
||||
| SQRS_SUBMISSION: BIGINT PK | FK SQ_SUBMISSION(SQS_ID) |
|
||||
+----------------------------+--------------------------------+
|
||||
| SQRS_VALUE: BIGINT | |
|
||||
+----------------------------+--------------------------------+
|
||||
|
||||
|
||||
@ -60,7 +60,7 @@ If you are keen on contributing to Sqoop and get your hands dirty building conne
|
||||
- `Developing a Sqoop Connector with Connector API <ConnectorDevelopment.html>`_
|
||||
- `Developing Sqoop application with REST API <RESTAPI.html>`_
|
||||
- `Developing Sqoop application using Sqoop Java Client API <ClientAPI.html>`_
|
||||
|
||||
- `Repository <Repository.html>`_
|
||||
|
||||
Administrator Guide
|
||||
--------------------
|
||||
|
||||
Loading…
Reference in New Issue
Block a user