github/sqoop
5
0
Fork 0
mirror of https://github.com/apache/sqoop.git synced 2025-05-06 19:30:52 +08:00
Code Issues Projects Releases Wiki Activity
958 Commits 21 Branches 30 Tags 24 MiB
c173f6a52a
Commit Graph

203 Commits

Author SHA1 Message Date
Jarek Jarcec Cecho
51a07bc352 SQOOP-1374: From/To: Metadata upgrade 
(Abraham Elmahrek via Jarek Jarcec Cecho)
 2014-10-09 17:58:17 -07:00
Gwen Shapira
4d9513cea8 SQOOP-1375: Sqoop2: From/To: Create HDFS connector 2014-10-09 17:58:16 -07:00
Jarek Jarcec Cecho
fa3ec92342 SQOOP-1451: Sqoop2: From/To: Add API for supported directions 
(Abraham Elmahrek via Jarek Jarcec Cecho)
 2014-10-09 17:58:16 -07:00
Jarek Jarcec Cecho
b69f4aca3b SQOOP-1447: Sqoop2: From/To: Change DirectionError messages 
(Abraham Elmahrek via Jarek Jarcec Cecho)
 2014-10-09 17:48:31 -07:00
Jarek Jarcec Cecho
1ebf1b89b3 SQOOP-1417: Sqoop2: From/To: Change ConnectorType name 
(Abraham Elmahrek via Jarek Jarcec Cecho)
 2014-10-09 17:48:31 -07:00
Jarek Jarcec Cecho
2e79f9094c SQOOP-1416: Sqoop2: From/To: Investigate multiple variables for "from" and "to" 
(Abraham Elmahrek via Jarek Jarcec Cecho)
 2014-10-09 17:30:56 -07:00
Jarek Jarcec Cecho
1d4b9e96cf SQOOP-1419: Sqoop2: From/To: Check TO connection is enabled 
(Abraham Elmahrek via Jarek Jarcec Cecho)
 2014-10-09 17:30:55 -07:00
Abraham Elmahrek
4511fc1a99 SQOOP-1428: Sqoop2: From/To: Rebase against Sqoop2 branch for SQOOP-777 2014-10-09 17:30:55 -07:00
Jarek Jarcec Cecho
0ad7401bb5 SQOOP-1376: Sqoop2: From/To: Refactor connector interface 
(Abraham Elmahrek via Jarek Jarcec Cecho)
 2014-10-09 17:30:55 -07:00
Jarek Jarcec Cecho
f18a29bf17 SQOOP-1379: Sqoop2: From/To: Disable tests 
(Abraham Elmahrek via Jarek Jarcec Cecho)
 2014-10-09 17:30:55 -07:00
Jarek Jarcec Cecho
3c93930bf3 SQOOP-777: Sqoop2: Implement intermediate data format representation policy 
(Abraham Elmahrek via Jarek Jarcec Cecho)
 2014-07-26 11:37:50 -07:00
Hari Shreedharan
17c7219b94 SQOOP-1236. Sqoop2: Classpath generated by Submission engine should contain only unique elements. 
(Jarcec via Hari)
 2014-07-21 12:14:22 -07:00
Venkat Ranganathan
e60fda87d4 SOOP-1235: Sqoop2: Repository should be immutable by default 
(Jarek Jarcec Cecho via Venkat Ranganathan)
 2014-07-07 10:55:51 -07:00
Jarek Jarcec Cecho
4bf7e0029c SQOOP-1331: Add method "findConnectors" to repository API 
(Gwen Shapira via Jarek Jarcec Cecho)
 2014-06-04 10:00:18 -07:00
Venkat Ranganathan
0eb2352411 SQOOP-1290 Sqoop2: Kill Tomcat in case that Sqoop Server fails to load 
(Jarek Jarcec Cecho via Venkat Ranganathan)
 2014-02-24 10:58:29 -08:00
Hari Shreedharan
528d1ee9db SQOOP-1254. Sqoop2: Tool: Add Upgrade tool 
(Jarek Jarcec Cecho via Hari Shreedharan)
 2013-12-13 21:46:19 -08:00
Hari Shreedharan
eab812d2f2 SQOOP-1258. Sqoop2: FrameworkMetadataUpgrade can't handle new inputs 
(Jarek Jarcec Cecho via Hari Shreedharan)
 2013-12-13 19:24:47 -08:00
Hari Shreedharan
ebe62d3663 SQOOP-1189. Sqoop2: Ensure that clone methods will correctly copy over all values from all parents 
(Jarek Jarcec Cecho via Hari Shreedharan)
 2013-10-23 22:09:55 -07:00
Jarek Jarcec Cecho
581f9b798d SQOOP-1196: Sqoop2: Add support for arbitrary compression codecs 
(Raghav Kumar Gautam via Jarek Jarcec Cecho)
 2013-09-21 16:11:31 -07:00
Jarek Jarcec Cecho
08a829fd6c SQOOP-1182: Expose compression options for Sqoop2 import 
(Raghav Kumar Gautam via Jarek Jarcec Cecho)
 2013-09-09 01:33:16 -07:00
Hari Shreedharan
f343133db3 SQOOP-1184. Sqoop2: Improve error messages during metadata upgrade 
(Jarek Jarcec Cecho via Hari Shreedharan)
 2013-08-28 14:52:39 -07:00
Hari Shreedharan
da06408f23 SQOOP-1180. Sqoop2: Metadata upgrade is calling validators with Model classes instead of configuration objects 
(Jarek Jarcec Cecho via Hari Shreedharan)
 2013-08-21 21:11:56 -07:00
Jarek Jarcec Cecho
c07c857589 SQOOP-1142: Sqoop2: Provide creater and last edited by to metadata structures 
(Mengwei Ding via Jarek Jarcec Cecho)
 2013-08-14 15:27:48 -07:00
Jarek Jarcec Cecho
ee4e4c476f SQOOP-1143: Sqoop2: Provide server audit log for operations upon metadata structures 
(Mengwei Ding via Jarek Jarcec Cecho)
 2013-08-14 09:01:59 -07:00
Venkat Ranganathan
72f0ff1c74 SQOOP-1169: Sqoop2: org.apache.sqoop.repository.schema.immutable should default to false 
(Hari Shreedharan via Venkat Ranganathan)
 2013-08-09 13:48:57 -07:00
Jarek Jarcec Cecho
a7d9b1b338 SQOOP-1075: Sqoop2: Persist Framework metadata version in repository 
(Raghav Kumar Gautam via Jarek Jarcec Cecho)
 2013-07-29 20:27:16 -07:00
Jarek Jarcec Cecho
69ecc93d4a SQOOP-1141: Sqoop2: Provide ability to enable/disable connections and jobs 
(Mengwei Ding via Jarek Jarcec Cecho)
 2013-07-27 17:42:34 -07:00
Jarek Jarcec Cecho
abb8cf8754 SQOOP-996: Sqoop2: Upgrade: Create upgrade tests 
(Mengwei Ding via Jarek Jarcec Cecho)
 2013-07-21 09:45:54 -07:00
Jarek Jarcec Cecho
e5f664508e SQOOP-1139: Sqoop2: JobManager.java indentation is inconsistent with rest of the codebase 
(Venkat Ranganathan via Jarek Jarcec Cecho)
 2013-07-17 14:31:55 -07:00
Jarek Jarcec Cecho
1f016abb93 SQOOP-672: Unify logger usage across entire code base 
(Raghav Kumar Gautam via Jarek Jarcec Cecho)
 2013-07-11 17:28:44 -07:00
Hari Shreedharan
76fbc835c9 SQOOP-1111. Sqoop2: Implement version detection and structure updates into Derby repository 
(Jarek Jarcec Cecho via Hari Shreedharan)
 2013-07-11 17:22:28 -07:00
Hari Shreedharan
1b2441d21d SQOOP-1121. Sqoop2: Serialize schema and make them available in the MR job. 
(Jarek Jarcec Cecho via Hari Shreedharan)
 2013-07-10 10:57:54 -07:00
Venkat Ranganathan
251c8334fa SQOOP-1128: Sqoop2: SqoopException being nested into another SqoopException in Repository 
(Mengwei Ding via Venkat Ranganathan)
 2013-07-08 20:14:08 -07:00
Jarek Jarcec Cecho
96a02dfee9 SQOOP-994: Sqoop2: Upgrade: Add calling validation to the upgrade method 
(Mengwei Ding via Jarek Jarcec Cecho)
 2013-07-03 16:33:52 -07:00
Jarek Jarcec Cecho
8c74223b59 SQOOP-997: Sqoop2: Upgrade: Provide ability to disable the automatic upgrade 
(Mengwei Ding via Jarek Jarcec Cecho)
 2013-07-02 12:54:44 -07:00
Venkat Ranganathan
aa8e1e7794 SQOOP-1073: Sqoop2: Introduce schema for transferred data 
(Jarek Jarcec Cecho via Venkat Ranganathan)
 2013-06-30 21:05:50 -07:00
Jarek Jarcec Cecho
d62567ddf3 SQOOP-971: Sqoop2: Component reconfigurability 
(Mengwei Ding via Jarek Jarcec Cecho)
 2013-06-24 09:31:22 -07:00
Hari Shreedharan
575a86531d SQOOP-1059. Sqoop2: Remove ad hoc modle cloning methods in common module. 
(Jarek Jarcec Cecho via Hari Shreedharan)
 2013-06-23 21:41:01 -07:00
Kate Ting
681914c851 SQOOP-943: Sqoop2: Repository wide configuration of immutable disk structures 
(Jarek Jarcec Cecho via Kate Ting)
 2013-06-22 21:00:42 -04:00
Kate Ting
fd4be5f5e8 SQOOP-1076: Sqoop2: Introduce Boolean model type 
(Jarek Jarcec Cecho via Kate Ting)
 2013-06-22 20:43:33 -04:00
Jarek Jarcec Cecho
f0e8fee3e0 SQOOP-1088: Sqoop2: Submission history API 
(Mengwei Ding via Jarek Jarcec Cecho)
 2013-06-22 15:33:49 -07:00
Venkat Ranganathan
05bd42e184 SQOOP-1065: Sqoop2: Remove special lock object in FrameworkManager 
(Jarek Jarcec Cecho via Venkat Ranganathan)
 2013-06-22 09:51:43 -07:00
Jarek Jarcec Cecho
9920db114d SQOOP-1017: Sqoop2: Split FrameworkManager into two classes 
(Venkat Ranganathan via Jarek Jarcec Cecho)
 2013-06-20 15:02:56 -07:00
Jarek Jarcec Cecho
4d5e07e9d2 SQOOP-1081: destroy() method of ConnectorManager has not been implemented yet. 
(Mengwei Ding via Jarek Jarcec Cecho)
 2013-06-13 20:28:01 -07:00
Jarek Jarcec Cecho
662f18a0a3 SQOOP-1069: Sqoop2: Incorrect error message from FrameworkValidator 
(Mengwei Ding via Jarek Jarcec Cecho)
 2013-06-06 17:11:54 -07:00
Kate Ting
c0a43d436f SQOOP-1041: Sqoop2: Framework upgrade in JdbcRepository returns the outdated value 
(Jarek Jarcec Cecho via Kate Ting)
 2013-05-13 00:46:06 -04:00
Kate Ting
b3ebfdda61 SQOOP-1044: Sqoop2: Improve logging in metadata upgrade methods 
(Jarek Jarcec Cecho via Kate Ting)
 2013-05-13 00:31:33 -04:00
Jarek Jarcec Cecho
66dd617da2 SQOOP-998: Sqoop2: Upgrade: Add framework upgrader 
(Hari Shreedharan via Jarek Jarcec Cecho)
 2013-05-03 12:27:33 -07:00
Jarek Jarcec Cecho
4f318755dd SQOOP-1025: Sqoop metadata upgrade must set job id in the new job being created 
(Hari Shreedharan via Jarek Jarcec Cecho)
 2013-04-30 21:12:56 -07:00
Jarek Jarcec Cecho
7440af2c58 SQOOP-1022: Metadata upgrade fails with NPE getConnector method 
(Hari Shreedharan via Jarek Jarcec Cecho)
 2013-04-30 15:21:22 -07:00
Jarek Jarcec Cecho
c4467c6770 SQOOP-659: Design metadata upgrade procedure 
(Hari Shreedharan via Jarek Jarcec Cecho)
 2013-04-18 21:15:28 -07:00
Cheolsoo Park
df3a266c6c SQOOP-892:: Validate acceptable number of mappers and reducers 
(Jarcec Cecho via Cheolsoo Park)
 2013-02-23 19:23:27 -08:00
Cheolsoo Park
cc5ae8f817 SQOOP-877: Increase field size of output.outputDirectory property value from 25 to 50 
(Jarcec Cecho via Cheolsoo Park)
 2013-02-22 13:26:54 -08:00
Cheolsoo Park
217cf297b1 SQOOP-880: NPE when stopping job that was never executed 
(Jarcec Cecho via Cheolsoo Park)
 2013-02-10 13:27:18 -08:00
Cheolsoo Park
ed9c514361 SQOOP-864: Introduce ETL context objects 
(Jarcec Cecho via Cheolsoo Park)
 2013-02-10 12:55:31 -08:00
Cheolsoo Park
687e13050c SQOOP-866: Introduce framework validations 
(Jarcec Cecho via Cheolsoo Park)
 2013-02-06 13:55:22 -08:00
Cheolsoo Park
8f074b07fc SQOOP-850: Move server initialization from server module to core 
(Jarcec Cecho via Cheolsoo Park)
 2013-01-31 09:47:56 -08:00
Cheolsoo Park
554e5aeffe SQOOP-841: Remove final keyword from manager classes 
(Jarcec Cecho via Cheolsoo Park)
 2013-01-30 15:43:43 -08:00
Cheolsoo Park
52dece8fa8 SQOOP-826: Add log4j test configuration to all maven modules 
(Jarek Jarcec Cecho via Cheolsoo Park)
 2013-01-22 15:53:42 -08:00
Cheolsoo Park
4c2a343d96 SQOOP-678: Add counters handling to map reduce submission engine 
(Jarek Jarcec Cecho via Cheolsoo Park)
 2013-01-04 02:50:02 -08:00
Bilung Lee
46b50e3f68 SQOOP-802 Convert main "Manager" classes to singletons 
(Jarek Jarcec Cecho)
 2012-12-26 12:38:39 -08:00
Cheolsoo Park
9ab7ea6ee0 SQOOP-765: JDBC repository is incorrectly checking if job exists in 
update handler

(Jarek Jarcec Cecho via Cheolsoo Park)
 2012-12-12 13:49:26 -08:00
Bilung Lee
671d84934e SQOOP-739 Connectors should be versionable 
(Jarek Jarcec Cecho)
 2012-12-07 20:04:46 -08:00
Bilung Lee
b16ca584f6 SQOOP-696 Throw exception for execution of non-existing job 
(Jarek Jarcec Cecho)
 2012-12-07 19:19:10 -08:00
Bilung Lee
9ac49d95c2 SQOOP-740 Provide export submission implementation 
(Jarek Jarcec Cecho)
 2012-12-07 15:07:34 -08:00
Bilung Lee
a633fb0b34 SQOOP-737 Option to set number of extractors and loaders 
(Jarek Jarcec Cecho)
 2012-12-04 12:28:23 -08:00
Bilung Lee
cf3d71049d SQOOP-735 Introduce output format option to Sqoop import 
(Jarek Jarcec Cecho)
 2012-12-03 18:03:31 -08:00
Bilung Lee
8a5cd6728a SQOOP-734 Change ConnectorBean to send JSON objects 
(Jarek Jarcec Cecho)
 2012-12-03 17:54:29 -08:00
Bilung Lee
0e7451f94f SQOOP-716 Create server notification REST callback 
(Jarek Jarcec Cecho)
 2012-11-26 08:55:20 -08:00
Bilung Lee
ef12bf508f SQOOP-677 Destroyer needs to be called from OutputCommitter 
(Jarek Jarcec Cecho)
 2012-11-26 08:45:02 -08:00
Bilung Lee
13c5c0689f SQOOP-673 Call update() only in job running status 
(Jarek Jarcec Cecho)
 2012-11-26 08:39:08 -08:00
Bilung Lee
211c67819e SQOOP-662 Introduce pluggable configuration objects 
(Jarek Jarcec Cecho)
 2012-11-13 20:34:18 -08:00
Bilung Lee
5f95f114f6 SQOOP-692 Move specific error codes to own error codes class 
(Jarek Jarcec Cecho)
 2012-11-12 11:43:20 -08:00
Bilung Lee
fa815557c5 SQOOP-646 Provide support for "set" metadata in Sqoop 2 
(Jarek Jarcec Cecho)
 2012-11-09 18:27:09 -08:00
Bilung Lee
30a1af0ebf SQOOP-684 Encode type of the job into executed map reduce job 
(Jarek Jarcec Cecho)
 2012-11-09 10:11:57 -08:00
Bilung Lee
c47a15368e SQOOP-686 Empty job name will result in NPE during job submission 
(Jarek Jarcec Cecho)
 2012-11-09 09:59:47 -08:00
Bilung Lee
379451fe3c SQOOP-676 Add lastUpdateDate to MSubmission 
(Jarek Jarcec Cecho)
 2012-11-06 16:25:39 -08:00
Bilung Lee
06e054bc8d SQOOP-666 Introduce execution engine 
(Jarek Jarcec Cecho)
 2012-11-06 12:43:03 -08:00
Bilung Lee
2481b7f8d0 SQOOP-656 End to end submission engine 
(Jarek Jarcec Cecho)
 2012-11-02 15:29:33 -07:00
Jarek Jarcec Cecho
7fbd3ba94b SQOOP-610: Job submission engine for import 
(Bilung Lee via Jarek Jarcec Cecho)
 2012-10-19 13:28:18 -07:00
Bilung Lee
6cd9e96888 SQOOP-630: Check if in-use connection/job before deletion 
(Jarek Jarcec Cecho)
 2012-10-17 09:42:06 -07:00
Bilung Lee
5807db8c3e SQOOP-629: Provide better exception handling during server-client communication 
(Jarek Jarcec Cecho)
 2012-10-17 09:38:15 -07:00
Bilung Lee
a46b25f5e5 SQOOP-627: Convert forms structures on connector level to configuration classes 
(Jarek Jarcec Cecho)
 2012-10-17 09:23:39 -07:00
Jarek Jarcec Cecho
27aa786793 SQOOP-533: Intermediate data format support for import 
(Bilung Lee via Jarek Jarcec Cecho)
 2012-10-15 15:07:38 -07:00
Bilung Lee
38dc198a5d SQOOP-618 Remove most important FindBugs warnings and errors in Sqoop 2 
(Jarek Jarcec Cecho)
 2012-10-08 18:25:07 -07:00
Jarek Jarcec Cecho
d4b4df8825 SQOOP-605: Generic JDBC connector for import 
(Bilung Lee via Jarek Jarcec Cecho)
 2012-10-08 16:28:18 -07:00
Bilung Lee
f1893ab9bb SQOOP-620: Introduce name to connection and job objects 
(Jarek Cecho)
 2012-10-05 13:22:32 -07:00
Bilung Lee
13439909f3 SQOOP-613 Add support for integer type to metadata model 2012-10-03 16:42:07 -07:00
Bilung Lee
866d46dfe7 SQOOP-608 Implement job resource from end to end 2012-09-20 14:47:31 -07:00
Jarek Jarcec Cecho
3a88f0b48c SQOOP-589: Framework-defined text/sequence loaders for HDFS. 
(Bilung Lee via Jarek Jarcec Cecho)
 2012-09-18 09:21:08 +02:00
Bilung Lee
47cb311a5f SQOOP-596: Implement connection resource end-to-end 2012-09-07 12:40:34 -07:00
Jarek Jarcec Cecho
50e8622caf SQOOP-588. Provide MapReduce classes for executing ETL classes. 
(Bilung Lee via Jarek Jarcec Cecho)


git-svn-id: https://svn.apache.org/repos/asf/sqoop/branches/sqoop2@1379307 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-31 06:24:05 +00:00
Bilung Lee
c1e44563d5 SQOOP-576 Implement command show framework 
git-svn-id: https://svn.apache.org/repos/asf/sqoop/branches/sqoop2@1377752 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-27 17:16:19 +00:00
Bilung Lee
c32afa5aea SQOOP-497: Enhance repository interface to support Connection CRUD 
git-svn-id: https://svn.apache.org/repos/asf/sqoop/branches/sqoop2@1376202 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-22 19:25:51 +00:00
Bilung Lee
ab2313875b SQOOP-544: Split job metadata for different job types (import, export) 
git-svn-id: https://svn.apache.org/repos/asf/sqoop/branches/sqoop2@1373069 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-14 20:37:26 +00:00
Bilung Lee
7158e2ff27 SQOOP-568: Configuration reload period should be configurable in PropertiesConfigurationProvider 
git-svn-id: https://svn.apache.org/repos/asf/sqoop/branches/sqoop2@1372557 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-13 19:23:28 +00:00
Bilung Lee
12e049b4ec SQOOP-567 Support connection validation in JDBC repository 
git-svn-id: https://svn.apache.org/repos/asf/sqoop/branches/sqoop2@1371512 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-09 22:10:35 +00:00
Bilung Lee
517c2c1bb7 SQOOP-535: Support splitting metadata to connector and framework specific 
git-svn-id: https://svn.apache.org/repos/asf/sqoop/branches/sqoop2@1370814 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-08 15:41:15 +00:00
Bilung Lee
867e18fb02 SQOOP-534: Add ability to find out if MPersistableEntity do already have persistent ID 
git-svn-id: https://svn.apache.org/repos/asf/sqoop/branches/sqoop2@1368623 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-02 17:51:30 +00:00
Jarek Jarcec Cecho
e19d63b625 SQOOP-502. Implement show connector command end-to-end. 
(Bilung Lee via Jarek Jarcec Cecho)


git-svn-id: https://svn.apache.org/repos/asf/sqoop/branches/sqoop2@1360620 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-12 10:34:34 +00:00
Bilung Lee
498da2b799 SQOOP-511 Need to set persistent id for connector if in repository already 
git-svn-id: https://svn.apache.org/repos/asf/sqoop/branches/sqoop2@1356323 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-02 16:27:45 +00:00
Bilung Lee
ed631e9250 SQOOP-365 Show connector command 
git-svn-id: https://svn.apache.org/repos/asf/sqoop/branches/sqoop2@1346742 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-06 05:57:26 +00:00
Daniel Shahaf
b71b4caccd move sqoop to TLP 
git-svn-id: https://svn.apache.org/repos/asf/sqoop/branches/sqoop2@1306066 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-28 00:04:41 +00:00