github/sqoop
5
0
Fork 0
mirror of https://github.com/apache/sqoop.git synced 2025-05-16 08:51:02 +08:00
Code Issues Projects Releases Wiki Activity
852 Commits 21 Branches 30 Tags 24 MiB
a50869437e
Commit Graph

715 Commits

Author SHA1 Message Date
Attila Szabo
7783f85f80 SQOOP-3083: Fixing fault injection targets to work 
together with try with resources statements
(introduced in SQOOP-2983)

(Anna Szonyi via Attila Szabo)
 2016-12-16 11:59:22 +01:00
Attila Szabo
be30a344ee SQOOP-3075: Simplify Unicode character support in 
source files (introduced by SQOOP-3074) by
defining explicit locales instead of using
EscapeUtils

(Attila Szabo)
 2016-12-16 11:48:52 +01:00
Attila Szabo
5771a2da5f SQOOP-3074: Fix Avro import not to fail with Javac 
errors in case of non UTF-8 locale

(Attila Szabo)
 2016-12-08 23:07:31 +01:00
Attila Szabo
642b41a9e8 SQOOP-3072: Reenable escaping in 
ImportTest#testProductWithWhiteSpaceImport for proper execution

(Attila Szabo)
 2016-12-06 17:54:52 +01:00
Attila Szabo
fe67a0a5c0 SQOOP-3071: Fix OracleManager to apply localTimeZone correctly in case of Date objects too 
(Attila Szabo)
 2016-12-06 17:53:31 +01:00
Attila Szabo
bd51d987b8 SQOOP-3069: Get OracleExportTest#testUpsertTestExport in line with SQOOP-3066 
(Attila Szabo)
 2016-12-06 13:52:01 +01:00
Attila Szabo
3e5bab49ea SQOOP-3066: Introduce an option + env variable to 
enable/disable SQOOP-2737 feature

(Attila Szabo)
 2016-12-05 20:25:50 +01:00
Attila Szabo
0e26d92a34 SQOOP-3053: Create a cmd line argument for 
sqoop.throwOnError and use it through SqoopOptions

(Boglarka Egyed via Attila Szabo)
 2016-12-02 15:34:50 +01:00
Attila Szabo
916cae19f8 SQOOP-3057: Fixing 3rd party Oracle tests failing 
due to invalid case of column names

(Anna Szonyi via Attila Szabo)
 2016-11-29 19:06:19 +01:00
Attila Szabo
48cecdc99e SQOOP-3055: Fixing MySQL tests failing due to 
ignored test inputs/configuration

(Anna Szonyi via Attila Szabo)
 2016-11-29 15:33:16 +01:00
Attila Szabo
b0b7b4779c SQOOP-2349: Add command line option for setting 
transaction isolation levels for metadata queries
 2016-11-23 21:45:03 +01:00
Attila Szabo
3fba26419b SQOOP-3027: Create fail fast for export with --hcatalog-table <HIVE_VIEW>, as it's not supported by HCat 
(Anna Szonyi via Attila Szabo)
 2016-11-23 11:39:49 +01:00
Attila Szabo
c08c194eb9 SQOOP-3028: Include stack trace in the logging of exceptions in ExportTool 
(Boglarka Egyed via Attila Szabo)
 2016-11-22 11:59:16 +01:00
Venkat Ranganathan
7d6667ddb6 SQOOP-2990: Sqoop(oracle) export [updateTableToOracle] with "--update-mode allowinsert" : app fails with java.sql.SQLException: Missing IN or OUT parameter at index 
(Sowmya Ramesh via Venkat Ranganathan)
 2016-11-11 14:06:21 -08:00
Attila Szabo
8e50060849 SQOOP-3013: Configuration "tmpjars" is not checked for empty strings before passing to MR 
(Erzsebet Szilagyi via Attila Szabo)
 2016-11-10 15:00:19 +01:00
Attila Szabo
64fd3d2d32 SQOOP-3038: export to table with "value" column causes all null values in the "value" column 
(Anna Szonyi via Attila Szabo)
 2016-11-08 19:10:00 +01:00
Attila Szabo
c3e868ce90 SQOOP-3044: Add missing ASF license information to .java files 
(Anna Szonyi via Attila Szabo)
 2016-11-08 18:30:13 +01:00
Attila Szabo
53b490e146 SQOOP-3034: HBase import should fail fast if using anything other than as-textfile 
(Boglarka Egyed via Attila Szabo)
 2016-11-08 18:23:52 +01:00
Venkat Ranganathan
b60aef1ffb SQOOP-3033: Sqoop option --skip-dist-cache is not saved as a parameter when saving Sqoop Job 
(Sowmya Ramesh via Venkat Ranganathan)
 2016-10-28 20:39:10 -07:00
Jarek Jarcec Cecho
3fc4ff7144 SQOOP-2983: OraOop export has degraded performance with wide tables 
(Attila Szabo via Jarek Jarcec Cecho)
 2016-10-26 12:20:17 -07:00
Attila Szabo
14754342d3 SQOOP-2986: Add validation check for --hive-import and --incremental lastmodified 
(Szabolcs Vasas via Attila Szabo)
 2016-10-13 15:21:56 +02:00
Attila Szabo
b4afcf4179 SQOOP-2952: Fixing bug (row key not added into column family using --hbase-bulkload) 
(Szabolcs Vasas via Attila Szabo)
 2016-10-13 14:38:21 +02:00
Attila Szabo
0f13c474bf SQOOP-3021: ClassWriter fails if a column name contains a backslash character 
(Szabolcs Vasas via Attila Szabo)
 2016-10-07 01:59:06 +02:00
Venkatesan Ranganathan
8f3989d9ad SQOOP-3010: Sqoop should not allow --as-parquetfile with hcatalog jobs or when hive import with create-hive-table is used 
(Sowmya Ramesh via Venkat Ranganathan)
 2016-09-26 05:34:13 -07:00
Attila Szabo
b007e4d59d SQOOP-2913: Make sqoop fail if user uses --direct connector for case when --direct connector is not available 
(Anna Szonyi via Attila Szabo)
 2016-09-23 16:30:36 +02:00
Venkat Ranganathan
b9794f98e3 SQOOP-2936: Provide Apache Atlas integration for hcatalog based exports 
(Balu Vellanki via Venkat Ranganathan)
 2016-08-20 23:40:28 -07:00
Kate Ting
3bd2952a9b SQOOP-2995: backward incompatibility introduced by Custom Tool options 
(Szabolcs Vasas via Kate Ting)
 2016-08-15 22:55:05 -07:00
Kate Ting
c8f9beec1d SQOOP-2884: Document --temporary-rootdir 
(Markus Kemper via Kate Ting)
 2016-08-09 17:46:40 -07:00
Kate Ting
7808c6bc3e SQOOP-2971: OraOop does not close connections properly 
(Szabolcs Vasas via Kate Ting)
 2016-07-01 16:00:03 -07:00
Venkat Ranganathan
c339b23b6c SQOOP-2950: Sqoop trunk has consistent UT failures - need fixing 
(Venkat Ranganathan via Venkat Ranganathan)
 2016-06-13 17:29:02 +05:30
Kate Ting
b302d89fae SQOOP-2938: Mainframe import module extension to support data sets on tape (Chris Teoh via Kate Ting) 2016-06-06 17:31:17 -07:00
Jarek Jarcec Cecho
28bbe4d461 SQOOP-2920: sqoop performance deteriorates significantly on wide datasets; sqoop 100% on cpu 
(Attila Szabo via Jarek Jarcec Cecho)
 2016-05-19 07:27:35 -07:00
Jarek Jarcec Cecho
5779aec031 SQOOP-2906: Optimization of AvroUtil.toAvroIdentifier 
(Joeri Hermans via Jarek Jarcec Cecho)
 2016-05-19 07:26:57 -07:00
Jarek Jarcec Cecho
ac217a032c Revert "SQOOP-2920: sqoop performance deteriorates significantly on wide datasets; sqoop 100% on cpu" 
I've mistakenly committed SQOOP-2920 and SQOOP-2906 inside this commit,
so I'll revert it and commit them separately.
 2016-05-19 07:26:12 -07:00
Jarek Jarcec Cecho
83f0c1a40b SQOOP-2911: Fix failing HCatalogExportTest caused by SQOOP-2863 
(Abraham Fine via Jarek Jarcec Cecho)
 2016-05-18 13:32:22 -07:00
Jarek Jarcec Cecho
db4a8fc2fe SQOOP-2920: sqoop performance deteriorates significantly on wide datasets; sqoop 100% on cpu 
(Attila Szabo via Jarek Jarcec Cecho)
 2016-05-18 13:27:55 -07:00
Jarek Jarcec Cecho
db8c247000 SQOOP-2915: Fixing Oracle related unit tests 
(Attila Szabo via Jarek Jarcec Cecho)
 2016-04-26 13:41:39 -07:00
Jarek Jarcec Cecho
660f3e8ad0 SQOOP-2910: Add capability to Sqoop to require an explicit option to be specified with --split-by for a String column 
(Attila Szabo via Jarek Jarcec Cecho)
 2016-04-19 15:54:58 -07:00
Jarek Jarcec Cecho
dc25684318 SQOOP-2909: Oracle related ImportTest fails after SQOOP-2737 
(Attila Szabo via Jarek Jarcec Cecho)
 2016-04-18 08:50:37 -07:00
Venkat Ranganathan
729e681f10 SQOOP-2896: Sqoop exec job fails with SQLException Access denied for user 
(Sowmya Ramesh via Venkat Ranganathan)
 2016-03-29 18:02:19 -07:00
Jarek Jarcec Cecho
1dd50cfb2a SQOOP-2561: Special Character removal from Column name as avro data results in duplicate column and fails the import 
(VISHNU S NAIR via Jarek Jarcec Cecho)
 2016-03-29 09:26:47 -07:00
Jarek Jarcec Cecho
0c8b105481 SQOOP-2846: Sqoop Export with update-key failing for avro data file 
(VISHNU S NAIR via Jarek Jarcec Cecho)
 2016-03-21 10:07:38 -07:00
Jarek Jarcec Cecho
a0b730c77e SQOOP-2880: Provide argument for overriding temporary directory 
(Attila Szabo via Jarek Jarcec Cecho)
 2016-03-14 09:21:16 -07:00
Venkat Ranganathan
152fe93002 SQOOP-2864: ClassWriter chokes on column names containing double quotes 
(Jarek Jarcec Cecho via Venkat Ranganathan)
 2016-03-04 11:37:40 -08:00
Venkat Ranganathan
d21699e2ab SQOOP-2863 Properly escape column names for generated INSERT statements 
(Jarek Jarcec Cecho via Venkat Ranganathan)
 2016-03-04 11:17:51 -08:00
Jarek Jarcec Cecho
2eecf68010 SQOOP-2647: Add option for drop-if-exists when using sqoop hcat import. 
(Muhammad Ehsan ul Haque via Jarek Jarcec Cecho)
 2016-03-04 08:50:15 -08:00
Hari Shreedharan
c2351511e6 SQOOP-2847. Sqoop --incremental + missing parent --target-dir reports success with no data 
(Jarcec via Hari)
 2016-03-01 10:42:21 -08:00
Venkat Ranganathan
5c4d06d727 SQOOP-2609: Provide Apache Atlas integration for hive and hcatalog based imports. 
(Balu Vellanki via Venkat Ranganathan)
 2016-02-29 15:20:13 -08:00
Jarek Jarcec Cecho
cfa7927c2a SQOOP-2839: Sqoop import failure due to data member conflict in ORM code for table 
(VISHNU S NAIR  via Jarek Jarcec Cecho)
 2016-02-24 08:46:24 -08:00
Venkat Ranganathan
b84ebf1e84 SQOOP-2787: MySql import and export fails with 5.1 server and 5.1.17+ drivers 
(Sowmya Ramesh via Venkat Ranganathan)
 2016-02-19 22:38:34 -08:00
Venkat Ranganathan
4e0149b3ed SQOOP-2737: Cannot import table from Oracle with column with spaces in name 
(Sowmya Ramesh via Venkat Ranganathan)
 2016-02-19 22:11:17 -08:00
Venkat Ranganathan
39feebff2a SQOOP-2779: Sqoop metastore doesn't seem to recognize --schema option 
(Sowmya Ramesh via Venkat Ranganathan)
 2016-01-31 20:19:30 -08:00
Jarek Jarcec Cecho
d25eb2e190 SQOOP-1493: Add ability to import/export true decimal in Avro instead of serializing it to String 
(Abraham Elmahrek via Jarek Jarcec Cecho)
 2016-01-29 08:42:23 -08:00
Jarek Jarcec Cecho
c9c3f6a3a2 SQOOP-2642: Document ability to specify commas in --map-column-hive option 
(Ze Jin via Jarek Jarcec Cecho)
 2016-01-27 06:35:22 -08:00
Jarek Jarcec Cecho
de2fa9a686 SQOOP-2795: Clean up useless code in class TestSqoopJsonUtil 
(Ze Jin via Jarek Jarcec Cecho)
 2016-01-25 13:15:19 -08:00
Kate Ting
43319f1d71 SQOOP-2753: TestSqoopJsonUtil.testGetJsonStringFromMap is depending on Map ordering in JDK 
(Jarek Jarcec Cecho via Kate Ting)
 2016-01-18 11:15:33 -08:00
Kate Ting
55d8b4a7b0 SQOOP-2767: Test is failing SystemImportTest 
(Jarek Jarcec Cecho via Kate Ting)
 2016-01-16 14:18:58 -08:00
Kate Ting
926d92bac1 SQOOP-2783: Query import with parquet fails on incompatible schema 
(Jarek Jarcec Cecho via Kate Ting)
 2016-01-15 16:00:34 -08:00
Jarek Jarcec Cecho
1e310ad9f5 SQOOP-2780: Sqoop 1 unit tests fail with TestIncrementalImport test 
(Sowmya Ramesh via Jarek Jarcec Cecho)
 2016-01-06 17:20:55 -08:00
Venkat Ranganathan
f19e2a5235 SQOOP-2747: Allow customizing test username and password for Oracle tests 
(Jarek Jarcec Cecho via Venkat Ranganathan)
 2015-12-16 20:49:31 -08:00
Venkat Ranganathan
e4f12a99bf SQOOP-2746: Add test case for Oracle incremental import using Timestamp 
(Jarek Jarcec Cecho via Venkat Ranganathan)
 2015-12-16 20:48:46 -08:00
Venkat Ranganathan
9c7638d741 SQOOP-2745: Using datetime column as a splitter for Oracle no longer works 
(Jarek Jarcec Cecho via Venkat Ranganathan)
 2015-12-16 20:47:59 -08:00
Kate Ting
e077f99783 SQOOP-2712: Run only one map task attempt during export (second edition) 
(Jarek Jarcec Cecho via Kate Ting)
 2015-12-11 21:25:03 -08:00
Jarek Jarcec Cecho
203d6b7267 SQOOP-2723: Oracle connector not working with lowercase columns 
(David Robson via Jarek Jarcec Cecho)
 2015-12-08 14:27:39 +01:00
Hari Shreedharan
77021d2a61 SQOOP-2651. Do not dump data on error in TextExportMapper by default 
(Jarcec via Hari)
 2015-11-09 21:55:25 -08:00
Hari Shreedharan
408358930d SQOOP-2617. Log file that is being used for LOB files 
(Jarcec via Hari)
 2015-10-15 13:56:07 -07:00
Jarek Jarcec Cecho
5d8152eac8 SQOOP-2607: Direct import from Netezza and encoding 
(Benjamin BONNET via Jarek Jarcec Cecho)
 2015-10-09 11:56:07 -07:00
Jarek Jarcec Cecho
02e36db2b8 SQOOP-1281: Support of glob paths during export 
(Clément MAHTIEU via Jarek Jarcec Cecho)
 2015-10-06 08:51:20 -07:00
Hari Shreedharan
967f0eb0f3 SQOOP-2597. Missing method AvroSchemaGenerator.generate() 
(Jarcec via Hari)
 2015-10-01 14:07:54 -04:00
Jarek Jarcec Cecho
6987d290fe SQOOP-2387: Sqoop should support importing from table with column names containing some special character 
(Venkat Ranganathan via Jarek Jarcec Cecho)
 2015-09-28 10:36:45 -07:00
Hari Shreedharan
ae9ab0fa37 SQOOP-2582. Query import won't work for parquet 
(Jarcec via Hari)
 2015-09-24 14:40:33 -07:00
Venkat Ranganathan
fdcbaf5c53 SQOOP-2399: BigDecimalSplitter java.lang.ArrayIndexOutOfBoundsException 
(Sowmya Ramesh via Venkat Ranganathan)
 2015-09-18 12:12:07 -07:00
Jarek Jarcec Cecho
5fc7a680f8 SQOOP-1369: Avro export ignores --columns option 
(Paul Mazak via Jarek Jarcec Cecho)
 2015-09-15 07:58:53 -07:00
Jarek Jarcec Cecho
fee80ac017 SQOOP-2535: Add error handling to HiveConf 
(Dian Fu via Jarek Jarcec Cecho)
 2015-09-01 13:42:28 +02:00
Jarek Jarcec Cecho
03ed21b4f1 SQOOP-2531: readlink -f not supported on OS X 
(Stuart Williams via Jarek Jarcec Cecho)
 2015-08-28 11:19:59 +02:00
Jarek Jarcec Cecho
de6fa8279d SQOOP-2470: Incremental Hive import with append not working after validation check for --hive-import and --import 
(Aaditya Raj via Jarek Jarcec Cecho)
 2015-08-13 17:10:42 -07:00
Venkat Ranganathan
73ef0c133b SQOOP-2333: Sqoop to support Custom options for User Defined Plugins(Tool) 
(Rakesh Sharma via Venkat Ranganathan)
 2015-08-07 20:42:11 -07:00
Jarek Jarcec Cecho
c081094e5a SQOOP-2437: Use hive configuration to connect to secure metastore 
(Abraham Elmahrek via Jarek Jarcec Cecho)
 2015-07-28 18:20:48 -07:00
Jarek Jarcec Cecho
b8df3a8033 SQOOP-1094: Add Avro support to merge tool 
(Yibing Shi via Jarek Jarcec Cecho)
 2015-07-24 09:05:23 -07:00
Jarek Jarcec Cecho
d2c062b202 SQOOP-2406: Add support for secure mode when importing Parquet files into Hive 
(Abraham Elmahrek via Jarek Jarcec Cecho)
 2015-07-20 09:22:12 -07:00
Venkat Ranganathan
c6627c04cd SQOOP-2370: Netezza - need to support additional options for full control character handling 
(Venkat Ranganathan)
 2015-07-06 20:13:19 -07:00
Venkat Ranganathan
e21529ac6a SQOOP-2334: Sqoop Volume Per Mapper 
(Rakesh Sharma via Venkat Ranganathan)
 2015-06-29 20:33:13 -07:00
Qian Xu
1ffda71a2d SQOOP-2400: hive.metastore.sasl.enabled should be set to true for Oozie integration 
(Abraham Elmahrek via Qian Xu)
 2015-06-26 01:22:06 +08:00
Qian Xu
71956cad3f SQOOP-2103: Not able define Decimal(n,p) data type in map-column-hive option 
(Abraham Elmahrek via Qian Xu)
 2015-06-25 11:23:35 +08:00
Jarek Jarcec Cecho
fa8b733e48 SQOOP-2381: Add test for mysql export with --escape-by option 
(Abraham Elmahrek via Jarek Jarcec Cecho)
 2015-05-31 09:52:58 -07:00
Abraham Elmahrek
9147967eee SQOOP-2372: Import all tables as parquet will throw NPE 
(Qian Xu via Abraham Elmahrek)
 2015-05-29 18:28:15 -07:00
Abraham Elmahrek
974b886c4a SQOOP-2363: wrong option for escape character with mysqlimport 
(John Pickard via Abraham Elmahrek)
 2015-05-29 15:26:00 -07:00
Abraham Elmahrek
2e4de0469d SQOOP-2371: Tests from *LobAvroImportTest might fail 
(Jarek Jarcec Cecho via Abraham Elmahrek)
 2015-05-27 14:42:05 -07:00
Abraham Elmahrek
7f25e88e8d SQOOP-2367: Tests from PostgresqlImportTest can fail on various versions of Hadoop 
(Jarek Jarcec Cecho via Abraham Elmahrek)
 2015-05-26 22:23:10 -07:00
Abraham Elmahrek
2184161449 SQOOP-2362: Add oracle direct mode in list of supported databases 
(Richard Zhou via Abraham Elmahrek)
 2015-05-19 10:21:54 -07:00
Abraham Elmahrek
df3a81b50f SQOOP-2295: Hive import with Parquet should append automatically 
(Qian Xu via Abraham Elmahrek)
 2015-05-08 00:19:34 -07:00
Jarek Jarcec Cecho
89366b49b3 SQOOP-2343: AsyncSqlRecordWriter stucks if any exception is thrown out in its close method 
(Yibing Shi via Jarek Jarcec Cecho)
 2015-04-30 18:45:52 -07:00
Jarek Jarcec Cecho
fa4e903650 SQOOP-2339: Move sub-directory might fail in append mode 
(Qian Xu via Jarek Jarcec Cecho)
 2015-04-29 13:36:34 -07:00
Jarek Jarcec Cecho
d54dddf0b8 SQOOP-2326: Fix Netezza trunc-string option handling and unnecessary log directory during imports 
(Venkat Ranganathan via Jarek Jarcec Cecho)
 2015-04-26 09:05:13 -07:00
Jarek Jarcec Cecho
ae54a2dd91 SQOOP-2324: Remove extra license handling for consistency 
(Qian Xu via Jarek Jarcec Cecho)
 2015-04-23 21:09:51 -07:00
Jarek Jarcec Cecho
4180d50c74 SQOOP-2294: Change to Avro schema name breaks some use cases 
(Qian Xu via Jarek Jarcec Cecho)
 2015-04-21 07:02:37 -07:00
Jarek Jarcec Cecho
c109f67170 SQOOP-2296: Merge statement incorrectly includes table hints 
(Keegan Witt via Jarek Jarcec Cecho)
 2015-04-20 19:52:31 -07:00
Jarek Jarcec Cecho
7bbaf99683 SQOOP-2298: TestAvroImport test case error 
(Abraham Elmahrek via Jarek Jarcec Cecho)
 2015-04-15 15:25:11 -07:00
Gwen Shapira
292fd08b7e SQOOP-2290: java.lang.ArrayIndexOutOfBoundsException thrown when malformed column mapping is provided 
(Venkat Ranganathan via Gwen Shapira)
 2015-04-09 07:58:02 -07:00
Jarek Jarcec Cecho
c51c7daeed SQOOP-2264: Exclude and remove SqoopUserGuide.xml from git repository 
(Qian Xu via Jarek Jarcec Cecho)
 2015-04-08 20:14:19 -07:00