github/sqoop
5
0
Fork 0
mirror of https://github.com/apache/sqoop.git synced 2025-05-02 17:22:25 +08:00
Code Issues Projects Releases Wiki Activity
95 Commits 21 Branches 30 Tags 24 MiB
Java 97.7%
Shell 0.9%
XSLT 0.4%
Python 0.3%
Batchfile 0.3%
Other 0.3%
1eb4226230
Go to file
Andrew Bayer 1eb4226230 SIP-3. File format for large object (LOB) storage. 
Introduce LobFile format for storing large objects.
Implemented LobFile.Reader, LobFile.Writer classes.
Added a performance test of LobFile reading/writing speed.
Build system: fix cobertura build deps.
Remove unused utility classes from o.a.h.s.io.
Use LobFile for external storage in {B,C}lobRef.
Added LobReaderCache.
Converted BlobRef to read from LobFiles (through LobReaderCache).
LargeObjectLoader writes to LobFiles.
Common code from BlobRef and ClobRef factored out into LobRef abstract
base class.
Updated Test{B,C}lobRef and TestLargeObjectLoader for new external LOB storage.
Updated *ImportMappers to close LargeObjectLoaders when they're done.
Added performance tests to build.
Added script to run perf tests; factored out common logic into config script.
Fixed ivy dependency resolution to use multiple configuration inheritance.
Added LobFileStressTest.
Added readme with instructions to src/perftest directory.
Added CodecMap that abstracts compression codec classes to names.

From: Aaron Kimball <aaron@cloudera.com>

git-svn-id: https://svn.apache.org/repos/asf/incubator/sqoop/trunk@1149897 13f79535-47bb-0310-9956-ffa450edef68
2011-07-22 20:03:46 +00:00
bin SIP-3. File format for large object (LOB) storage. 2011-07-22 20:03:46 +00:00
conf Changed license headers to reference Cloudera instead of the ASF. 2011-07-22 20:03:36 +00:00
ivy SIP-1. Refactor operations into distinct tools. 2011-07-22 20:03:45 +00:00
lib Add shim classes to allow compilation against different Hadoop distributions 2011-07-22 20:03:41 +00:00
src SIP-3. File format for large object (LOB) storage. 2011-07-22 20:03:46 +00:00
testdata/hive Changed license headers to reference Cloudera instead of the ASF. 2011-07-22 20:03:36 +00:00
.gitignore SIP-1. Refactor operations into distinct tools. 2011-07-22 20:03:45 +00:00
build.xml SIP-3. File format for large object (LOB) storage. 2011-07-22 20:03:46 +00:00
COMPILING.txt Add ability to compile against Cloudera or Apache Hadoop. 2011-07-22 20:03:41 +00:00
ivy.xml SIP-3. File format for large object (LOB) storage. 2011-07-22 20:03:46 +00:00
LICENSE.txt Changed license headers to reference Cloudera instead of the ASF. 2011-07-22 20:03:36 +00:00
NOTICE.txt Changed license headers to reference Cloudera instead of the ASF. 2011-07-22 20:03:36 +00:00
README.txt Add ability to compile against Cloudera or Apache Hadoop. 2011-07-22 20:03:41 +00:00
svn-placeholder Placeholder commit 2011-07-22 19:59:57 +00:00

README.txt 

= Welcome to Sqoop!

This is the Sqoop (SQL-to-Hadoop) tool. Sqoop allows easy imports and
exports of data sets between databases and HDFS.


== More Documentation

Sqoop ships with additional documentation: a user guide and a manual page.

Asciidoc sources for both of these are in +src/docs/+. Run +ant docs+ to build
the documentation. It will be created in +build/docs/+.

If you got Sqoop in release form, documentation will already be built and
available in the +docs/+ directory.


== Compiling Sqoop

Compiling Sqoop requires the following tools:

* Apache ant (1.7.1)
* Java JDK 1.6

Additionally, building the documentation requires these tools:

* asciidoc
* make
* python 2.5+
* xmlto
* tar
* gzip

To compile Sqoop, run +ant package+. There will be a fully self-hosted build
provided in the +build/sqoop-(version)/+ directory. 

You can build just the jar by running +ant jar+.

See the COMPILING.txt document for for information.

== This is also an Asciidoc file!

* Try running +asciidoc README.txt+
* For more information about asciidoc, see http://www.methods.co.nz/asciidoc/