edu.yale.cs.hadoopdb.connector
Class AbstractDBRecordReader

java.lang.Object
  extended by edu.yale.cs.hadoopdb.connector.AbstractDBRecordReader
Direct Known Subclasses:
DBRecordReader, SMSRecordReader

public abstract class AbstractDBRecordReader
extends java.lang.Object

Base DBRecordReader class. Several extensions of this class are possible to allow different constructors and SQL query specification through abstract method getSqlQuery(). Class implements all methods needed by a Hadoop's RecordReader interface except for next().


Field Summary
protected  java.sql.Connection connection
           
protected  long connTime
           
static org.apache.commons.logging.Log LOG
           
static int MAX_CONNECTION_TRIALS
          Maximum number of connection trials
protected  long pos
           
protected  long queryTime
           
protected  java.sql.ResultSet results
           
protected  long startTime
           
protected  java.sql.Statement statement
           
 
Constructor Summary
AbstractDBRecordReader()
           
 
Method Summary
 void close()
          After query execution is complete, the database connection is closed cleanly.
 org.apache.hadoop.io.LongWritable createKey()
           
protected  java.sql.Connection getConnection(DBInputSplit dbSplit)
          Connects to a database of a particular chunk (specified within the split).
private static java.lang.String getLocatHostAddres()
          Helper method to retrieve local host name or null if not possible
 long getPos()
          Returns the number of rows retrieved so far.
 float getProgress()
          Returns a float [0,1] indicating progress (currently, progress is always 0 as there is no easy way for progress estimation).
protected abstract  java.lang.String getSqlQuery()
          Abstract method definition.
protected  void setupDB(DBInputSplit split, org.apache.hadoop.mapred.JobConf conf)
          Method sets up a connection to a database and provides query optimization parameters.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

connection

protected java.sql.Connection connection

connTime

protected long connTime

LOG

public static final org.apache.commons.logging.Log LOG

MAX_CONNECTION_TRIALS

public static final int MAX_CONNECTION_TRIALS
Maximum number of connection trials

See Also:
Constant Field Values

pos

protected long pos

queryTime

protected long queryTime

results

protected java.sql.ResultSet results

startTime

protected long startTime

statement

protected java.sql.Statement statement
Constructor Detail

AbstractDBRecordReader

public AbstractDBRecordReader()
Method Detail

close

public void close()
           throws java.io.IOException
After query execution is complete, the database connection is closed cleanly.

Throws:
java.io.IOException

createKey

public org.apache.hadoop.io.LongWritable createKey()

getConnection

protected java.sql.Connection getConnection(DBInputSplit dbSplit)
Connects to a database of a particular chunk (specified within the split). If a particular host fails during connection, it is avoided and another host is found. The method fails after a set number of maximum connection trials.


getLocatHostAddres

private static java.lang.String getLocatHostAddres()
Helper method to retrieve local host name or null if not possible


getPos

public long getPos()
            throws java.io.IOException
Returns the number of rows retrieved so far. This value is updated by record reader sub-classes.

Throws:
java.io.IOException

getProgress

public float getProgress()
                  throws java.io.IOException
Returns a float [0,1] indicating progress (currently, progress is always 0 as there is no easy way for progress estimation).

Throws:
java.io.IOException

getSqlQuery

protected abstract java.lang.String getSqlQuery()
Abstract method definition. All extensions need to provide a sql query necessary for retrieving rows from the database.

Returns:
String standard SQL query

setupDB

protected void setupDB(DBInputSplit split,
                       org.apache.hadoop.mapred.JobConf conf)
                throws java.sql.SQLException
Method sets up a connection to a database and provides query optimization parameters. Then it executes the query.

Throws:
java.sql.SQLException