org.apache.manifoldcf.crawler.connectors.webcrawler

Class DNSManager

java.lang.Object

org.apache.manifoldcf.core.database.BaseTable

org.apache.manifoldcf.crawler.connectors.webcrawler.DNSManager

public class DNSManager
extends org.apache.manifoldcf.core.database.BaseTable

This class manages the database table into which we DNS entries for hosts. The data resides in the database, as well as in cache (up to a certain point). The result is that there is a memory limited, database-backed repository of DNS entries that we can draw on. Note that this code is also responsible for efficiently caching the mapping of IP address to a canonical host name.

dnsdata

Field	Type	Description
hostname	VARCHAR(255)	Primary Key
canonicalhostname	VARCHAR(255)
ipaddress	VARCHAR(16)
expirationtime	BIGINT

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	DNSManager.DNSCacheClass Cache class for robots.
protected static class	DNSManager.DNSInfo This is a cached data item.
protected static class	DNSManager.HostDescription This is the object description for a robots host object.
protected static class	DNSManager.HostExecutor This is the executor object for locating robots host objects.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	_rcsid
protected static DNSManager.DNSCacheClass	dnsCacheClass
protected static java.lang.String	expirationField
protected static java.lang.String	fqdnField
protected static java.lang.String	hostField
protected static java.lang.String	ipaddressField

Fields inherited from class org.apache.manifoldcf.core.database.BaseTable

dbInterface, tableName

Constructor Summary

Constructors

Constructor and Description
DNSManager(org.apache.manifoldcf.core.interfaces.IThreadContext tc, org.apache.manifoldcf.core.interfaces.IDBInterface database) Constructor.

Method Summary

Modifier and Type	Method and Description
void	deinstall() Uninstall the manager.
protected static java.lang.String	getDNSKey(java.lang.String hostName) Construct a key which represents an individual host name.
void	install() Install the manager.
DNSManager.DNSInfo	lookup(java.lang.String hostName, long currentTime) Given a host name, look up the ip address and fqdn.
protected DNSManager.DNSInfo	readDNSInfo(java.lang.String hostName) Read DNS data, if it exists.
void	writeDNSData(java.lang.String hostName, java.lang.String fqdn, java.lang.String ipaddress, long expirationTime) Write DNS data, replacing any existing row.

Methods inherited from class org.apache.manifoldcf.core.database.BaseTable

addTableIndex, analyzeTable, beginTransaction, buildConjunctionClause, constructCountClause, constructDistinctOnClause, constructDoubleCastClause, constructOffsetLimitClause, constructRegexpClause, constructSubstringClause, endTransaction, findConjunctionClauseMax, getDatabaseCacheKey, getDBInterface, getMaxInClause, getMaxOrClause, getSleepAmt, getTableIndexes, getTableName, getTableSchema, getTransactionID, getWindowedReportMaxRows, makeTableKey, noteModifications, performAddIndex, performAlter, performCommit, performCreate, performDelete, performDrop, performInsert, performModification, performQuery, performQuery, performRemoveIndex, performUpdate, prepareRowForSave, readRow, reindexTable, signalRollback, sleepFor

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

_rcsid

public static final java.lang.String _rcsid

See Also:

Constant Field Values

dnsCacheClass

protected static DNSManager.DNSCacheClass dnsCacheClass

hostField

protected static final java.lang.String hostField

See Also:

Constant Field Values

fqdnField

protected static final java.lang.String fqdnField

See Also:

Constant Field Values

ipaddressField

protected static final java.lang.String ipaddressField

See Also:

Constant Field Values

expirationField

protected static final java.lang.String expirationField

See Also:

Constant Field Values

Constructor Detail

DNSManager

public DNSManager(org.apache.manifoldcf.core.interfaces.IThreadContext tc,
                  org.apache.manifoldcf.core.interfaces.IDBInterface database)
           throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Constructor. Note that one robotsmanager handle is only useful within a specific thread context, so the calling connector object logic must recreate the handle whenever the thread context changes.

Parameters:

tc - is the thread context.

database - is the database handle.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

Method Detail

install

public void install()
             throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Install the manager.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

deinstall

public void deinstall()
               throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Uninstall the manager.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

lookup

public DNSManager.DNSInfo lookup(java.lang.String hostName,
                                 long currentTime)
                          throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Given a host name, look up the ip address and fqdn.

Returns:

null if there is no available cached version of this info.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

writeDNSData

public void writeDNSData(java.lang.String hostName,
                         java.lang.String fqdn,
                         java.lang.String ipaddress,
                         long expirationTime)
                  throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Write DNS data, replacing any existing row.

Parameters:

hostName - is the host.

fqdn - is the canonical host name.

ipaddress - is the host ip address, in standard form.

expirationTime - is the time this data should expire.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

getDNSKey

protected static java.lang.String getDNSKey(java.lang.String hostName)

Construct a key which represents an individual host name.

Parameters:

hostName - is the name of the connector.

Returns:

the cache key.

readDNSInfo

protected DNSManager.DNSInfo readDNSInfo(java.lang.String hostName)
                                  throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Read DNS data, if it exists.

Returns:

null if the data doesn't exist at all. Return DNS data if it does.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException