org.apache.manifoldcf.crawler.jobs

Class JobDescription

java.lang.Object

org.apache.manifoldcf.crawler.jobs.JobDescription

All Implemented Interfaces:

IJobDescription

public class JobDescription
extends java.lang.Object
implements IJobDescription

This is a paper object describing a job. Each job in the lcf framework has: - an identifier; - a description; - a repository connection; - one of a number of scheduling options: starting every n hours/days/weeks/months, on specific dates, or "continuous" (which basically establishes a priority queue based on modification frequency); - "seeds" (or starting points), which are the places that scanning begins. Also remember that since incremental deletion must occur on a job-by-job basis, the scanning data also records the job that performed the scan, so that each job can rescan previous ingested data, and delete documents that have been removed.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	JobDescription.Notification
protected static class	JobDescription.PipelineStage

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	_rcsid
protected java.lang.String	connectionName
protected java.lang.String	description
protected Specification	documentSpecification
protected java.lang.Long	expiration
protected java.util.HashMap	hopCountFilters
protected int	hopcountMode
protected java.lang.Long	id
protected java.lang.Long	interval
protected boolean	isNew
protected java.lang.Long	maxInterval
protected java.util.List<JobDescription.Notification>	notifications
protected java.util.List<JobDescription.PipelineStage>	pipelineStages
protected int	priority
protected java.lang.Float	rate
protected boolean	readOnly
protected java.lang.Long	reseedInterval
protected ScheduleList	scheduleList
protected int	startMethod
protected int	type

Fields inherited from interface org.apache.manifoldcf.crawler.interfaces.IJobDescription

HOPCOUNT_ACCURATE, HOPCOUNT_NEVERDELETE, HOPCOUNT_NODELETE, START_DISABLE, START_WINDOWBEGIN, START_WINDOWINSIDE, TYPE_CONTINUOUS, TYPE_SPECIFIED

Constructor Summary

Constructors

Constructor and Description
JobDescription()

Method Summary

Modifier and Type	Method and Description
void	addHopCountFilter(java.lang.String linkType, java.lang.Long maxHops) Add a hopcount filter to the job.
Specification	addNotification(java.lang.String notificationConnectionName, java.lang.String notificationDescription) Add a notification.
Specification	addPipelineStage(int prerequisiteStage, boolean isOutput, java.lang.String pipelineStageConnectionName, java.lang.String pipelineStageDescription) Add a pipeline connection.
void	addScheduleRecord(ScheduleRecord record) Add a record.
void	clearHopCountFilters() Clear the set of hopcount filters for the job.
void	clearNotifications() Clear notification connections.
void	clearPipeline() Clear pipeline connections
void	clearScheduleRecords() Clear all the scheduling records.
int	countNotifications() Get a count of pipeline connections.
int	countPipelineStages() Get a count of pipeline stages
void	deleteNotification(int index) Delete a notification.
void	deletePipelineStage(int index) Delete a pipeline stage.
void	deleteScheduleRecord(int index) Delete a specified schedule record.
JobDescription	duplicate(boolean readOnly) Duplicate method, with optional "readonly" flag.
java.lang.String	getConnectionName() Get the connection name.
java.lang.String	getDescription() Get the description.
java.lang.Long	getExpiration() Get the expiration time, in milliseconds.
java.util.Map	getHopCountFilters() Get the set of hopcount filters the job has defined.
int	getHopcountMode() Get the hopcount mode.
java.lang.Long	getID() Get the id.
java.lang.Long	getInterval() Get the rescheduling interval, in milliseconds.
boolean	getIsNew() Get isnew.
java.lang.Long	getMaxInterval() Get the maximum rescheduling interval, in milliseconds.
java.lang.String	getNotificationConnectionName(int index) Get a specific notification connection name.
java.lang.String	getNotificationDescription(int index) Get a specific notification description.
Specification	getNotificationSpecification(int index) Get a specific notification specification.
java.lang.String	getPipelineStageConnectionName(int index) Get a specific pipeline connection name.
java.lang.String	getPipelineStageDescription(int index) Get a specific pipeline stage description.
boolean	getPipelineStageIsOutputConnection(int index) Check if a pipeline stage is an output connection.
int	getPipelineStagePrerequisite(int index) Get the prerequisite stage number for a pipeline stage.
Specification	getPipelineStageSpecification(int index) Get a specific pipeline stage specification.
int	getPriority() Get the job priority.
java.lang.Long	getReseedInterval() Get the reseeding interval, in milliseconds.
ScheduleRecord	getScheduleRecord(int index) Get a specified schedule record.
int	getScheduleRecordCount() Get the number of schedule records.
Specification	getSpecification() Get the document specification.
int	getStartMethod() Get the job's start method.
int	getType() Get the job type.
Specification	insertNotification(int index, java.lang.String notificationConnectionName, java.lang.String notificationDescription) Insert a new notification.
Specification	insertPipelineStage(int index, boolean isOutput, java.lang.String pipelineStageConnectionName, java.lang.String pipelineStageDescription) Insert a new pipeline stage.
void	makeReadOnly() Make the description "read only".
void	setConnectionName(java.lang.String connectionName) Set the connection name.
void	setDescription(java.lang.String description) Set the description.
void	setExpiration(java.lang.Long time) Set the expiration time, in milliseconds.
void	setHopcountMode(int mode) Set the hopcount mode.
void	setID(java.lang.Long id) Set the id.
void	setInterval(java.lang.Long interval) Set the rescheduling interval, in milliseconds.
void	setIsNew(boolean isNew) Set isnew.
void	setMaxInterval(java.lang.Long interval) Set the maximum rescheduling interval, in milliseconds, or null if forever.
void	setPriority(int priority) Set the job priority.
void	setReseedInterval(java.lang.Long interval) Set the reseeding interval, in milliseconds.
void	setStartMethod(int startMethod) Set the job's start method.
void	setType(int type) Set the job type.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

_rcsid

public static final java.lang.String _rcsid

See Also:

Constant Field Values

isNew

protected boolean isNew

id

protected java.lang.Long id

description

protected java.lang.String description

connectionName

protected java.lang.String connectionName

pipelineStages

protected final java.util.List<JobDescription.PipelineStage> pipelineStages

notifications

protected final java.util.List<JobDescription.Notification> notifications

type

protected int type

startMethod

protected int startMethod

priority

protected int priority

scheduleList

protected ScheduleList scheduleList

rate

protected java.lang.Float rate

interval

protected java.lang.Long interval

maxInterval

protected java.lang.Long maxInterval

expiration

protected java.lang.Long expiration

reseedInterval

protected java.lang.Long reseedInterval

documentSpecification

protected Specification documentSpecification

hopCountFilters

protected java.util.HashMap hopCountFilters

hopcountMode

protected int hopcountMode

readOnly

protected boolean readOnly

Constructor Detail

JobDescription

public JobDescription()

Method Detail

duplicate

public JobDescription duplicate(boolean readOnly)

Duplicate method, with optional "readonly" flag.

makeReadOnly

public void makeReadOnly()

Make the description "read only". This must be done after the object has been complete specified. Once a document is read-only, it cannot be made writable without duplication.

setIsNew

public void setIsNew(boolean isNew)

Set isnew.

Parameters:

isNew - is true if the object is new.

getIsNew

public boolean getIsNew()

Get isnew.

Specified by:

getIsNew in interface IJobDescription

Returns:

true if the object is new.

setID

public void setID(java.lang.Long id)

Set the id.

Parameters:

id - is the id.

getID

public java.lang.Long getID()

Get the id.

Specified by:

getID in interface IJobDescription

Returns:

the id.

setDescription

public void setDescription(java.lang.String description)

Set the description.

Specified by:

setDescription in interface IJobDescription

Parameters:

description - is the description.

getDescription

public java.lang.String getDescription()

Get the description.

Specified by:

getDescription in interface IJobDescription

Returns:

the description

setConnectionName

public void setConnectionName(java.lang.String connectionName)

Set the connection name.

Specified by:

setConnectionName in interface IJobDescription

Parameters:

connectionName - is the connection name.

getConnectionName

public java.lang.String getConnectionName()

Get the connection name.

Specified by:

getConnectionName in interface IJobDescription

Returns:

the connection name.

clearPipeline

public void clearPipeline()

Clear pipeline connections

Specified by:

clearPipeline in interface IJobDescription

addPipelineStage

public Specification addPipelineStage(int prerequisiteStage,
                                      boolean isOutput,
                                      java.lang.String pipelineStageConnectionName,
                                      java.lang.String pipelineStageDescription)

Add a pipeline connection.

Specified by:

addPipelineStage in interface IJobDescription

Parameters:

prerequisiteStage - is the prerequisite stage number for this connection, or -1 if there is none.

isOutput - is true if the pipeline stage is an output connection.

pipelineStageConnectionName - is the name of the pipeline connection to add.

pipelineStageDescription - is a description of the pipeline stage being added.

Returns:

the empty output specification for this pipeline stage.

countPipelineStages

public int countPipelineStages()

Get a count of pipeline stages

Specified by:

countPipelineStages in interface IJobDescription

Returns:

the current number of pipeline connections.

insertPipelineStage

public Specification insertPipelineStage(int index,
                                         boolean isOutput,
                                         java.lang.String pipelineStageConnectionName,
                                         java.lang.String pipelineStageDescription)

Insert a new pipeline stage.

Specified by:

insertPipelineStage in interface IJobDescription

Parameters:

index - is the index to insert pipeline stage before

pipelineStageConnectionName - is the connection name.

pipelineStageDescription - is the description.

Returns:

the newly-created output specification.

getPipelineStagePrerequisite

public int getPipelineStagePrerequisite(int index)

Get the prerequisite stage number for a pipeline stage.

Specified by:

getPipelineStagePrerequisite in interface IJobDescription

Parameters:

index - is the index of the pipeline stage to get.

Returns:

the preceding stage number for that stage, or -1 if there is none.

getPipelineStageIsOutputConnection

public boolean getPipelineStageIsOutputConnection(int index)

Check if a pipeline stage is an output connection.

Specified by:

getPipelineStageIsOutputConnection in interface IJobDescription

Parameters:

index - is the index of the pipeline stage to check.

Returns:

true if it is an output connection.

getPipelineStageConnectionName

public java.lang.String getPipelineStageConnectionName(int index)

Get a specific pipeline connection name.

Specified by:

getPipelineStageConnectionName in interface IJobDescription

Parameters:

index - is the index of the pipeline stage whose connection name to get.

Returns:

the name of the connection.

getPipelineStageDescription

public java.lang.String getPipelineStageDescription(int index)

Get a specific pipeline stage description.

Specified by:

getPipelineStageDescription in interface IJobDescription

Parameters:

index - is the index of the pipeline stage whose description to get.

Returns:

the name of the connection.

getPipelineStageSpecification

public Specification getPipelineStageSpecification(int index)

Get a specific pipeline stage specification.

Specified by:

getPipelineStageSpecification in interface IJobDescription

Parameters:

index - is the index of the pipeline stage whose specification is needed.

Returns:

the specification for the connection.

deletePipelineStage

public void deletePipelineStage(int index)

Delete a pipeline stage.

Specified by:

deletePipelineStage in interface IJobDescription

Parameters:

index - is the index of the pipeline stage to delete.

clearNotifications

public void clearNotifications()

Clear notification connections.

Specified by:

clearNotifications in interface IJobDescription

addNotification

public Specification addNotification(java.lang.String notificationConnectionName,
                                     java.lang.String notificationDescription)

Add a notification.

Specified by:

addNotification in interface IJobDescription

Parameters:

notificationConnectionName - is the name of the notification connection to add.

notificationDescription - is a description of the notification being added.

Returns:

the empty specification for this notification.

countNotifications

public int countNotifications()

Get a count of pipeline connections.

Specified by:

countNotifications in interface IJobDescription

Returns:

the current number of pipeline connections.

getNotificationConnectionName

public java.lang.String getNotificationConnectionName(int index)

Get a specific notification connection name.

Specified by:

getNotificationConnectionName in interface IJobDescription

Parameters:

index - is the index of the notification whose connection name to get.

Returns:

the name of the connection.

getNotificationDescription

public java.lang.String getNotificationDescription(int index)

Get a specific notification description.

Specified by:

getNotificationDescription in interface IJobDescription

Parameters:

index - is the index of the notification whose description to get.

Returns:

the name of the connection.

getNotificationSpecification

public Specification getNotificationSpecification(int index)

Get a specific notification specification.

Specified by:

getNotificationSpecification in interface IJobDescription

Parameters:

index - is the index of the notification whose specification is needed.

Returns:

the specification for the connection.

deleteNotification

public void deleteNotification(int index)

Delete a notification.

Specified by:

deleteNotification in interface IJobDescription

Parameters:

index - is the index of the notification to delete.

insertNotification

public Specification insertNotification(int index,
                                        java.lang.String notificationConnectionName,
                                        java.lang.String notificationDescription)

Insert a new notification.

Specified by:

insertNotification in interface IJobDescription

Parameters:

index - is the index to insert pipeline stage before

notificationConnectionName - is the connection name.

notificationDescription - is the description.

Returns:

the newly-created output specification.

setType

public void setType(int type)

Set the job type.

Specified by:

setType in interface IJobDescription

Parameters:

type - is the type (as an integer).

getType

public int getType()

Get the job type.

Specified by:

getType in interface IJobDescription

Returns:

the type (as an integer).

setStartMethod

public void setStartMethod(int startMethod)

Set the job's start method.

Specified by:

setStartMethod in interface IJobDescription

Parameters:

startMethod - is the start description.

getStartMethod

public int getStartMethod()

Get the job's start method.

Specified by:

getStartMethod in interface IJobDescription

Returns:

the start method.

clearScheduleRecords

public void clearScheduleRecords()

Clear all the scheduling records.

Specified by:

clearScheduleRecords in interface IJobDescription

addScheduleRecord

public void addScheduleRecord(ScheduleRecord record)

Add a record.

Specified by:

addScheduleRecord in interface IJobDescription

Parameters:

record - is the record to add.

getScheduleRecordCount

public int getScheduleRecordCount()

Get the number of schedule records.

Specified by:

getScheduleRecordCount in interface IJobDescription

Returns:

the count.

getScheduleRecord

public ScheduleRecord getScheduleRecord(int index)

Get a specified schedule record.

Specified by:

getScheduleRecord in interface IJobDescription

Parameters:

index - is the record number.

Returns:

the record.

deleteScheduleRecord

public void deleteScheduleRecord(int index)

Delete a specified schedule record.

Specified by:

deleteScheduleRecord in interface IJobDescription

Parameters:

index - is the record number.

setInterval

public void setInterval(java.lang.Long interval)

Set the rescheduling interval, in milliseconds.

Specified by:

setInterval in interface IJobDescription

Parameters:

interval - is the default interval, or null for infinite.

getInterval

public java.lang.Long getInterval()

Get the rescheduling interval, in milliseconds.

Specified by:

getInterval in interface IJobDescription

Returns:

the default interval, or null for infinite.

setMaxInterval

public void setMaxInterval(java.lang.Long interval)

Set the maximum rescheduling interval, in milliseconds, or null if forever.

Specified by:

setMaxInterval in interface IJobDescription

Parameters:

interval - is the maximum interval.

getMaxInterval

public java.lang.Long getMaxInterval()

Get the maximum rescheduling interval, in milliseconds.

Specified by:

getMaxInterval in interface IJobDescription

Returns:

the max interval, or null if forever.

setExpiration

public void setExpiration(java.lang.Long time)

Set the expiration time, in milliseconds.

Specified by:

setExpiration in interface IJobDescription

Parameters:

time - is the maximum expiration time of a document, in milliseconds, or null if none.

getExpiration

public java.lang.Long getExpiration()

Get the expiration time, in milliseconds.

Specified by:

getExpiration in interface IJobDescription

Returns:

the maximum expiration time of a document, or null if none.

setReseedInterval

public void setReseedInterval(java.lang.Long interval)

Set the reseeding interval, in milliseconds.

Specified by:

setReseedInterval in interface IJobDescription

Parameters:

interval - is the interval, or null for infinite.

getReseedInterval

public java.lang.Long getReseedInterval()

Get the reseeding interval, in milliseconds.

Specified by:

getReseedInterval in interface IJobDescription

Returns:

the interval, or null if infinite.

getSpecification

public Specification getSpecification()

Get the document specification.

Specified by:

getSpecification in interface IJobDescription

Returns:

the document specification object.

setPriority

public void setPriority(int priority)

Set the job priority. This is a simple integer between 1 and 10, where 1 is the highest priority.

Specified by:

setPriority in interface IJobDescription

Parameters:

priority - is the priority.

getPriority

public int getPriority()

Get the job priority.

Specified by:

getPriority in interface IJobDescription

Returns:

the priority (a number between 1 and 10).

getHopCountFilters

public java.util.Map getHopCountFilters()

Get the set of hopcount filters the job has defined.

Specified by:

getHopCountFilters in interface IJobDescription

Returns:

the set as a map, keyed by Strings and containing Longs.

clearHopCountFilters

public void clearHopCountFilters()

Clear the set of hopcount filters for the job.

Specified by:

clearHopCountFilters in interface IJobDescription

addHopCountFilter

public void addHopCountFilter(java.lang.String linkType,
                              java.lang.Long maxHops)

Add a hopcount filter to the job.

Specified by:

addHopCountFilter in interface IJobDescription

Parameters:

linkType - is the type of link the filter applies to.

maxHops - is the maximum hop count. Use null to remove a filter.

getHopcountMode

public int getHopcountMode()

Get the hopcount mode.

Specified by:

getHopcountMode in interface IJobDescription

setHopcountMode

public void setHopcountMode(int mode)

Set the hopcount mode.

Specified by:

setHopcountMode in interface IJobDescription