Package | Description |
---|---|
org.apache.manifoldcf.crawler.interfaces | |
org.apache.manifoldcf.crawler.jobs | |
org.apache.manifoldcf.crawler.system |
Modifier and Type | Field and Description |
---|---|
protected DocumentDescription[] |
DocumentSetAndFlags.documentSet |
Modifier and Type | Method and Description |
---|---|
DocumentDescription[] |
IJobManager.finishDocuments(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
java.lang.String[] parentIdentifierHashes,
int hopcountMethod)
Complete adding child documents to the queue, for a set of documents.
|
DocumentDescription |
BlockingDocuments.getBlockingDocument()
Pop a document from the set.
|
DocumentDescription[] |
DocumentSetAndFlags.getDocumentSet()
Get the document set.
|
DocumentDescription[] |
IJobManager.getNextDeletableDocuments(java.lang.String processID,
int n,
long currentTime)
Get list of deletable document descriptions.
|
DocumentDescription[] |
IJobManager.getNextDocuments(java.lang.String processID,
int n,
long currentTime,
long interval,
BlockingDocuments blockingDocuments,
PerformanceStatistics statistics,
DepthStatistics scanRecord)
Get up to the next n document(s) to be fetched and processed.
|
DocumentDescription[] |
IJobManager.getNextNotYetProcessedReprioritizationDocuments(java.lang.String processID,
int n)
Get a list of not-yet-processed documents to reprioritize.
|
DocumentDescription[] |
IJobManager.markDocumentCleanedUp(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription documentDescription,
int hopcountMethod)
Delete from queue as a result of cleaning up an unreachable document.
|
DocumentDescription[] |
IJobManager.markDocumentCleanedUpMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Delete from queue as a result of cleaning up an unreachable document.
|
DocumentDescription[] |
IJobManager.markDocumentDeleted(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription documentDescription,
int hopcountMethod)
Delete from queue as a result of processing of an active document.
|
DocumentDescription[] |
IJobManager.markDocumentDeletedMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Delete from queue as a result of processing of an active document.
|
DocumentDescription[] |
IJobManager.markDocumentExpired(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription documentDescription,
int hopcountMethod)
Delete from queue as a result of expiration of an active document.
|
DocumentDescription[] |
IJobManager.markDocumentExpiredMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Delete from queue as a result of expiration of an active document.
|
DocumentDescription[] |
IJobManager.markDocumentHopcountRemoval(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription documentDescription,
int hopcountMethod)
Mark hopcount removal from queue as a result of processing of an active document.
|
DocumentDescription[] |
IJobManager.markDocumentHopcountRemovalMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Mark hopcount removal from queue as a result of processing of an active document.
|
Modifier and Type | Method and Description |
---|---|
void |
BlockingDocuments.addBlockingDocument(DocumentDescription dd)
Add a document to the set
|
void |
IJobManager.carrydownChangeDocument(DocumentDescription documentDescription,
IPriorityCalculator docPriority)
Requeue a document because of carrydown changes.
|
void |
IJobManager.carrydownChangeDocumentMultiple(DocumentDescription[] documentDescriptions,
IPriorityCalculator[] docPriorities)
Requeue a document set because of carrydown changes.
|
void |
IJobManager.deleteIngestedDocumentIdentifiers(DocumentDescription[] identifiers)
Delete ingested document identifiers (as part of deleting the owning job).
|
DocumentDescription[] |
IJobManager.markDocumentCleanedUp(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription documentDescription,
int hopcountMethod)
Delete from queue as a result of cleaning up an unreachable document.
|
DocumentDescription[] |
IJobManager.markDocumentCleanedUpMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Delete from queue as a result of cleaning up an unreachable document.
|
void |
IJobManager.markDocumentCompleted(DocumentDescription documentDescription)
Note completion of document processing by a job thread of a document.
|
void |
IJobManager.markDocumentCompletedMultiple(DocumentDescription[] documentDescriptions)
Note completion of document processing by a job thread of a document.
|
DocumentDescription[] |
IJobManager.markDocumentDeleted(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription documentDescription,
int hopcountMethod)
Delete from queue as a result of processing of an active document.
|
DocumentDescription[] |
IJobManager.markDocumentDeletedMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Delete from queue as a result of processing of an active document.
|
DocumentDescription[] |
IJobManager.markDocumentExpired(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription documentDescription,
int hopcountMethod)
Delete from queue as a result of expiration of an active document.
|
DocumentDescription[] |
IJobManager.markDocumentExpiredMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Delete from queue as a result of expiration of an active document.
|
DocumentDescription[] |
IJobManager.markDocumentHopcountRemoval(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription documentDescription,
int hopcountMethod)
Mark hopcount removal from queue as a result of processing of an active document.
|
DocumentDescription[] |
IJobManager.markDocumentHopcountRemovalMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Mark hopcount removal from queue as a result of processing of an active document.
|
void |
IJobManager.requeueDocument(DocumentDescription documentDescription,
java.lang.Long executeTime,
int action)
Requeue a document for further processing in the future.
|
void |
IJobManager.requeueDocumentMultiple(DocumentDescription[] documentDescriptions,
java.lang.Long[] executeTimes,
int[] actions)
Requeue a document for further processing in the future.
|
void |
IJobManager.resetCleaningDocument(DocumentDescription documentDescription,
long checkTime)
Reset a cleaning document back to its former state.
|
void |
IJobManager.resetCleaningDocumentMultiple(DocumentDescription[] documentDescriptions,
long checkTime)
Reset a set of cleaning documents for further processing in the future.
|
void |
IJobManager.resetDeletingDocument(DocumentDescription documentDescription,
long checkTime)
Reset a deleting document back to its former state.
|
void |
IJobManager.resetDeletingDocumentMultiple(DocumentDescription[] documentDescriptions,
long checkTime)
Reset a set of deleting documents for further processing in the future.
|
void |
IJobManager.resetDocument(DocumentDescription documentDescription,
long executeTime,
int action,
long failTime,
int failCount)
Reset an active document back to its former state.
|
void |
IJobManager.resetDocumentMultiple(DocumentDescription[] documentDescriptions,
long executeTime,
int action,
long failTime,
int failCount)
Reset documents for further processing in the future.
|
void |
IJobManager.writeDocumentPriorities(DocumentDescription[] descriptions,
IPriorityCalculator[] priorities)
Save a set of document priorities.
|
Constructor and Description |
---|
DocumentSetAndFlags(DocumentDescription[] documentSet,
boolean[] flags)
Constructor.
|
Modifier and Type | Field and Description |
---|---|
protected java.util.List<DocumentDescription> |
JobManager.ThrottleLimit.blockingDocumentArray |
Modifier and Type | Method and Description |
---|---|
protected DocumentDescription[] |
JobManager.calculateAffectedDeleteCarrydownChildren(java.lang.Long jobID,
java.lang.String[] docIDHashes)
Helper method: Find the document descriptions that will be affected due to carrydown row deletions.
|
protected DocumentDescription[] |
JobManager.calculateAffectedRestoreCarrydownChildren(java.lang.Long jobID,
java.lang.String[] parentIDHashes)
Helper method: Calculate the unique set of affected carrydown children resulting from a "restoreRecords" operation.
|
protected DocumentDescription[] |
JobManager.doDeleteMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Delete documents with no repercussions.
|
DocumentDescription[] |
JobManager.finishDocuments(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
java.lang.String[] parentIdentifierHashes,
int hopcountMethod)
Complete adding child documents to the queue, for a set of documents.
|
DocumentDescription[] |
JobManager.getNextDeletableDocuments(java.lang.String processID,
int maxCount,
long currentTime)
Get list of deletable document descriptions.
|
DocumentDescription[] |
JobManager.getNextDocuments(java.lang.String processID,
int n,
long currentTime,
long interval,
BlockingDocuments blockingDocuments,
PerformanceStatistics statistics,
DepthStatistics scanRecord)
/** Get up to the next n document(s) to be fetched and processed.
|
DocumentDescription[] |
JobManager.getNextNotYetProcessedReprioritizationDocuments(java.lang.String processID,
int n)
Get a list of not-yet-processed documents to reprioritize.
|
DocumentDescription[] |
JobManager.markDocumentCleanedUp(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription documentDescription,
int hopcountMethod)
Delete from queue as a result of cleaning up an unreachable document.
|
DocumentDescription[] |
JobManager.markDocumentCleanedUpMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Delete from queue as a result of cleaning up an unreachable document.
|
DocumentDescription[] |
JobManager.markDocumentDeleted(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription documentDescription,
int hopcountMethod)
Delete from queue as a result of processing of an active document.
|
DocumentDescription[] |
JobManager.markDocumentDeletedMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Delete from queue as a result of processing of an active document.
|
DocumentDescription[] |
JobManager.markDocumentExpired(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription documentDescription,
int hopcountMethod)
Delete from queue as a result of expiration of an active document.
|
DocumentDescription[] |
JobManager.markDocumentExpiredMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Delete from queue as a result of expiration of an active document.
|
DocumentDescription[] |
JobManager.markDocumentHopcountRemoval(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription documentDescription,
int hopcountMethod)
Mark hopcount removal from queue as a result of processing of an active document.
|
DocumentDescription[] |
JobManager.markDocumentHopcountRemovalMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Mark hopcount removal from queue as a result of processing of an active document.
|
Modifier and Type | Method and Description |
---|---|
void |
JobManager.carrydownChangeDocument(DocumentDescription documentDescription,
IPriorityCalculator docPriority)
Requeue a document because of carrydown changes.
|
void |
JobManager.carrydownChangeDocumentMultiple(DocumentDescription[] documentDescriptions,
IPriorityCalculator[] docPriorities)
Requeue a document set because of carrydown changes.
|
void |
JobManager.deleteIngestedDocumentIdentifiers(DocumentDescription[] identifiers)
Delete ingested document identifiers (as part of deleting the owning job).
|
void |
JobQueue.deleteIngestedDocumentIdentifiers(DocumentDescription[] identifiers)
Delete ingested document identifiers (as part of deleting the owning job).
|
protected DocumentDescription[] |
JobManager.doDeleteMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Delete documents with no repercussions.
|
protected java.lang.String[] |
JobManager.getUnindexableDocumentIdentifiers(DocumentDescription[] documentIdentifiers,
java.lang.String connectionName)
Get a list of document identifiers that should actually be deleted from the index, from a list that
might contain identifiers that are shared with other jobs, which are targeted to the same output connection.
|
DocumentDescription[] |
JobManager.markDocumentCleanedUp(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription documentDescription,
int hopcountMethod)
Delete from queue as a result of cleaning up an unreachable document.
|
DocumentDescription[] |
JobManager.markDocumentCleanedUpMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Delete from queue as a result of cleaning up an unreachable document.
|
void |
JobManager.markDocumentCompleted(DocumentDescription documentDescription)
Note completion of document processing by a job thread of a document.
|
void |
JobManager.markDocumentCompletedMultiple(DocumentDescription[] documentDescriptions)
Note completion of document processing by a job thread of a document.
|
DocumentDescription[] |
JobManager.markDocumentDeleted(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription documentDescription,
int hopcountMethod)
Delete from queue as a result of processing of an active document.
|
DocumentDescription[] |
JobManager.markDocumentDeletedMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Delete from queue as a result of processing of an active document.
|
DocumentDescription[] |
JobManager.markDocumentExpired(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription documentDescription,
int hopcountMethod)
Delete from queue as a result of expiration of an active document.
|
DocumentDescription[] |
JobManager.markDocumentExpiredMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Delete from queue as a result of expiration of an active document.
|
DocumentDescription[] |
JobManager.markDocumentHopcountRemoval(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription documentDescription,
int hopcountMethod)
Mark hopcount removal from queue as a result of processing of an active document.
|
DocumentDescription[] |
JobManager.markDocumentHopcountRemovalMultiple(java.lang.Long jobID,
java.lang.String[] legalLinkTypes,
DocumentDescription[] documentDescriptions,
int hopcountMethod)
Mark hopcount removal from queue as a result of processing of an active document.
|
void |
JobManager.requeueDocument(DocumentDescription documentDescription,
java.lang.Long executeTime,
int action)
Requeue a document for further processing in the future.
|
void |
JobManager.requeueDocumentMultiple(DocumentDescription[] documentDescriptions,
java.lang.Long[] executeTimes,
int[] actions)
Requeue a document for further processing in the future.
|
void |
JobManager.resetCleaningDocument(DocumentDescription documentDescription,
long checkTime)
Reset a cleaning document back to its former state.
|
void |
JobManager.resetCleaningDocumentMultiple(DocumentDescription[] documentDescriptions,
long checkTime)
Reset a set of cleaning documents for further processing in the future.
|
void |
JobManager.resetDeletingDocument(DocumentDescription documentDescription,
long checkTime)
Reset a deleting document back to its former state.
|
void |
JobManager.resetDeletingDocumentMultiple(DocumentDescription[] documentDescriptions,
long checkTime)
Reset a set of deleting documents for further processing in the future.
|
void |
JobManager.resetDocument(DocumentDescription documentDescription,
long executeTime,
int action,
long failTime,
int failCount)
Reset an active document back to its former state.
|
void |
JobManager.resetDocumentMultiple(DocumentDescription[] documentDescriptions,
long executeTime,
int action,
long failTime,
int failCount)
Reset a set of documents for further processing in the future.
|
void |
JobManager.writeDocumentPriorities(DocumentDescription[] documentDescriptions,
IPriorityCalculator[] priorities)
Save a set of document priorities.
|
Modifier and Type | Method and Description |
---|---|
protected void |
JobManager.fetchAndProcessDocuments(java.util.List<DocumentDescription> answers,
java.lang.Long currentTimeValue,
java.lang.Long currentPriorityValue,
JobManager.ThrottleLimit vList,
IRepositoryConnection[] connections,
java.lang.String processID)
Fetch and process documents matching the passed-in criteria
|
Modifier and Type | Field and Description |
---|---|
protected DocumentDescription |
DeleteQueuedDocument.documentDescription
The document description.
|
protected DocumentDescription |
QueuedDocument.documentDescription
The document description.
|
Modifier and Type | Method and Description |
---|---|
DocumentDescription |
DeleteQueuedDocument.getDocumentDescription()
Get the document description.
|
DocumentDescription |
QueuedDocument.getDocumentDescription()
Get the document description.
|
Modifier and Type | Method and Description |
---|---|
protected static java.lang.String |
WorkerThread.makeListString(DocumentDescription[] sourceList) |
static void |
ManifoldCF.requeueDocumentsDueToCarrydown(IJobManager jobManager,
DocumentDescription[] requeueCandidates,
IRepositoryConnector connector,
IRepositoryConnection connection,
IReprioritizationTracker rt,
long currentTime)
Requeue documents due to carrydown.
|
static void |
ManifoldCF.writeDocumentPriorities(IThreadContext threadContext,
DocumentDescription[] descs,
java.util.Map<java.lang.String,IRepositoryConnection> connectionMap,
java.util.Map<java.lang.Long,IJobDescription> jobDescriptionMap)
Write a set of document priorities, based on the current queue tracker.
|
Constructor and Description |
---|
CleanupQueuedDocument(DocumentDescription documentDescription,
boolean deleteFromIndex)
Constructor.
|
DeleteQueuedDocument(DocumentDescription documentDescription)
Constructor.
|
QueuedDocument(DocumentDescription documentDescription,
java.util.Map<java.lang.String,DocumentIngestStatusSet> lastIngestedStatus,
java.lang.String[] binNames)
Constructor.
|