Package | Description |
---|---|
org.apache.manifoldcf.agents.incrementalingest | |
org.apache.manifoldcf.agents.interfaces | |
org.apache.manifoldcf.crawler.system |
Modifier and Type | Method and Description |
---|---|
void |
IncrementalIngester.documentCheck(IPipelineSpecificationBasic pipelineSpecificationBasic,
java.lang.String identifierClass,
java.lang.String identifierHash,
long checkTime)
Note the fact that we checked a document (and found that it did not need to be ingested, because the
versions agreed).
|
void |
IncrementalIngester.documentCheckMultiple(IPipelineSpecificationBasic pipelineSpecificationBasic,
java.lang.String[] identifierClasses,
java.lang.String[] identifierHashes,
long checkTime)
Note the fact that we checked a document (and found that it did not need to be ingested, because the
versions agreed).
|
void |
IncrementalIngester.documentRecord(IPipelineSpecificationBasic pipelineSpecificationBasic,
java.lang.String identifierClass,
java.lang.String identifierHash,
java.lang.String componentHash,
java.lang.String documentVersion,
long recordTime)
Record a document version, but don't ingest it.
|
protected static java.lang.String[] |
IncrementalIngester.extractOutputConnectionNames(IPipelineSpecificationBasic pipelineSpecificationBasic) |
long |
IncrementalIngester.getDocumentUpdateInterval(IPipelineSpecificationBasic pipelineSpecificationBasic,
java.lang.String identifierClass,
java.lang.String identifierHash)
Calculate the average time interval between changes for a document.
|
long[] |
IncrementalIngester.getDocumentUpdateIntervalMultiple(IPipelineSpecificationBasic pipelineSpecificationBasic,
java.lang.String[] identifierClasses,
java.lang.String[] identifierHashes)
Calculate the average time interval between changes for a document.
|
java.lang.String |
IncrementalIngester.getFirstIndexedOutputConnectionName(IPipelineSpecificationBasic pipelineSpecificationBasic)
From a pipeline specification, get the name of the output connection that will be indexed first
in the pipeline.
|
java.lang.String |
IncrementalIngester.getLastIndexedOutputConnectionName(IPipelineSpecificationBasic pipelineSpecificationBasic)
From a pipeline specification, get the name of the output connection that will be indexed last
in the pipeline.
|
void |
IncrementalIngester.getPipelineDocumentIngestData(IngestStatuses rval,
IPipelineSpecificationBasic pipelineSpecificationBasic,
java.lang.String identifierClass,
java.lang.String identifierHash)
Look up ingestion data for a document.
|
void |
IncrementalIngester.getPipelineDocumentIngestDataMultiple(IngestStatuses rval,
IPipelineSpecificationBasic[] pipelineSpecificationBasics,
java.lang.String[] identifierClasses,
java.lang.String[] identifierHashes)
Look up ingestion data for a set of documents.
|
void |
IncrementalIngester.getPipelineDocumentIngestDataMultiple(IngestStatuses rval,
IPipelineSpecificationBasic pipelineSpecificationBasic,
java.lang.String[] identifierClasses,
java.lang.String[] identifierHashes)
Look up ingestion data for a SET of documents.
|
Modifier and Type | Interface and Description |
---|---|
interface |
IPipelineConnections
This interface caches IOutputConnection and ITransformationConnection objects
required by an IPipelineSpecification.
|
interface |
IPipelineSpecification
This interface describes a multi-output pipeline, where each stage has an already-computed
description string.
|
interface |
IPipelineSpecificationWithVersions
This interface describes a multi-output pipeline, with existing document version information from
each output..
|
Modifier and Type | Method and Description |
---|---|
void |
IIncrementalIngester.documentCheck(IPipelineSpecificationBasic pipelineSpecificationBasic,
java.lang.String identifierClass,
java.lang.String identifierHash,
long checkTime)
Note the fact that we checked a document (and found that it did not need to be ingested, because the
versions agreed).
|
void |
IIncrementalIngester.documentCheckMultiple(IPipelineSpecificationBasic pipelineSpecificationBasic,
java.lang.String[] identifierClasses,
java.lang.String[] identifierHashes,
long checkTime)
Note the fact that we checked a document (and found that it did not need to be ingested, because the
versions agreed).
|
void |
IIncrementalIngester.documentRecord(IPipelineSpecificationBasic pipelineSpecificationBasic,
java.lang.String identifierClass,
java.lang.String identifierHash,
java.lang.String componentHash,
java.lang.String documentVersion,
long recordTime)
Record a document version, but don't ingest it.
|
long |
IIncrementalIngester.getDocumentUpdateInterval(IPipelineSpecificationBasic pipelineSpecificationBasic,
java.lang.String identifierClass,
java.lang.String identifierHash)
Calculate the average time interval between changes for a document.
|
long[] |
IIncrementalIngester.getDocumentUpdateIntervalMultiple(IPipelineSpecificationBasic pipelineSpecificationBasic,
java.lang.String[] identifierClasses,
java.lang.String[] identifierHashes)
Calculate the average time interval between changes for a document.
|
java.lang.String |
IIncrementalIngester.getFirstIndexedOutputConnectionName(IPipelineSpecificationBasic pipelineSpecificationBasic)
From a pipeline specification, get the name of the output connection that will be indexed first
in the pipeline.
|
java.lang.String |
IIncrementalIngester.getLastIndexedOutputConnectionName(IPipelineSpecificationBasic pipelineSpecificationBasic)
From a pipeline specification, get the name of the output connection that will be indexed last
in the pipeline.
|
void |
IIncrementalIngester.getPipelineDocumentIngestData(IngestStatuses rval,
IPipelineSpecificationBasic pipelineSpecificationBasic,
java.lang.String identifierClass,
java.lang.String identifierHash)
Look up ingestion data for a document.
|
void |
IIncrementalIngester.getPipelineDocumentIngestDataMultiple(IngestStatuses rval,
IPipelineSpecificationBasic[] pipelineSpecificationBasics,
java.lang.String[] identifierClasses,
java.lang.String[] identifierHashes)
Look up ingestion data for a set of documents.
|
void |
IIncrementalIngester.getPipelineDocumentIngestDataMultiple(IngestStatuses rval,
IPipelineSpecificationBasic pipelineSpecificationBasic,
java.lang.String[] identifierClasses,
java.lang.String[] identifierHashes)
Look up ingestion data for a SET of documents.
|
Modifier and Type | Class and Description |
---|---|
class |
PipelineConnections
Pipeline connections implementation.
|
class |
PipelineSpecification
Class which handles pipeline specifications that include current (new) description strings.
|
class |
PipelineSpecificationBasic
Basic pipeline specification implementation.
|
class |
PipelineSpecificationWithVersions
Class which handles pipeline specifications, including both new versions and old versions.
|
Modifier and Type | Field and Description |
---|---|
protected IPipelineSpecificationBasic |
PipelineConnections.spec |
Constructor and Description |
---|
PipelineConnections(IPipelineSpecificationBasic spec,
ITransformationConnectionManager transformationConnectionManager,
IOutputConnectionManager outputConnectionManager) |