Package org.apache.manifoldcf.crawler.interfaces

Interface IJobManager

  • All Known Implementing Classes:
    JobManager
    public interface IJobManager
    This manager deals with jobs. Each job is associated with a repository connection, and has a number of scheduling options: starting every n hours/days/weeks/months, on specific dates, or "continuous" (which basically establishes a priority queue based on modification frequency). The job itself also specifies "seeds" (or starting points), which are the places that scanning begins. NOTE WELL: Every job is incremental. This means that the job will check for deletions among all the documents that it has scanned in the past, as part of the process of ingesting.

    • Method Detail

      • exportConfiguration

        void exportConfiguration​(java.io.OutputStream os)
                  throws java.io.IOException,
                         ManifoldCFException
        Export configuration
        Throws:
        java.io.IOException
        ManifoldCFException

      • importConfiguration

        void importConfiguration​(java.io.InputStream is)
                  throws java.io.IOException,
                         ManifoldCFException
        Import configuration
        Throws:
        java.io.IOException
        ManifoldCFException

      • deleteJob

        void deleteJob​(java.lang.Long id)
        throws ManifoldCFException
        Delete a job.
        Parameters:
        id - is the job's identifier. This method will purge all the records belonging to the job from the database, as well as remove all documents indexed by the job from the index.
        Throws:
        ManifoldCFException

      • load

        IJobDescription load​(java.lang.Long id,
                     boolean readOnly)
              throws ManifoldCFException
        Load a job.
        Parameters:
        id - is the job's identifier.
        readOnly - is true if a read-only object is desired.
        Returns:
        null if the job doesn't exist.
        Throws:
        ManifoldCFException

      • checkIfReference

        boolean checkIfReference​(java.lang.String connectionName)
                  throws ManifoldCFException
        See if there's a reference to a connection name.
        Parameters:
        connectionName - is the name of the connection.
        Returns:
        true if there is a reference, false otherwise.
        Throws:
        ManifoldCFException

      • checkIfNotificationReference

        boolean checkIfNotificationReference​(java.lang.String connectionName)
                              throws ManifoldCFException
        See if there's a reference to a notification connection name.
        Parameters:
        connectionName - is the name of the connection.
        Returns:
        true if there is a reference, false otherwise.
        Throws:
        ManifoldCFException

      • checkIfOutputReference

        boolean checkIfOutputReference​(java.lang.String connectionName)
                        throws ManifoldCFException
        See if there's a reference to an output connection name.
        Parameters:
        connectionName - is the name of the connection.
        Returns:
        true if there is a reference, false otherwise.
        Throws:
        ManifoldCFException

      • checkIfTransformationReference

        boolean checkIfTransformationReference​(java.lang.String connectionName)
                                throws ManifoldCFException
        See if there's a reference to a transformation connection name.
        Parameters:
        connectionName - is the name of the connection.
        Returns:
        true if there is a reference, false otherwise.
        Throws:
        ManifoldCFException

      • findJobsForConnection

        IJobDescription[] findJobsForConnection​(java.lang.String connectionName)
                                 throws ManifoldCFException
        Get the job IDs associated with a given connection name.
        Parameters:
        connectionName - is the name of the connection.
        Returns:
        the set of job id's associated with that connection.
        Throws:
        ManifoldCFException

      • clearJobSeedingState

        void clearJobSeedingState​(java.lang.Long jobID)
                   throws ManifoldCFException
        Clear job seeding state.
        Parameters:
        jobID - is the job ID.
        Throws:
        ManifoldCFException

      • cleanupProcessData

        void cleanupProcessData​(java.lang.String processID)
                 throws ManifoldCFException
        Reset the job queue for an individual process ID. If a node was shut down in the middle of doing something, sufficient information should be around in the database to allow the node's activities to be cleaned up.
        Parameters:
        processID - is the process ID of the node we want to clean up after.
        Throws:
        ManifoldCFException

      • cleanupProcessData

        void cleanupProcessData()
                 throws ManifoldCFException
        Reset the job queue for all process IDs. If a node was shut down in the middle of doing something, sufficient information should be around in the database to allow the node's activities to be cleaned up.
        Throws:
        ManifoldCFException

      • prepareForClusterStart

        void prepareForClusterStart()
                     throws ManifoldCFException
        Prepare to start the entire cluster. If there are no other nodes alive, then at the time the first node comes up, we need to reset the job queue for ALL processes that had been running before. This method must be called in addition to cleanupProcessData().
        Throws:
        ManifoldCFException

      • resetDocumentWorkerStatus

        void resetDocumentWorkerStatus​(java.lang.String processID)
                        throws ManifoldCFException
        Reset as part of restoring document worker threads.
        Parameters:
        processID - is the current process ID.
        Throws:
        ManifoldCFException

      • resetDocDeleteWorkerStatus

        void resetDocDeleteWorkerStatus​(java.lang.String processID)
                         throws ManifoldCFException
        Reset as part of restoring doc delete threads.
        Parameters:
        processID - is the current process ID.
        Throws:
        ManifoldCFException

      • resetDocCleanupWorkerStatus

        void resetDocCleanupWorkerStatus​(java.lang.String processID)
                          throws ManifoldCFException
        Reset as part of restoring doc cleanup threads.
        Parameters:
        processID - is the current process ID.
        Throws:
        ManifoldCFException

      • resetDeleteStartupWorkerStatus

        void resetDeleteStartupWorkerStatus​(java.lang.String processID)
                             throws ManifoldCFException
        Reset as part of restoring delete startup threads.
        Parameters:
        processID - is the current process ID.
        Throws:
        ManifoldCFException

      • resetNotificationWorkerStatus

        void resetNotificationWorkerStatus​(java.lang.String processID)
                            throws ManifoldCFException
        Reset as part of restoring notification threads.
        Parameters:
        processID - is the current process ID.
        Throws:
        ManifoldCFException

      • resetStartupWorkerStatus

        void resetStartupWorkerStatus​(java.lang.String processID)
                       throws ManifoldCFException
        Reset as part of restoring startup threads.
        Parameters:
        processID - is the current process ID.
        Throws:
        ManifoldCFException

      • clearAllDocumentPriorities

        void clearAllDocumentPriorities()
                         throws ManifoldCFException
        Clear all document priorities, in preparation for reprioritization of all previously-prioritized documents. This method is called to start the dynamic reprioritization cycle, which follows this method with explicit prioritization of all documents, piece-meal, using getNextNotYetProcessedReprioritizationDocuments(), and writeDocumentPriorities().
        Throws:
        ManifoldCFException

      • getNextNotYetProcessedReprioritizationDocuments

        DocumentDescription[] getNextNotYetProcessedReprioritizationDocuments​(java.lang.String processID,
                                                                      int n)
                                                               throws ManifoldCFException
        Get a list of not-yet-processed documents to reprioritize. Documents in all jobs will be returned by this method. Up to n document descriptions will be returned.
        Parameters:
        processID - is the process that requests the reprioritization documents.
        n - is the maximum number of document descriptions desired.
        Returns:
        the document descriptions.
        Throws:
        ManifoldCFException

      • writeDocumentPriorities

        void writeDocumentPriorities​(DocumentDescription[] descriptions,
                             IPriorityCalculator[] priorities)
                      throws ManifoldCFException
        Save a set of document priorities. In the case where a document was eligible to have its priority set, but it no longer is eligible, then the provided priority will not be written.
        Parameters:
        descriptions - are the document descriptions.
        priorities - are the desired priorities.
        Throws:
        ManifoldCFException

      • getExpiredDocuments

        DocumentSetAndFlags getExpiredDocuments​(java.lang.String processID,
                                        int n,
                                        long currentTime)
                                 throws ManifoldCFException
        Get up to the next n documents to be expired. This method marks the documents whose descriptions have been returned as "being processed", or active. The same marking is used as is used for documents that have been queued for worker threads. The model is thus identical.
        Parameters:
        processID - is the current process ID.
        n - is the maximum number of records desired.
        currentTime - is the current time.
        Returns:
        the array of document descriptions to expire.
        Throws:
        ManifoldCFException

      • getNextDocuments

        DocumentDescription[] getNextDocuments​(java.lang.String processID,
                                       int n,
                                       long currentTime,
                                       long interval,
                                       BlockingDocuments blockingDocuments,
                                       PerformanceStatistics statistics,
                                       DepthStatistics scanRecord)
                                throws ManifoldCFException
        Get up to the next n document(s) to be fetched and processed. This fetch returns records that contain the document identifier, plus all instructions pertaining to the document's handling (e.g. whether it should be refetched if the version has not changed). This method also marks the documents whose descriptions have be returned as "being processed".
        Parameters:
        processID - is the current process ID.
        n - is the number of documents desired.
        currentTime - is the current time; some fetches do not occur until a specific time.
        interval - is the number of milliseconds that this set of documents should represent (for throttling).
        blockingDocuments - is the place to record documents that were encountered, are eligible for reprioritization, but could not be queued due to throttling considerations.
        statistics - are the current performance statistics per connection, which are used to balance the queue stuffing so that individual connections are not overwhelmed.
        scanRecord - retains the bins from all documents encountered from the query, even those that were skipped due to being overcommitted.
        Returns:
        the array of document descriptions to fetch and process.
        Throws:
        ManifoldCFException

      • checkJobActive

        boolean checkJobActive​(java.lang.Long jobID)
                throws ManifoldCFException
        Verify that a specific job is indeed still active. This is used to permit abort or pause to be relatively speedy. The query done within MUST be cached in order to not cause undue performance degradation.
        Parameters:
        jobID - is the job identifier.
        Returns:
        true if the job is in one of the "active" states.
        Throws:
        ManifoldCFException

      • checkJobBusy

        boolean checkJobBusy​(java.lang.Long jobID)
              throws ManifoldCFException
        Verify if a job is still processing documents, or no longer has any outstanding active documents
        Throws:
        ManifoldCFException

      • markDocumentCompletedMultiple

        void markDocumentCompletedMultiple​(DocumentDescription[] documentDescriptions)
                            throws ManifoldCFException
        Note completion of document processing by a job thread of a document. This method causes the state of the document to be marked as "completed".
        Parameters:
        documentDescriptions - are the description objects for the documents that were processed.
        Throws:
        ManifoldCFException

      • markDocumentCompleted

        void markDocumentCompleted​(DocumentDescription documentDescription)
                    throws ManifoldCFException
        Note completion of document processing by a job thread of a document. This method causes the state of the document to be marked as "completed".
        Parameters:
        documentDescription - is the description object for the document that was processed.
        Throws:
        ManifoldCFException

      • markDocumentDeletedMultiple

        DocumentDescription[] markDocumentDeletedMultiple​(java.lang.Long jobID,
                                                  java.lang.String[] legalLinkTypes,
                                                  DocumentDescription[] documentDescriptions,
                                                  int hopcountMethod)
                                           throws ManifoldCFException
        Delete from queue as a result of processing of an active document. The document is expected to be in one of the active states: ACTIVE, ACTIVESEEDING, ACTIVENEEDSRESCAN, ACTIVESEEDINGNEEDSRESCAN. The RESCAN variants are interpreted as meaning that the document should not be deleted, but should instead be popped back on the queue for a repeat processing attempt.
        Parameters:
        documentDescriptions - are the set of description objects for the documents that were processed.
        hopcountMethod - describes how to handle deletions for hopcount purposes.
        Returns:
        the set of documents for which carrydown data was changed by this operation. These documents are likely to be requeued as a result of the change.
        Throws:
        ManifoldCFException

      • markDocumentDeleted

        DocumentDescription[] markDocumentDeleted​(java.lang.Long jobID,
                                          java.lang.String[] legalLinkTypes,
                                          DocumentDescription documentDescription,
                                          int hopcountMethod)
                                   throws ManifoldCFException
        Delete from queue as a result of processing of an active document. The document is expected to be in one of the active states: ACTIVE, ACTIVESEEDING, ACTIVENEEDSRESCAN, ACTIVESEEDINGNEEDSRESCAN. The RESCAN variants are interpreted as meaning that the document should not be deleted, but should instead be popped back on the queue for a repeat processing attempt.
        Parameters:
        documentDescription - is the description object for the document that was processed.
        hopcountMethod - describes how to handle deletions for hopcount purposes.
        Returns:
        the set of documents for which carrydown data was changed by this operation. These documents are likely to be requeued as a result of the change.
        Throws:
        ManifoldCFException

      • markDocumentHopcountRemovalMultiple

        DocumentDescription[] markDocumentHopcountRemovalMultiple​(java.lang.Long jobID,
                                                          java.lang.String[] legalLinkTypes,
                                                          DocumentDescription[] documentDescriptions,
                                                          int hopcountMethod)
                                                   throws ManifoldCFException
        Mark hopcount removal from queue as a result of processing of an active document. The document is expected to be in one of the active states: ACTIVE, ACTIVESEEDING, ACTIVENEEDSRESCAN, ACTIVESEEDINGNEEDSRESCAN. The RESCAN variants are interpreted as meaning that the document should not be marked as removed, but should instead be popped back on the queue for a repeat processing attempt.
        Parameters:
        documentDescriptions - are the set of description objects for the documents that were processed.
        hopcountMethod - describes how to handle deletions for hopcount purposes.
        Returns:
        the set of documents for which carrydown data was changed by this operation. These documents are likely to be requeued as a result of the change.
        Throws:
        ManifoldCFException

      • markDocumentHopcountRemoval

        DocumentDescription[] markDocumentHopcountRemoval​(java.lang.Long jobID,
                                                  java.lang.String[] legalLinkTypes,
                                                  DocumentDescription documentDescription,
                                                  int hopcountMethod)
                                           throws ManifoldCFException
        Mark hopcount removal from queue as a result of processing of an active document. The document is expected to be in one of the active states: ACTIVE, ACTIVESEEDING, ACTIVENEEDSRESCAN, ACTIVESEEDINGNEEDSRESCAN. The RESCAN variants are interpreted as meaning that the document should not be marked as removed, but should instead be popped back on the queue for a repeat processing attempt.
        Parameters:
        documentDescription - is the description object for the document that was processed.
        hopcountMethod - describes how to handle deletions for hopcount purposes.
        Returns:
        the set of documents for which carrydown data was changed by this operation. These documents are likely to be requeued as a result of the change.
        Throws:
        ManifoldCFException

      • markDocumentExpiredMultiple

        DocumentDescription[] markDocumentExpiredMultiple​(java.lang.Long jobID,
                                                  java.lang.String[] legalLinkTypes,
                                                  DocumentDescription[] documentDescriptions,
                                                  int hopcountMethod)
                                           throws ManifoldCFException
        Delete from queue as a result of expiration of an active document. The document is expected to be in one of the active states: ACTIVE, ACTIVESEEDING, ACTIVENEEDSRESCAN, ACTIVESEEDINGNEEDSRESCAN. Since the document expired, no special activity takes place as a result of the document being in a RESCAN state.
        Parameters:
        documentDescriptions - are the set of description objects for the documents that were processed.
        hopcountMethod - describes how to handle deletions for hopcount purposes.
        Returns:
        the set of documents for which carrydown data was changed by this operation. These documents are likely to be requeued as a result of the change.
        Throws:
        ManifoldCFException

      • markDocumentExpired

        DocumentDescription[] markDocumentExpired​(java.lang.Long jobID,
                                          java.lang.String[] legalLinkTypes,
                                          DocumentDescription documentDescription,
                                          int hopcountMethod)
                                   throws ManifoldCFException
        Delete from queue as a result of expiration of an active document. The document is expected to be in one of the active states: ACTIVE, ACTIVESEEDING, ACTIVENEEDSRESCAN, ACTIVESEEDINGNEEDSRESCAN. Since the document expired, no special activity takes place as a result of the document being in a RESCAN state.
        Parameters:
        documentDescription - is the description object for the document that was processed.
        hopcountMethod - describes how to handle deletions for hopcount purposes.
        Returns:
        the set of documents for which carrydown data was changed by this operation. These documents are likely to be requeued as a result of the change.
        Throws:
        ManifoldCFException

      • markDocumentCleanedUpMultiple

        DocumentDescription[] markDocumentCleanedUpMultiple​(java.lang.Long jobID,
                                                    java.lang.String[] legalLinkTypes,
                                                    DocumentDescription[] documentDescriptions,
                                                    int hopcountMethod)
                                             throws ManifoldCFException
        Delete from queue as a result of cleaning up an unreachable document. The document is expected to be in the PURGATORY state. There is never any need to reprocess the document.
        Parameters:
        documentDescriptions - are the set of description objects for the documents that were processed.
        hopcountMethod - describes how to handle deletions for hopcount purposes.
        Returns:
        the set of documents for which carrydown data was changed by this operation. These documents are likely to be requeued as a result of the change.
        Throws:
        ManifoldCFException

      • markDocumentCleanedUp

        DocumentDescription[] markDocumentCleanedUp​(java.lang.Long jobID,
                                            java.lang.String[] legalLinkTypes,
                                            DocumentDescription documentDescription,
                                            int hopcountMethod)
                                     throws ManifoldCFException
        Delete from queue as a result of cleaning up an unreachable document. The document is expected to be in the PURGATORY state. There is never any need to reprocess the document.
        Parameters:
        documentDescription - is the description object for the document that was processed.
        hopcountMethod - describes how to handle deletions for hopcount purposes.
        Returns:
        the set of documents for which carrydown data was changed by this operation. These documents are likely to be requeued as a result of the change.
        Throws:
        ManifoldCFException

      • carrydownChangeDocumentMultiple

        void carrydownChangeDocumentMultiple​(DocumentDescription[] documentDescriptions,
                                     IPriorityCalculator[] docPriorities)
                              throws ManifoldCFException
        Requeue a document set because of carrydown changes. This method is called when carrydown data is modified for a set of documents. The documents must be requeued for immediate reprocessing, even to the extent that if one is *already* being processed, it will need to be done over again.
        Parameters:
        documentDescriptions - is the set of description objects for the documents that have had their parent carrydown information changed.
        docPriorities - are the document priorities to assign to the documents, if needed.
        Throws:
        ManifoldCFException

      • carrydownChangeDocument

        void carrydownChangeDocument​(DocumentDescription documentDescription,
                             IPriorityCalculator docPriority)
                      throws ManifoldCFException
        Requeue a document because of carrydown changes. This method is called when carrydown data is modified for a document. The document must be requeued for immediate reprocessing, even to the extent that if it is *already* being processed, it will need to be done over again.
        Parameters:
        documentDescription - is the description object for the document that has had its parent carrydown information changed.
        docPriority - is the document priority to assign to the document, if needed.
        Throws:
        ManifoldCFException

      • requeueDocumentMultiple

        void requeueDocumentMultiple​(DocumentDescription[] documentDescriptions,
                             java.lang.Long[] executeTimes,
                             int[] actions)
                      throws ManifoldCFException
        Requeue a document for further processing in the future. This method is called after a document is processed, when the job is a "continuous" one. It is essentially equivalent to noting that the document processing is complete, except the document remains on the queue.
        Parameters:
        documentDescriptions - is the set of description objects for the document that was processed.
        executeTimes - are the times that the documents should be rescanned. Null indicates "never".
        actions - are what should be done when the time arrives. Choices are ACTION_RESCAN or ACTION_REMOVE.
        Throws:
        ManifoldCFException

      • requeueDocument

        void requeueDocument​(DocumentDescription documentDescription,
                     java.lang.Long executeTime,
                     int action)
              throws ManifoldCFException
        Requeue a document for further processing in the future. This method is called after a document is processed, when the job is a "continuous" one. It is essentially equivalent to noting that the document processing is complete, except the document remains on the queue.
        Parameters:
        documentDescription - is the description object for the document that was processed.
        executeTime - is the time that the document should be rescanned. Null indicates "never".
        action - is what should be done when the time arrives. Choices include ACTION_RESCAN or ACTION_REMOVE.
        Throws:
        ManifoldCFException

      • resetDocumentMultiple

        void resetDocumentMultiple​(DocumentDescription[] documentDescriptions,
                           long executeTime,
                           int action,
                           long failTime,
                           int failCount)
                    throws ManifoldCFException
        Reset documents for further processing in the future. This method is called after a service interruption is thrown. It is essentially equivalent to resetting the time for documents to be reprocessed.
        Parameters:
        documentDescriptions - is the set of description objects for the document that was processed.
        executeTime - is the time that the documents should be rescanned.
        failTime - is the time beyond which hard failure should occur.
        failCount - is the number of permitted failures before a hard error is signalled.
        Throws:
        ManifoldCFException

      • resetDocument

        void resetDocument​(DocumentDescription documentDescription,
                   long executeTime,
                   int action,
                   long failTime,
                   int failCount)
            throws ManifoldCFException
        Reset an active document back to its former state. This gets done when there's a service interruption and the document cannot be processed yet.
        Parameters:
        documentDescription - is the description object for the document that was processed.
        executeTime - is the time that the document should be rescanned.
        failTime - is the time that the document should be considered to have failed, if it has not been successfully read until then.
        failCount - is the number of permitted failures before a hard error is signalled.
        Throws:
        ManifoldCFException

      • resetDeletingDocumentMultiple

        void resetDeletingDocumentMultiple​(DocumentDescription[] documentDescriptions,
                                   long checkTime)
                            throws ManifoldCFException
        Reset a set of deleting documents for further processing in the future. This method is called after some unknown number of the documents were deleted, but then an ingestion service interruption occurred. Note well: The logic here basically presumes that we cannot know whether the documents were indeed processed or not. If we knew for a fact that none of the documents had been handled, it would be possible to look at the document's current status and decide what the new status ought to be, based on a true rollback scenario. Such cases, however, are rare enough so that special logic is probably not worth it.
        Parameters:
        documentDescriptions - is the set of description objects for the document that was processed.
        checkTime - is the minimum time for the next cleaning attempt.
        Throws:
        ManifoldCFException

      • resetDeletingDocument

        void resetDeletingDocument​(DocumentDescription documentDescription,
                           long checkTime)
                    throws ManifoldCFException
        Reset a deleting document back to its former state. This gets done when a deleting thread sees a service interruption, etc., from the ingestion system.
        Parameters:
        documentDescription - is the description object for the document that was cleaned.
        checkTime - is the minimum time for the next cleaning attempt.
        Throws:
        ManifoldCFException

      • resetCleaningDocument

        void resetCleaningDocument​(DocumentDescription documentDescription,
                           long checkTime)
                    throws ManifoldCFException
        Reset a cleaning document back to its former state. This gets done when a cleaning thread sees a service interruption, etc., from the ingestion system.
        Parameters:
        documentDescription - is the description object for the document that was cleaned.
        checkTime - is the minimum time for the next cleaning attempt.
        Throws:
        ManifoldCFException

      • resetCleaningDocumentMultiple

        void resetCleaningDocumentMultiple​(DocumentDescription[] documentDescriptions,
                                   long checkTime)
                            throws ManifoldCFException
        Reset a set of cleaning documents for further processing in the future. This method is called after some unknown number of the documents were cleaned, but then an ingestion service interruption occurred. Note well: The logic here basically presumes that we cannot know whether the documents were indeed cleaned or not. If we knew for a fact that none of the documents had been handled, it would be possible to look at the document's current status and decide what the new status ought to be, based on a true rollback scenario. Such cases, however, are rare enough so that special logic is probably not worth it.
        Parameters:
        documentDescriptions - is the set of description objects for the document that was cleaned.
        checkTime - is the minimum time for the next cleaning attempt.
        Throws:
        ManifoldCFException

      • retryStartup

        void retryStartup​(JobStartRecord jobStartRecord,
                  long failTime,
                  int failRetryCount)
           throws ManifoldCFException
        Retry startup.
        Parameters:
        jobStartRecord - is the current job startup record.
        failTime - is the new fail time (-1L if none).
        failRetryCount - is the new fail retry count (-1 if none).
        Throws:
        ManifoldCFException

      • retrySeeding

        void retrySeeding​(JobSeedingRecord jobSeedingRecord,
                  long failTime,
                  int failRetryCount)
           throws ManifoldCFException
        Retry seeding.
        Parameters:
        jobSeedingRecord - is the current job seeding record.
        failTime - is the new fail time (-1L if none).
        failRetryCount - is the new fail retry count (-1 if none).
        Throws:
        ManifoldCFException

      • retryNotification

        void retryNotification​(JobNotifyRecord jobNotifyRecord,
                       long failTime,
                       int failRetryCount)
                throws ManifoldCFException
        Retry notification.
        Parameters:
        jobNotifyRecord - is the current job notification record.
        failTime - is the new fail time (-1L if none).
        failRetryCount - is the new fail retry count (-1 if none).
        Throws:
        ManifoldCFException

      • retryDeleteNotification

        void retryDeleteNotification​(JobNotifyRecord jnr,
                             long failTime,
                             int failCount)
                      throws ManifoldCFException
        Retry delete notification.
        Parameters:
        jnr - is the current job notification record.
        failTime - is the new fail time (-1L if none).
        failCount - is the new fail retry count (-1 if none).
        Throws:
        ManifoldCFException

      • addDocumentsInitial

        void addDocumentsInitial​(java.lang.String processID,
                         java.lang.Long jobID,
                         java.lang.String[] legalLinkTypes,
                         java.lang.String[] docIDHashes,
                         java.lang.String[] docIDs,
                         boolean overrideSchedule,
                         int hopcountMethod,
                         IPriorityCalculator[] documentPriorities,
                         java.lang.String[][] prereqEventNames)
                  throws ManifoldCFException
        Add an initial set of documents to the queue. This method is called during job startup, when the queue is being loaded. A set of document references is passed to this method, which updates the status of the document in the specified job's queue, according to specific state rules.
        Parameters:
        processID - is the current process ID.
        jobID - is the job identifier.
        legalLinkTypes - is the set of legal link types that this connector generates.
        docIDHashes - are the hashes of the local document identifiers (primary key).
        docIDs - are the local document identifiers.
        overrideSchedule - is true if any existing document schedule should be overridden.
        hopcountMethod - is either accurate, nodelete, or neverdelete.
        documentPriorities - are the document priorities corresponding to the document identifiers.
        prereqEventNames - are the events that must be completed before each document can be processed.
        Throws:
        ManifoldCFException

      • addRemainingDocumentsInitial

        void addRemainingDocumentsInitial​(java.lang.String processID,
                                  java.lang.Long jobID,
                                  java.lang.String[] legalLinkTypes,
                                  java.lang.String[] docIDHashes,
                                  int hopcountMethod)
                           throws ManifoldCFException
        Add an initial set of remaining documents to the queue. This method is called during job startup, when the queue is being loaded, to list documents that were NOT included by calling addDocumentsInitial(). Documents listed here are simply designed to enable the framework to get rid of old, invalid seeds. They are not queued for processing.
        Parameters:
        processID - is the current process ID.
        jobID - is the job identifier.
        legalLinkTypes - is the set of legal link types that this connector generates.
        docIDHashes - are the hash values of the local document identifiers.
        hopcountMethod - is either accurate, nodelete, or neverdelete.
        Throws:
        ManifoldCFException

      • doneDocumentsInitial

        void doneDocumentsInitial​(java.lang.Long jobID,
                          java.lang.String[] legalLinkTypes,
                          boolean isPartial,
                          int hopcountMethod)
                   throws ManifoldCFException
        Signal that a seeding pass has been done. Call this method at the end of a seeding pass. It is used to perform the bookkeeping necessary to maintain the hopcount table.
        Parameters:
        jobID - is the job identifier.
        legalLinkTypes - is the set of legal link types that this connector generates.
        isPartial - is set if the seeds provided are only a partial list. Some connectors cannot supply a full list of seeds on every seeding iteration; this acknowledges that limitation.
        hopcountMethod - describes how to handle deletions for hopcount purposes.
        Throws:
        ManifoldCFException

      • beginEventSequence

        boolean beginEventSequence​(java.lang.String processID,
                           java.lang.String eventName)
                    throws ManifoldCFException
        Begin an event sequence.
        Parameters:
        processID - is the current process ID.
        eventName - is the name of the event.
        Returns:
        true if the event could be created, or false if it's already there.
        Throws:
        ManifoldCFException

      • completeEventSequence

        void completeEventSequence​(java.lang.String eventName)
                    throws ManifoldCFException
        Complete an event sequence.
        Parameters:
        eventName - is the name of the event.
        Throws:
        ManifoldCFException

      • findHopCounts

        boolean[] findHopCounts​(java.lang.Long jobID,
                        java.lang.String[] legalLinkTypes,
                        java.lang.String[] docIDHashes,
                        java.lang.String linkType,
                        int limit,
                        int hopcountMethod)
                 throws ManifoldCFException
        Get the specified hop counts, with the limit as described.
        Parameters:
        jobID - is the job identifier.
        legalLinkTypes - is the set of legal link types that this connector generates.
        docIDHashes - is the set of document hashes to find the hopcount for.
        linkType - is the kind of link to find the hopcount for.
        limit - is the limit, beyond which a negative distance may be returned.
        hopcountMethod - describes how to handle deletions for hopcount purposes.
        Returns:
        a vector of booleans corresponding to the documents requested. A true value is returned if the document is within the specified limit, false otherwise.
        Throws:
        ManifoldCFException

      • getAllSeeds

        java.lang.String[] getAllSeeds​(java.lang.Long jobID)
                        throws ManifoldCFException
        Get all the current seeds. Returns the seed document identifiers for a job.
        Parameters:
        jobID - is the job identifier.
        Returns:
        the document identifier hashes that are currently considered to be seeds.
        Throws:
        ManifoldCFException

      • addDocument

        void addDocument​(java.lang.String processID,
                 java.lang.Long jobID,
                 java.lang.String[] legalLinkTypes,
                 java.lang.String docIDHash,
                 java.lang.String docID,
                 java.lang.String parentIdentifierHash,
                 java.lang.String relationshipType,
                 int hopcountMethod,
                 java.lang.String[] dataNames,
                 java.lang.Object[][] dataValues,
                 IPriorityCalculator priority,
                 java.lang.String[] prereqEventNames)
          throws ManifoldCFException
        Add a document to the queue. This method is called during document processing, when a document reference is discovered. The document reference is passed to this method, which updates the status of the document in the specified job's queue, according to specific state rules.
        Parameters:
        processID - is the current process ID.
        jobID - is the job identifier.
        legalLinkTypes - is the set of legal link types that this connector generates.
        docIDHash - is the local document identifier hash value.
        parentIdentifierHash - is the optional parent identifier hash value for this document. Pass null if none. MUST be present in the case of carrydown information.
        relationshipType - is the optional link type between this document and its parent. Pass null if there is no relationship with a parent.
        hopcountMethod - is either accurate, nodelete, or neverdelete.
        dataNames - are the names of the data to carry down to the child from this parent.
        dataValues - are the values to carry down to the child from this parent, corresponding to dataNames above. If CharacterInput objects are passed in here, it is the caller's responsibility to clean these up.
        priority - is the desired document priority for the document.
        prereqEventNames - are the events that must be completed before the document can be processed.
        Throws:
        ManifoldCFException

      • addDocuments

        void addDocuments​(java.lang.String processID,
                  java.lang.Long jobID,
                  java.lang.String[] legalLinkTypes,
                  java.lang.String[] docIDHashes,
                  java.lang.String[] docIDs,
                  java.lang.String parentIdentifierHash,
                  java.lang.String relationshipType,
                  int hopcountMethod,
                  java.lang.String[][] dataNames,
                  java.lang.Object[][][] dataValues,
                  IPriorityCalculator[] priorities,
                  java.lang.String[][] prereqEventNames)
           throws ManifoldCFException
        Add documents to the queue in bulk. This method is called during document processing, when a set of document references are discovered. The document references are passed to this method, which updates the status of the document(s) in the specified job's queue, according to specific state rules.
        Parameters:
        processID - is the current process ID.
        jobID - is the job identifier.
        legalLinkTypes - is the set of legal link types that this connector generates.
        docIDHashes - are the hashes of the local document identifiers.
        docIDs - are the local document identifiers.
        parentIdentifierHash - is the optional parent identifier hash of these documents. Pass null if none. MUST be present in the case of carrydown information.
        relationshipType - is the optional link type between this document and its parent. Pass null if there is no relationship with a parent.
        hopcountMethod - is either accurate, nodelete, or neverdelete.
        dataNames - are the names of the data to carry down to the child from this parent.
        dataValues - are the values to carry down to the child from this parent, corresponding to dataNames above. If CharacterInput objects are passed in here, it is the caller's responsibility to clean these up.
        priorities - are the desired document priorities for the documents.
        prereqEventNames - are the events that must be completed before each document can be processed.
        Throws:
        ManifoldCFException

      • finishDocuments

        DocumentDescription[] finishDocuments​(java.lang.Long jobID,
                                      java.lang.String[] legalLinkTypes,
                                      java.lang.String[] parentIdentifierHashes,
                                      int hopcountMethod)
                               throws ManifoldCFException
        Complete adding child documents to the queue, for a set of documents. This method is called at the end of document processing, to help the hopcount tracking engine do its bookkeeping.
        Parameters:
        jobID - is the job identifier.
        legalLinkTypes - is the set of legal link types that this connector generates.
        parentIdentifierHashes - are the hashes of the document identifiers for whom child link extraction just took place.
        hopcountMethod - describes how to handle deletions for hopcount purposes.
        Returns:
        the set of documents for which carrydown data was changed by this operation. These documents are likely to be requeued as a result of the change.
        Throws:
        ManifoldCFException

      • revertDocuments

        void revertDocuments​(java.lang.Long jobID,
                     java.lang.String[] legalLinkTypes,
                     java.lang.String[] parentIdentifierHashes)
              throws ManifoldCFException
        Undo the addition of child documents to the queue, for a set of documents. This method is called at the end of document processing, to back out any incomplete additions to the queue, and restore the status quo ante prior to the incomplete additions. Call this method instead of finishDocuments() if the addition of documents was not completed.
        Parameters:
        jobID - is the job identifier.
        legalLinkTypes - is the set of legal link types that this connector generates.
        parentIdentifierHashes - are the hashes of the document identifiers for whom child link extraction just took place.
        Throws:
        ManifoldCFException

      • retrieveParentData

        java.lang.String[] retrieveParentData​(java.lang.Long jobID,
                                      java.lang.String docIDHash,
                                      java.lang.String dataName)
                               throws ManifoldCFException
        Retrieve specific parent data for a given document.
        Parameters:
        jobID - is the job identifier.
        docIDHash - is the hash of the document identifier.
        dataName - is the kind of data to retrieve.
        Returns:
        the unique data values.
        Throws:
        ManifoldCFException

      • retrieveParentDataAsFiles

        CharacterInput[] retrieveParentDataAsFiles​(java.lang.Long jobID,
                                           java.lang.String docIDHash,
                                           java.lang.String dataName)
                                    throws ManifoldCFException
        Retrieve specific parent data for a given document.
        Parameters:
        jobID - is the job identifier.
        docIDHash - is the document identifier hash value.
        dataName - is the kind of data to retrieve.
        Returns:
        the unique data values.
        Throws:
        ManifoldCFException

      • manualStart

        void manualStart​(java.lang.Long jobID,
                 boolean requestMinimum)
          throws ManifoldCFException
        Manually start a job. The specified job will be run REGARDLESS of the timed windows, and will not cease until complete. If the job is already running, this operation will assure that the job does not pause when its window ends. The job can be manually paused, or manually aborted.
        Parameters:
        jobID - is the ID of the job to start.
        requestMinimum - is true if a minimal job run is requested.
        Throws:
        ManifoldCFException

      • manualStart

        void manualStart​(java.lang.Long jobID)
          throws ManifoldCFException
        Manually start a job. The specified job will be run REGARDLESS of the timed windows, and will not cease until complete. If the job is already running, this operation will assure that the job does not pause when its window ends. The job can be manually paused, or manually aborted.
        Parameters:
        jobID - is the ID of the job to start.
        Throws:
        ManifoldCFException

      • manualAbort

        void manualAbort​(java.lang.Long jobID)
          throws ManifoldCFException
        Manually abort a running job. The job will be permanently stopped, and will not run again until automatically started based on schedule, or manually started.
        Parameters:
        jobID - is the job to abort.
        Throws:
        ManifoldCFException

      • manualAbortRestart

        void manualAbortRestart​(java.lang.Long jobID,
                        boolean requestMinimum)
                 throws ManifoldCFException
        Manually restart a running job. The job will be stopped and restarted. Any schedule affinity will be lost, until the job finishes on its own.
        Parameters:
        jobID - is the job to abort.
        requestMinimum - is true if a minimal job run is requested.
        Throws:
        ManifoldCFException

      • manualAbortRestart

        void manualAbortRestart​(java.lang.Long jobID)
                 throws ManifoldCFException
        Manually restart a running job. The job will be stopped and restarted. Any schedule affinity will be lost, until the job finishes on its own.
        Parameters:
        jobID - is the job to abort.
        Throws:
        ManifoldCFException

      • resetJobSchedule

        void resetJobSchedule​(java.lang.Long jobID)
               throws ManifoldCFException
        Reset job schedule. This re-evaluates whether the job should be started now. This method would typically be called after a job's scheduling window has been changed.
        Parameters:
        jobID - is the job identifier.
        Throws:
        ManifoldCFException

      • startJobs

        void startJobs​(long currentTime,
               java.util.List<java.lang.Long> unwaitList)
        throws ManifoldCFException
        Start jobs based on schedule. This method marks all the appropriate jobs as "in progress", which is all that should be needed to start them.
        Parameters:
        currentTime - is the current time in milliseconds since epoch.
        unwaitList - is filled in with the set of job id's that were resumed (Long's).
        Throws:
        ManifoldCFException

      • waitJobs

        void waitJobs​(long currentTime,
              java.util.List<java.lang.Long> waitList)
       throws ManifoldCFException
        Put active or paused jobs in wait state, if they've exceeded their window.
        Parameters:
        currentTime - is the current time in milliseconds since epoch.
        waitList - is filled in with the set of job id's that were put into a wait state (Long's).
        Throws:
        ManifoldCFException

      • getJobsReadyForSeeding

        JobSeedingRecord[] getJobsReadyForSeeding​(java.lang.String processID,
                                          long currentTime)
                                   throws ManifoldCFException
        Get the list of jobs that are ready for seeding.
        Parameters:
        processID - is the current process ID.
        currentTime - is the current time in milliseconds since epoch.
        Returns:
        jobs that are active and are running in adaptive mode. These will be seeded based on what the connector says should be added to the queue.
        Throws:
        ManifoldCFException

      • resetSeedJob

        void resetSeedJob​(java.lang.Long jobID)
           throws ManifoldCFException
        Reset a seeding job back to "active" state.
        Parameters:
        jobID - is the job id.
        Throws:
        ManifoldCFException

      • getJobsReadyForDeleteCleanup

        JobDeleteRecord[] getJobsReadyForDeleteCleanup​(java.lang.String processID)
                                        throws ManifoldCFException
        Get the list of jobs that are ready for delete cleanup.
        Parameters:
        processID - is the current process ID.
        Returns:
        jobs that were in the "readyfordelete" state.
        Throws:
        ManifoldCFException

      • getJobsReadyForStartup

        JobStartRecord[] getJobsReadyForStartup​(java.lang.String processID)
                                 throws ManifoldCFException
        Get the list of jobs that are ready for startup.
        Parameters:
        processID - is the current process ID.
        Returns:
        jobs that were in the "readyforstartup" state. These will be marked as being in the "starting up" state.
        Throws:
        ManifoldCFException

      • getJobsReadyForInactivity

        JobNotifyRecord[] getJobsReadyForInactivity​(java.lang.String processID)
                                     throws ManifoldCFException
        Find the list of jobs that need to have their connectors notified of job completion.
        Parameters:
        processID - is the current process ID.
        Returns:
        the ID's of jobs that need their output connectors notified in order to become inactive.
        Throws:
        ManifoldCFException

      • getJobsReadyForDelete

        JobNotifyRecord[] getJobsReadyForDelete​(java.lang.String processID)
                                 throws ManifoldCFException
        Find the list of jobs that need to have their connectors notified of job deletion.
        Parameters:
        processID - is the process ID.
        Returns:
        the ID's of jobs that need their output connectors notified in order to be removed.
        Throws:
        ManifoldCFException

      • inactivateJob

        void inactivateJob​(java.lang.Long jobID)
            throws ManifoldCFException
        Inactivate a job, from the notification state.
        Parameters:
        jobID - is the ID of the job to inactivate.
        Throws:
        ManifoldCFException

      • removeJob

        void removeJob​(java.lang.Long jobID)
        throws ManifoldCFException
        Remove a job, from the notification state.
        Parameters:
        jobID - is the ID of the job to remove.
        Throws:
        ManifoldCFException

      • resetStartDeleteJob

        void resetStartDeleteJob​(java.lang.Long jobID)
                  throws ManifoldCFException
        Reset a job starting for delete back to "ready for delete" state.
        Parameters:
        jobID - is the job id.
        Throws:
        ManifoldCFException

      • resetNotifyJob

        void resetNotifyJob​(java.lang.Long jobID)
             throws ManifoldCFException
        Reset a job that is notifying back to "ready for notify" state.
        Parameters:
        jobID - is the job id.
        Throws:
        ManifoldCFException

      • resetDeleteNotifyJob

        void resetDeleteNotifyJob​(java.lang.Long jobID)
                   throws ManifoldCFException
        Reset a job that is delete notifying back to "ready for delete notify" state.
        Parameters:
        jobID - is the job id.
        Throws:
        ManifoldCFException

      • resetStartupJob

        void resetStartupJob​(java.lang.Long jobID)
              throws ManifoldCFException
        Reset a starting job back to "ready for startup" state.
        Parameters:
        jobID - is the job id.
        Throws:
        ManifoldCFException

      • prepareJobScan

        void prepareJobScan​(java.lang.Long jobID,
                    java.lang.String[] legalLinkTypes,
                    int hopcountMethod,
                    int connectorModel,
                    boolean continuousJob,
                    boolean fromBeginningOfTime,
                    boolean requestMinimum)
             throws ManifoldCFException
        Prepare a job to be run. This method is called regardless of the details of the job; what differs is only the flags that are passed in. The code inside will determine the appropriate procedures. (This method replaces prepareFullScan() and prepareIncrementalScan(). )
        Parameters:
        jobID - is the job id.
        legalLinkTypes - are the link types allowed for the job.
        hopcountMethod - describes how to handle deletions for hopcount purposes.
        connectorModel - is the model used by the connector for the job.
        continuousJob - is true if the job is a continuous one.
        fromBeginningOfTime - is true if the job is running starting from time 0.
        requestMinimum - is true if the minimal amount of work is requested for the job run.
        Throws:
        ManifoldCFException

      • noteJobDeleteStarted

        void noteJobDeleteStarted​(java.lang.Long jobID,
                          long startTime)
                   throws ManifoldCFException
        Note job delete started.
        Parameters:
        jobID - is the job id.
        startTime - is the job start time.
        Throws:
        ManifoldCFException

      • noteJobStarted

        void noteJobStarted​(java.lang.Long jobID,
                    long startTime,
                    java.lang.String seedingVersion)
             throws ManifoldCFException
        Note job started.
        Parameters:
        jobID - is the job id.
        startTime - is the job start time.
        seedingVersion - is the seeding version to record with the job start.
        Throws:
        ManifoldCFException

      • noteJobSeeded

        void noteJobSeeded​(java.lang.Long jobID,
                   java.lang.String seedingVersion)
            throws ManifoldCFException
        Note job seeded.
        Parameters:
        jobID - is the job id.
        seedingVersion - is the seeding version string to record.
        Throws:
        ManifoldCFException

      • noteConnectorDeregistration

        void noteConnectorDeregistration​(java.lang.String[] connectionNames)
                          throws ManifoldCFException
        Note the deregistration of a connector used by the specified connections. This method will be called when the connector is deregistered. Jobs that use these connections must therefore enter appropriate states.
        Parameters:
        connectionNames - is the set of connection names.
        Throws:
        ManifoldCFException

      • noteConnectorRegistration

        void noteConnectorRegistration​(java.lang.String[] connectionNames)
                        throws ManifoldCFException
        Note the registration of a connector used by the specified connections. This method will be called when a connector is registered, on which the specified connections depend.
        Parameters:
        connectionNames - is the set of connection names.
        Throws:
        ManifoldCFException

      • noteNotificationConnectorDeregistration

        void noteNotificationConnectorDeregistration​(java.lang.String[] connectionNames)
                                      throws ManifoldCFException
        Note the deregistration of a notification connector used by the specified connections. This method will be called when the connector is deregistered. Jobs that use these connections must therefore enter appropriate states.
        Parameters:
        connectionNames - is the set of connection names.
        Throws:
        ManifoldCFException

      • noteNotificationConnectorRegistration

        void noteNotificationConnectorRegistration​(java.lang.String[] connectionNames)
                                    throws ManifoldCFException
        Note the registration of a notification connector used by the specified connections. This method will be called when a connector is registered, on which the specified connections depend.
        Parameters:
        connectionNames - is the set of connection names.
        Throws:
        ManifoldCFException

      • noteConnectionChange

        void noteConnectionChange​(java.lang.String connectionName)
                   throws ManifoldCFException
        Note a change in connection configuration. This method will be called whenever a connection's configuration is modified, or when an external repository change is signalled.
        Throws:
        ManifoldCFException

      • noteNotificationConnectionChange

        void noteNotificationConnectionChange​(java.lang.String connectionName)
                               throws ManifoldCFException
        Note a change in notification connection configuration. This method will be called whenever a connection's configuration is modified, or when an external repository change is signalled.
        Throws:
        ManifoldCFException

      • noteOutputConnectorDeregistration

        void noteOutputConnectorDeregistration​(java.lang.String[] connectionNames)
                                throws ManifoldCFException
        Note the deregistration of an output connector used by the specified connections. This method will be called when the connector is deregistered. Jobs that use these connections must therefore enter appropriate states.
        Parameters:
        connectionNames - is the set of connection names.
        Throws:
        ManifoldCFException

      • noteOutputConnectorRegistration

        void noteOutputConnectorRegistration​(java.lang.String[] connectionNames)
                              throws ManifoldCFException
        Note the registration of an output connector used by the specified connections. This method will be called when a connector is registered, on which the specified connections depend.
        Parameters:
        connectionNames - is the set of connection names.
        Throws:
        ManifoldCFException

      • noteOutputConnectionChange

        void noteOutputConnectionChange​(java.lang.String connectionName)
                         throws ManifoldCFException
        Note a change in output connection configuration. This method will be called whenever a connection's configuration is modified, or when an external output target change is signalled.
        Throws:
        ManifoldCFException

      • noteTransformationConnectorDeregistration

        void noteTransformationConnectorDeregistration​(java.lang.String[] connectionNames)
                                        throws ManifoldCFException
        Note the deregistration of a transformation connector used by the specified connections. This method will be called when the connector is deregistered. Jobs that use these connections must therefore enter appropriate states.
        Parameters:
        connectionNames - is the set of connection names.
        Throws:
        ManifoldCFException

      • noteTransformationConnectorRegistration

        void noteTransformationConnectorRegistration​(java.lang.String[] connectionNames)
                                      throws ManifoldCFException
        Note the registration of a transformation connector used by the specified connections. This method will be called when a connector is registered, on which the specified connections depend.
        Parameters:
        connectionNames - is the set of connection names.
        Throws:
        ManifoldCFException

      • noteTransformationConnectionChange

        void noteTransformationConnectionChange​(java.lang.String connectionName)
                                 throws ManifoldCFException
        Note a change in transformation connection configuration. This method will be called whenever a connection's configuration is modified.
        Throws:
        ManifoldCFException

      • deleteJobsReadyForDelete

        void deleteJobsReadyForDelete()
                       throws ManifoldCFException
        Delete jobs in need of being deleted (which are marked "ready for delete"). This method is meant to be called periodically to perform delete processing on jobs.
        Throws:
        ManifoldCFException

      • getNextDeletableDocuments

        DocumentDescription[] getNextDeletableDocuments​(java.lang.String processID,
                                                int n,
                                                long currentTime)
                                         throws ManifoldCFException
        Get list of deletable document descriptions. This list will take into account multiple jobs that may own the same document.
        Parameters:
        processID - is the current process ID.
        n - is the maximum number of documents to return.
        currentTime - is the current time; some fetches do not occur until a specific time.
        Returns:
        the document descriptions for these documents.
        Throws:
        ManifoldCFException

      • getNextCleanableDocuments

        DocumentSetAndFlags getNextCleanableDocuments​(java.lang.String processID,
                                              int n,
                                              long currentTime)
                                       throws ManifoldCFException
        Get list of cleanable document descriptions. This list will take into account multiple jobs that may own the same document.
        Parameters:
        processID - is the current process ID.
        n - is the maximum number of documents to return.
        currentTime - is the current time; some fetches do not occur until a specific time.
        Returns:
        the document descriptions for these documents.
        Throws:
        ManifoldCFException

      • deleteIngestedDocumentIdentifiers

        void deleteIngestedDocumentIdentifiers​(DocumentDescription[] identifiers)
                                throws ManifoldCFException
        Delete ingested document identifiers (as part of deleting the owning job). The number of identifiers specified is guaranteed to be less than the maxInClauseCount for the database.
        Parameters:
        identifiers - is the set of document identifiers.
        Throws:
        ManifoldCFException

      • errorAbort

        boolean errorAbort​(java.lang.Long jobID,
                   java.lang.String errorText)
            throws ManifoldCFException
        Abort a running job due to a fatal error condition.
        Parameters:
        jobID - is the job to abort.
        errorText - is the error text.
        Returns:
        true if this is the first abort for the job.
        Throws:
        ManifoldCFException

      • finishJobStops

        void finishJobStops​(long timestamp,
                    java.util.List<IJobDescription> modifiedJobs,
                    java.util.List<java.lang.Integer> stopNotificationTypes)
             throws ManifoldCFException
        Complete the sequence that stops jobs, either for abort, pause, or because of a scheduling window. The logic will move the job to its next state (INACTIVE, PAUSED, ACTIVEWAIT), and will record the jobs that have been so modified.
        Parameters:
        timestamp - is the current time in milliseconds since epoch.
        modifiedJobs - is filled in with the set of IJobDescription objects that were stopped.
        stopNotificationTypes - is filled in with the type of stop notification.
        Throws:
        ManifoldCFException

      • finishJobResumes

        void finishJobResumes​(long timestamp,
                      java.util.List<IJobDescription> modifiedJobs)
               throws ManifoldCFException
        Complete the sequence that resumes jobs, either from a pause or from a scheduling window wait. The logic will restore the job to an active state (many possibilities depending on connector status), and will record the jobs that have been so modified.
        Parameters:
        timestamp - is the current time in milliseconds since epoch.
        modifiedJobs - is filled in with the set of IJobDescription objects that were resumed.
        Throws:
        ManifoldCFException

      • resetJobs

        void resetJobs​(long currentTime,
               java.util.List<IJobDescription> resetJobs)
        throws ManifoldCFException
        Reset eligible jobs either back to the "inactive" state, or make them active again. The latter will occur if the cleanup phase of the job generated more pending documents. This method is used to pick up all jobs in the shutting down state whose purgatory or being-cleaned records have been all processed.
        Parameters:
        currentTime - is the current time in milliseconds since epoch.
        resetJobs - is filled in with the set of IJobDescription objects that were reset.
        Throws:
        ManifoldCFException

      • getStatus

        JobStatus getStatus​(java.lang.Long jobID,
                    boolean includeCounts)
             throws ManifoldCFException
        Get the status of a job.
        Parameters:
        jobID - is the job ID.
        includeCounts - is true if document counts should be included.
        Returns:
        the status object for the specified job.
        Throws:
        ManifoldCFException

      • getAllStatus

        JobStatus[] getAllStatus​(boolean includeCounts)
                  throws ManifoldCFException
        Get a list of all jobs, and their status information.
        Parameters:
        includeCounts - is true if document counts should be included.
        Returns:
        an ordered array of job status objects.
        Throws:
        ManifoldCFException

      • getRunningJobs

        JobStatus[] getRunningJobs​(boolean includeCounts)
                    throws ManifoldCFException
        Get a list of running jobs. This is for status reporting.
        Parameters:
        includeCounts - is true if document counts should be included.
        Returns:
        an array of the job status objects.
        Throws:
        ManifoldCFException

      • getFinishedJobs

        JobStatus[] getFinishedJobs​(boolean includeCounts)
                     throws ManifoldCFException
        Get a list of completed jobs, and their statistics.
        Parameters:
        includeCounts - is true if document counts should be included.
        Returns:
        an array of the job status objects.
        Throws:
        ManifoldCFException

      • getStatus

        JobStatus getStatus​(java.lang.Long jobID,
                    boolean includeCounts,
                    int maxCount)
             throws ManifoldCFException
        Get the status of a job.
        Parameters:
        jobID - is the job ID.
        includeCounts - is true if document counts should be included.
        maxCount - is the maximum number of documents we want to count for each status.
        Returns:
        the status object for the specified job.
        Throws:
        ManifoldCFException

      • getAllStatus

        JobStatus[] getAllStatus​(boolean includeCounts,
                         int maxCount)
                  throws ManifoldCFException
        Get a list of all jobs, and their status information.
        Parameters:
        includeCounts - is true if document counts should be included.
        maxCount - is the maximum number of documents we want to count for each status.
        Returns:
        an ordered array of job status objects.
        Throws:
        ManifoldCFException

      • getRunningJobs

        JobStatus[] getRunningJobs​(boolean includeCounts,
                           int maxCount)
                    throws ManifoldCFException
        Get a list of running jobs. This is for status reporting.
        Parameters:
        includeCounts - is true if document counts should be included.
        maxCount - is the maximum number of documents we want to count for each status.
        Returns:
        an array of the job status objects.
        Throws:
        ManifoldCFException

      • getFinishedJobs

        JobStatus[] getFinishedJobs​(boolean includeCounts,
                            int maxCount)
                     throws ManifoldCFException
        Get a list of completed jobs, and their statistics.
        Parameters:
        includeCounts - is true if document counts should be included.
        maxCount - is the maximum number of documents we want to count for each status.
        Returns:
        an array of the job status objects.
        Throws:
        ManifoldCFException

      • genDocumentStatus

        IResultSet genDocumentStatus​(java.lang.String connectionName,
                             StatusFilterCriteria filterCriteria,
                             SortOrder sortOrder,
                             int startRow,
                             int rowCount)
                      throws ManifoldCFException
        Run a 'document status' report.
        Parameters:
        connectionName - is the name of the connection.
        filterCriteria - are the criteria used to limit the records considered for the report.
        sortOrder - is the specified sort order of the final report.
        startRow - is the first row to include.
        rowCount - is the number of rows to include.
        Returns:
        the results, with the following columns: identifier, job, state, status, scheduled, action, retrycount, retrylimit. The "scheduled" column and the "retrylimit" column are long values representing a time; all other values will be user-friendly strings.
        Throws:
        ManifoldCFException

      • genQueueStatus

        IResultSet genQueueStatus​(java.lang.String connectionName,
                          StatusFilterCriteria filterCriteria,
                          SortOrder sortOrder,
                          BucketDescription idBucketDescription,
                          int startRow,
                          int rowCount)
                   throws ManifoldCFException
        Run a 'queue status' report.
        Parameters:
        connectionName - is the name of the connection.
        filterCriteria - are the criteria used to limit the records considered for the report.
        sortOrder - is the specified sort order of the final report.
        idBucketDescription - is the bucket description for generating the identifier class.
        startRow - is the first row to include.
        rowCount - is the number of rows to include.
        Returns:
        the results, with the following columns: idbucket, inactive, processing, expiring, deleting, processready, expireready, processwaiting, expirewaiting
        Throws:
        ManifoldCFException