Package org.apache.manifoldcf.crawler.repository

Class RepositoryConnectionManager

  • All Implemented Interfaces:
    IRepositoryConnectionManager
    public class RepositoryConnectionManager
extends BaseTable
implements IRepositoryConnectionManager
    This class is the manager of the repository connection description. Inside, multiple database tables are managed, with appropriate caching. Note well: The database handle is instantiated here using the DBInterfaceFactory. This is acceptable because the actual database that this table is located in is fixed.

    repoconnections
    FieldTypeDescription        
    connectionnameVARCHAR(32)Primary Key
    descriptionVARCHAR(255)
    classnameVARCHAR(255)
    groupnameVARCHAR(32)
    maxcountBIGINT
    configxmlLONGTEXT


    • Method Detail

      • recordHistory

        public void recordHistory​(java.lang.String connectionName,
                          java.lang.Long startTime,
                          java.lang.String activityType,
                          java.lang.Long dataSize,
                          java.lang.String entityIdentifier,
                          java.lang.String resultCode,
                          java.lang.String resultDescription,
                          java.lang.String[] childIdentifiers)
                   throws ManifoldCFException
        Record time-stamped information about the activity of the connection. This information can originate from either the connector or from the framework. The reason it is here is that it is viewed as 'belonging' to an individual connection, and is segregated accordingly.
        Specified by:
        recordHistory in interface IRepositoryConnectionManager
        Parameters:
        connectionName - is the connection to which the record belongs. If the connection is deleted, the corresponding records will also be deleted. Cannot be null.
        startTime - is either null or the time since the start of epoch in milliseconds (Jan 1, 1970). Every activity has an associated time; the startTime field records when the activity began. A null value indicates that the start time and the finishing time are the same.
        activityType - is a string which is fully interpretable only in the context of the connector involved, which is used to categorize what kind of activity is being recorded. For example, a web connector might record a "fetch document" activity, while the framework might record "ingest document", "job start", "job finish", "job abort", etc. Cannot be null.
        dataSize - is the number of bytes of data involved in the activity, or null if not applicable.
        entityIdentifier - is a (possibly long) string which identifies the object involved in the history record. The interpretation of this field will differ from connector to connector. May be null.
        resultCode - contains a terse description of the result of the activity. The description is limited in size to 255 characters, and can be interpreted only in the context of the current connector. May be null.
        resultDescription - is a (possibly long) human-readable string which adds detail, if required, to the result described in the resultCode field. This field is not meant to be queried on. May be null.
        childIdentifiers - is a set of child entity identifiers associated with this activity. May be null.
        Throws:
        ManifoldCFException

      • countHistoryRows

        public long countHistoryRows​(java.lang.String connectionName,
                             FilterCriteria criteria)
                      throws ManifoldCFException
        Count the number of rows specified by a given set of criteria. This can be used to make decisions as to whether a query based on those rows will complete in an acceptable amount of time.
        Specified by:
        countHistoryRows in interface IRepositoryConnectionManager
        Parameters:
        connectionName - is the name of the connection.
        criteria - is the filtering criteria, which selects the records of interest.
        Returns:
        the number of rows included by the criteria.
        Throws:
        ManifoldCFException

      • genHistorySimple

        public IResultSet genHistorySimple​(java.lang.String connectionName,
                                   FilterCriteria criteria,
                                   SortOrder sort,
                                   int startRow,
                                   int maxRowCount)
                            throws ManifoldCFException
        Generate a report, listing the start time, elapsed time, result code and description, number of bytes, and entity identifier. The records selected for this report are based on the filtering criteria object passed into this method. The record order is based on the sorting criteria object passed into this method. The resultset returned should have the following columns: "starttime","elapsedtime","resultcode","resultdesc","bytes","identifier".
        Specified by:
        genHistorySimple in interface IRepositoryConnectionManager
        Parameters:
        connectionName - is the name of the connection.
        criteria - is the filtering criteria, which selects the records of interest.
        sort - is the sorting order, which can specify sort based on the result columns.
        startRow - is the first row to include (beginning with 0)
        maxRowCount - is the maximum number of rows to include.
        Throws:
        ManifoldCFException

      • genHistoryActivityCount

        public IResultSet genHistoryActivityCount​(java.lang.String connectionName,
                                          FilterCriteria criteria,
                                          SortOrder sort,
                                          BucketDescription idBucket,
                                          long interval,
                                          int startRow,
                                          int maxRowCount)
                                   throws ManifoldCFException
        Generate a report, listing the start time, activity count, and identifier bucket, given a time slice (interval) size. The records selected for this report are based on the filtering criteria object passed into this method. The record order is based on the sorting criteria object passed into this method. The identifier bucket description is specified by the bucket description object. The resultset returned should have the following columns: "starttime","endtime","activitycount","idbucket".
        Specified by:
        genHistoryActivityCount in interface IRepositoryConnectionManager
        Parameters:
        connectionName - is the name of the connection.
        criteria - is the filtering criteria, which selects the records of interest.
        sort - is the sorting order, which can specify sort based on the result columns.
        idBucket - is the description of the bucket based on processed entity identifiers.
        interval - is the time interval, in milliseconds, to locate. There will be one row in the resultset for each distinct idBucket value, and the returned activity count will the maximum found over the specified interval size.
        startRow - is the first row to include (beginning with 0)
        maxRowCount - is the maximum number of rows to include.
        Throws:
        ManifoldCFException

      • genHistoryByteCount

        public IResultSet genHistoryByteCount​(java.lang.String connectionName,
                                      FilterCriteria criteria,
                                      SortOrder sort,
                                      BucketDescription idBucket,
                                      long interval,
                                      int startRow,
                                      int maxRowCount)
                               throws ManifoldCFException
        Generate a report, listing the start time, bytes processed, and identifier bucket, given a time slice (interval) size. The records selected for this report are based on the filtering criteria object passed into this method. The record order is based on the sorting criteria object passed into this method. The identifier bucket description is specified by the bucket description object. The resultset returned should have the following columns: "starttime","endtime","bytecount","idbucket".
        Specified by:
        genHistoryByteCount in interface IRepositoryConnectionManager
        Parameters:
        connectionName - is the name of the connection.
        criteria - is the filtering criteria, which selects the records of interest.
        sort - is the sorting order, which can specify sort based on the result columns.
        idBucket - is the description of the bucket based on processed entity identifiers.
        interval - is the time interval, in milliseconds, to locate. There will be one row in the resultset for each distinct idBucket value, and the returned activity count will the maximum found over the specified interval size.
        startRow - is the first row to include (beginning with 0)
        maxRowCount - is the maximum number of rows to include.
        Throws:
        ManifoldCFException

      • genHistoryResultCodes

        public IResultSet genHistoryResultCodes​(java.lang.String connectionName,
                                        FilterCriteria criteria,
                                        SortOrder sort,
                                        BucketDescription resultCodeBucket,
                                        BucketDescription idBucket,
                                        int startRow,
                                        int maxRowCount)
                                 throws ManifoldCFException
        Generate a report, listing the result bucket and identifier bucket. The records selected for this report are based on the filtering criteria object passed into this method. The record order is based on the sorting criteria object passed into this method. The result code bucket description is specified by a bucket description object. The identifier bucket description is specified by a bucket description object. The resultset returned should have the following columns: "resultcodebucket","idbucket".
        Specified by:
        genHistoryResultCodes in interface IRepositoryConnectionManager
        Parameters:
        connectionName - is the name of the connection.
        criteria - is the filtering criteria, which selects the records of interest.
        sort - is the sorting order, which can specify sort based on the result columns.
        resultCodeBucket - is the description of the bucket based on processed result codes.
        idBucket - is the description of the bucket based on processed entity identifiers.
        startRow - is the first row to include (beginning with 0)
        maxRowCount - is the maximum number of rows to include.
        Throws:
        ManifoldCFException

      • getRepositoryConnectionsKey

        protected static java.lang.String getRepositoryConnectionsKey()
        Construct a key which represents the general list of repository connectors.
        Returns:
        the cache key.

      • getRepositoryConnectionKey

        protected static java.lang.String getRepositoryConnectionKey​(java.lang.String connectionName)
        Construct a key which represents an individual repository connection.
        Parameters:
        connectionName - is the name of the connector.
        Returns:
        the cache key.

      • getRepositoryConnectionsMultiple

        protected RepositoryConnection[] getRepositoryConnectionsMultiple​(java.lang.String[] connectionNames)
                                                           throws ManifoldCFException
        Fetch multiple repository connections at a single time.
        Parameters:
        connectionNames - are a list of connection names.
        Returns:
        the corresponding repository connection objects.
        Throws:
        ManifoldCFException

      • maxClauseGetRepositoryConnectionsChunk

        protected int maxClauseGetRepositoryConnectionsChunk()
        Calculate how many repository connections to get at once.

      • getRepositoryConnectionsChunk

        protected void getRepositoryConnectionsChunk​(RepositoryConnection[] rval,
                                             java.util.Map returnIndex,
                                             java.util.ArrayList params)
                                      throws ManifoldCFException
        Read a chunk of repository connections.
        Parameters:
        rval - is the place to put the read policies.
        returnIndex - is a map from the object id (resource id) and the rval index.
        params - is the set of parameters.
        Throws:
        ManifoldCFException