Package org.apache.manifoldcf.crawler.connectors.sharedrive

Class SharedDriveConnector

  • All Implemented Interfaces:
    org.apache.manifoldcf.core.interfaces.IConnector, org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector
    public class SharedDriveConnector
extends org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector
    This is the "repository connector" for a smb/cifs shared drive file system. It's a relative of the share crawler, and should have comparable basic functionality.

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      protected static void addSecuritySet​(java.lang.StringBuilder description, boolean enabled, java.lang.String[] allowTokens, java.lang.String[] denyTokens)  
      java.lang.String addSeedDocuments​(org.apache.manifoldcf.crawler.interfaces.ISeedingActivity activities, org.apache.manifoldcf.core.interfaces.Specification spec, java.lang.String lastSeedVersion, long seedTime, int jobMode)
      Queue "seed" documents.
      java.lang.String check()
      Check status of connection.
      protected boolean checkInclude​(boolean isDirectory, java.lang.String fileName, org.apache.manifoldcf.core.interfaces.Specification documentSpecification)
      Check if a file or directory should be included, given a document specification.
      protected static boolean checkIncludeFile​(long fileLength, java.lang.String fileName, org.apache.manifoldcf.core.interfaces.Specification documentSpecification, org.apache.manifoldcf.crawler.interfaces.IFingerprintActivity activities)
      Check if a file's stats are OK for inclusion.
      protected boolean checkIngest​(java.io.File localFile, java.lang.String fileName, org.apache.manifoldcf.core.interfaces.Specification documentSpecification, org.apache.manifoldcf.crawler.interfaces.IFingerprintActivity activities)
      Check if a file should be ingested, given a document specification and a local copy of the file.
      protected static boolean checkMatch​(java.lang.String sourceMatch, int sourceIndex, java.lang.String match)
      Check a match between two strings with wildcards.
      protected boolean checkNeedFileData​(java.lang.String fileName, org.apache.manifoldcf.core.interfaces.Specification documentSpecification)
      Check to see whether we need the contents of the file for anything.
      void connect​(org.apache.manifoldcf.core.interfaces.ConfigParams configParameters)
      Connect.
      protected void convertACEs​(java.util.List<java.lang.String> allowList, java.util.List<java.lang.String> denyList, jcifs.ACE[] aces)  
      protected static java.lang.String convertToURI​(java.lang.String documentIdentifier, MatchMap fileMap, MatchMap uriMap)
      Convert a document identifier to a URI.
      void disconnect()
      Close the connection.
      protected static boolean equivalentIOExceptions​(java.io.IOException e1, java.io.IOException e2)
      Check if two IOExceptions are equivalent
      protected static boolean equivalentSmbExceptions​(jcifs.smb.SmbException e1, jcifs.smb.SmbException e2)
      Check if two SmbExceptions are equivalent
      protected static boolean fileExists​(jcifs.smb.SmbFile file)
      Check for file/directory existence
      protected static boolean fileIsDirectory​(jcifs.smb.SmbFile file)
      Check if file is a directory
      protected static long fileLastModified​(jcifs.smb.SmbFile file)
      Get last modified date for file
      protected static long fileLength​(jcifs.smb.SmbFile file)
      Get file length
      protected static jcifs.smb.SmbFile[] fileListFiles​(jcifs.smb.SmbFile file, jcifs.smb.SmbFileFilter filter)
      List files
      java.lang.String[] getActivitiesList()
      Return the list of activities that this connector supports (i.e.
      java.lang.String[] getBinNames​(java.lang.String documentIdentifier)
      Get the bin name string for a document identifier.
      java.lang.String[] getChildFolderNames​(java.lang.String folder)
      given a smb uri, return all children directories
      protected static java.lang.String getFileCanonicalPath​(jcifs.smb.SmbFile file)
      Get canonical path
      protected static java.io.InputStream getFileInputStream​(jcifs.smb.SmbFile file)
      Get input stream for file
      protected static jcifs.ACE[] getFileSecurity​(jcifs.smb.SmbFile file, boolean useSIDs)
      Get file security
      protected boolean getFileSecuritySet​(java.util.List<java.lang.String> allowList, java.util.List<java.lang.String> denyList, jcifs.smb.SmbFile file, java.lang.String[] forced)  
      protected static jcifs.ACE[] getFileShareSecurity​(jcifs.smb.SmbFile file, boolean useSIDs)
      Get share security
      protected boolean getFileShareSecuritySet​(java.util.List<java.lang.String> allowList, java.util.List<java.lang.String> denyList, jcifs.smb.SmbFile file, java.lang.String[] forced)  
      protected static int getFileType​(jcifs.smb.SmbFile file)
      Get file type
      protected static java.lang.String[] getForcedAcls​(org.apache.manifoldcf.core.interfaces.Specification spec)
      Grab forced acl out of document specification.
      protected static java.lang.String[] getForcedParentFolderAcls​(org.apache.manifoldcf.core.interfaces.Specification spec)
      Grab forced parent folder acls out of document specification.
      protected static java.lang.String[] getForcedShareAcls​(org.apache.manifoldcf.core.interfaces.Specification spec)
      Grab forced share acls out of document specification.
      protected void getSession()
      Establish a "session".
      jcifs.smb.SmbFile[] getShareNames​(java.lang.String serverURI)
      given a server uri, return all shares
      protected static void handleIOException​(java.lang.String documentIdentifier, java.io.IOException e)  
      protected static java.lang.String mapExtensionToMimeType​(java.lang.String fileName)
      Map an extension to a mime type
      protected java.lang.String mapToIdentifier​(java.lang.String path)
      Map a "path" specification to a full identifier.
      protected static int matchSubPath​(java.lang.String subPath, java.lang.String fullPath)
      Match a sub-path.
      void outputConfigurationBody​(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext, org.apache.manifoldcf.core.interfaces.IHTTPOutput out, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.ConfigParams parameters, java.lang.String tabName)
      Output the configuration body section.
      void outputConfigurationHeader​(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext, org.apache.manifoldcf.core.interfaces.IHTTPOutput out, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.ConfigParams parameters, java.util.List<java.lang.String> tabsArray)
      Output the configuration header section.
      void outputSpecificationBody​(org.apache.manifoldcf.core.interfaces.IHTTPOutput out, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.Specification ds, int connectionSequenceNumber, int actualSequenceNumber, java.lang.String tabName)
      Output the specification body section.
      void outputSpecificationHeader​(org.apache.manifoldcf.core.interfaces.IHTTPOutput out, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.Specification ds, int connectionSequenceNumber, java.util.List<java.lang.String> tabsArray)
      Output the specification header section.
      protected static boolean processCheck​(boolean caseSensitive, java.lang.String sourceMatch, int sourceIndex, java.lang.String match, int matchIndex)
      Recursive worker method for checkMatch.
      java.lang.String processConfigurationPost​(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext, org.apache.manifoldcf.core.interfaces.IPostParameters variableContext, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.ConfigParams parameters)
      Process a configuration post.
      void processDocuments​(java.lang.String[] documentIdentifiers, org.apache.manifoldcf.crawler.interfaces.IExistingVersions statuses, org.apache.manifoldcf.core.interfaces.Specification spec, org.apache.manifoldcf.crawler.interfaces.IProcessActivity activities, int jobMode, boolean usesDefaultAuthority)
      Process a set of documents.
      protected static void processSMBException​(jcifs.smb.SmbException se, java.lang.String documentIdentifier, java.lang.String activity, java.lang.String operation)  
      java.lang.String processSpecificationPost​(org.apache.manifoldcf.core.interfaces.IPostParameters variableContext, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.Specification ds, int connectionSequenceNumber)
      Process a specification post.
      boolean requestInfo​(org.apache.manifoldcf.core.interfaces.Configuration output, java.lang.String command)
      Request arbitrary connector information.
      protected static void setDocumentSecurity​(org.apache.manifoldcf.agents.interfaces.RepositoryDocument rd, java.lang.String[] shareAllow, java.lang.String[] shareDeny, java.lang.String[] parentAllow, java.lang.String[] parentDeny, java.lang.String[] allow, java.lang.String[] deny)  
      protected static void setPathMetadata​(org.apache.manifoldcf.agents.interfaces.RepositoryDocument rd, java.lang.String pathAttributeName, java.lang.String pathAttributeValue)  
      void setThreadContext​(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext)
      Set thread context.
      java.lang.String validateFolderName​(java.lang.String folder)
      Given a folder path, determine if the folder is in fact legal and accessible (and is a folder).
      void viewConfiguration​(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext, org.apache.manifoldcf.core.interfaces.IHTTPOutput out, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.ConfigParams parameters)
      View configuration.
      void viewSpecification​(org.apache.manifoldcf.core.interfaces.IHTTPOutput out, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.Specification ds, int connectionSequenceNumber)
      View specification.
      protected boolean wouldFileBeIncluded​(java.lang.String fileName, org.apache.manifoldcf.core.interfaces.Specification documentSpecification, boolean pretendIndexable)
      Pretend that a file is either indexable or not, and return whether or not it would be ingested.

      • Methods inherited from class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

        getConnectorModel, getFormCheckJavascriptMethodName, getFormPresaveCheckJavascriptMethodName, getMaxDocumentRequest, getRelationshipTypes

      • Methods inherited from class org.apache.manifoldcf.core.connector.BaseConnector

        clearThreadContext, deinstall, getConfiguration, install, isConnected, outputConfigurationBody, outputConfigurationHeader, outputConfigurationHeader, pack, packFixedList, packList, packList, poll, processConfigurationPost, unpack, unpackFixedList, unpackList, viewConfiguration

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

      • Methods inherited from interface org.apache.manifoldcf.core.interfaces.IConnector

        clearThreadContext, deinstall, getConfiguration, install, isConnected, poll

    • Constructor Detail

      • SharedDriveConnector

        public SharedDriveConnector()
        Constructor.

    • Method Detail

      • setThreadContext

        public void setThreadContext​(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext)
                      throws org.apache.manifoldcf.core.interfaces.ManifoldCFException
        Set thread context. Use the opportunity to set the system properties we'll need.
        Specified by:
        setThreadContext in interface org.apache.manifoldcf.core.interfaces.IConnector
        Overrides:
        setThreadContext in class org.apache.manifoldcf.core.connector.BaseConnector
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException

      • getSession

        protected void getSession()
                   throws org.apache.manifoldcf.core.interfaces.ManifoldCFException
        Establish a "session". In the case of the jcifs connector, this just builds the appropriate smbconnectionPath string, and does the necessary checks.
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException

      • getActivitiesList

        public java.lang.String[] getActivitiesList()
        Return the list of activities that this connector supports (i.e. writes into the log).
        Specified by:
        getActivitiesList in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector
        Overrides:
        getActivitiesList in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector
        Returns:
        the list.

      • disconnect

        public void disconnect()
                throws org.apache.manifoldcf.core.interfaces.ManifoldCFException
        Close the connection. Call this before discarding the repository connector.
        Specified by:
        disconnect in interface org.apache.manifoldcf.core.interfaces.IConnector
        Overrides:
        disconnect in class org.apache.manifoldcf.core.connector.BaseConnector
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException

      • connect

        public void connect​(org.apache.manifoldcf.core.interfaces.ConfigParams configParameters)
        Connect.
        Specified by:
        connect in interface org.apache.manifoldcf.core.interfaces.IConnector
        Overrides:
        connect in class org.apache.manifoldcf.core.connector.BaseConnector
        Parameters:
        configParameters - is the set of configuration parameters, which in this case describe the root directory.

      • getBinNames

        public java.lang.String[] getBinNames​(java.lang.String documentIdentifier)
        Get the bin name string for a document identifier. The bin name describes the queue to which the document will be assigned for throttling purposes. Throttling controls the rate at which items in a given queue are fetched; it does not say anything about the overall fetch rate, which may operate on multiple queues or bins. For example, if you implement a web crawler, a good choice of bin name would be the server name, since that is likely to correspond to a real resource that will need real throttle protection.
        Specified by:
        getBinNames in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector
        Overrides:
        getBinNames in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector
        Parameters:
        documentIdentifier - is the document identifier.
        Returns:
        the bin name.

      • convertToURI

        protected static java.lang.String convertToURI​(java.lang.String documentIdentifier,
                                               MatchMap fileMap,
                                               MatchMap uriMap)
                                        throws org.apache.manifoldcf.core.interfaces.ManifoldCFException
        Convert a document identifier to a URI. The URI is the URI that will be the unique key from the search index, and will be presented to the user as part of the search results.
        Parameters:
        documentIdentifier - is the document identifier.
        Returns:
        the document uri.
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException

      • requestInfo

        public boolean requestInfo​(org.apache.manifoldcf.core.interfaces.Configuration output,
                           java.lang.String command)
                    throws org.apache.manifoldcf.core.interfaces.ManifoldCFException
        Request arbitrary connector information. This method is called directly from the API in order to allow API users to perform any one of several connector-specific queries.
        Specified by:
        requestInfo in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector
        Overrides:
        requestInfo in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector
        Parameters:
        output - is the response object, to be filled in by this method.
        command - is the command, which is taken directly from the API request.
        Returns:
        true if the resource is found, false if not. In either case, output may be filled in.
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException

      • addSeedDocuments

        public java.lang.String addSeedDocuments​(org.apache.manifoldcf.crawler.interfaces.ISeedingActivity activities,
                                         org.apache.manifoldcf.core.interfaces.Specification spec,
                                         java.lang.String lastSeedVersion,
                                         long seedTime,
                                         int jobMode)
                                  throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                         org.apache.manifoldcf.agents.interfaces.ServiceInterruption
        Queue "seed" documents. Seed documents are the starting places for crawling activity. Documents are seeded when this method calls appropriate methods in the passed in ISeedingActivity object. This method can choose to find repository changes that happen only during the specified time interval. The seeds recorded by this method will be viewed by the framework based on what the getConnectorModel() method returns. It is not a big problem if the connector chooses to create more seeds than are strictly necessary; it is merely a question of overall work required. The end time and seeding version string passed to this method may be interpreted for greatest efficiency. For continuous crawling jobs, this method will be called once, when the job starts, and at various periodic intervals as the job executes. When a job's specification is changed, the framework automatically resets the seeding version string to null. The seeding version string may also be set to null on each job run, depending on the connector model returned by getConnectorModel(). Note that it is always ok to send MORE documents rather than less to this method. The connector will be connected before this method can be called.
        Specified by:
        addSeedDocuments in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector
        Overrides:
        addSeedDocuments in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector
        Parameters:
        activities - is the interface this method should use to perform whatever framework actions are desired.
        spec - is a document specification (that comes from the job).
        seedTime - is the end of the time range of documents to consider, exclusive.
        lastSeedVersion - is the last seeding version string for this job, or null if the job has no previous seeding version string.
        jobMode - is an integer describing how the job is being run, whether continuous or once-only.
        Returns:
        an updated seeding version string, to be stored with the job.
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException
        org.apache.manifoldcf.agents.interfaces.ServiceInterruption

      • processDocuments

        public void processDocuments​(java.lang.String[] documentIdentifiers,
                             org.apache.manifoldcf.crawler.interfaces.IExistingVersions statuses,
                             org.apache.manifoldcf.core.interfaces.Specification spec,
                             org.apache.manifoldcf.crawler.interfaces.IProcessActivity activities,
                             int jobMode,
                             boolean usesDefaultAuthority)
                      throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                             org.apache.manifoldcf.agents.interfaces.ServiceInterruption
        Process a set of documents. This is the method that should cause each document to be fetched, processed, and the results either added to the queue of documents for the current job, and/or entered into the incremental ingestion manager. The document specification allows this class to filter what is done based on the job. The connector will be connected before this method can be called.
        Specified by:
        processDocuments in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector
        Overrides:
        processDocuments in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector
        Parameters:
        documentIdentifiers - is the set of document identifiers to process.
        statuses - are the currently-stored document versions for each document in the set of document identifiers passed in above.
        activities - is the interface this method should use to queue up new document references and ingest documents.
        jobMode - is an integer describing how the job is being run, whether continuous or once-only.
        usesDefaultAuthority - will be true only if the authority in use for these documents is the default one.
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException
        org.apache.manifoldcf.agents.interfaces.ServiceInterruption

      • handleIOException

        protected static void handleIOException​(java.lang.String documentIdentifier,
                                        java.io.IOException e)
                                 throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                        org.apache.manifoldcf.agents.interfaces.ServiceInterruption
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException
        org.apache.manifoldcf.agents.interfaces.ServiceInterruption

      • mapExtensionToMimeType

        protected static java.lang.String mapExtensionToMimeType​(java.lang.String fileName)
        Map an extension to a mime type

      • addSecuritySet

        protected static void addSecuritySet​(java.lang.StringBuilder description,
                                     boolean enabled,
                                     java.lang.String[] allowTokens,
                                     java.lang.String[] denyTokens)

      • getFileSecuritySet

        protected boolean getFileSecuritySet​(java.util.List<java.lang.String> allowList,
                                     java.util.List<java.lang.String> denyList,
                                     jcifs.smb.SmbFile file,
                                     java.lang.String[] forced)
                              throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                     java.io.IOException
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException
        java.io.IOException

      • getFileShareSecuritySet

        protected boolean getFileShareSecuritySet​(java.util.List<java.lang.String> allowList,
                                          java.util.List<java.lang.String> denyList,
                                          jcifs.smb.SmbFile file,
                                          java.lang.String[] forced)
                                   throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                          java.io.IOException
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException
        java.io.IOException

      • convertACEs

        protected void convertACEs​(java.util.List<java.lang.String> allowList,
                           java.util.List<java.lang.String> denyList,
                           jcifs.ACE[] aces)

      • processSMBException

        protected static void processSMBException​(jcifs.smb.SmbException se,
                                          java.lang.String documentIdentifier,
                                          java.lang.String activity,
                                          java.lang.String operation)
                                   throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                          org.apache.manifoldcf.agents.interfaces.ServiceInterruption
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException
        org.apache.manifoldcf.agents.interfaces.ServiceInterruption

      • setDocumentSecurity

        protected static void setDocumentSecurity​(org.apache.manifoldcf.agents.interfaces.RepositoryDocument rd,
                                          java.lang.String[] shareAllow,
                                          java.lang.String[] shareDeny,
                                          java.lang.String[] parentAllow,
                                          java.lang.String[] parentDeny,
                                          java.lang.String[] allow,
                                          java.lang.String[] deny)

      • setPathMetadata

        protected static void setPathMetadata​(org.apache.manifoldcf.agents.interfaces.RepositoryDocument rd,
                                      java.lang.String pathAttributeName,
                                      java.lang.String pathAttributeValue)
                               throws org.apache.manifoldcf.core.interfaces.ManifoldCFException
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException

      • check

        public java.lang.String check()
                       throws org.apache.manifoldcf.core.interfaces.ManifoldCFException
        Check status of connection.
        Specified by:
        check in interface org.apache.manifoldcf.core.interfaces.IConnector
        Overrides:
        check in class org.apache.manifoldcf.core.connector.BaseConnector
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException

      • checkIncludeFile

        protected static boolean checkIncludeFile​(long fileLength,
                                          java.lang.String fileName,
                                          org.apache.manifoldcf.core.interfaces.Specification documentSpecification,
                                          org.apache.manifoldcf.crawler.interfaces.IFingerprintActivity activities)
                                   throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                          org.apache.manifoldcf.agents.interfaces.ServiceInterruption
        Check if a file's stats are OK for inclusion.
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException
        org.apache.manifoldcf.agents.interfaces.ServiceInterruption

      • checkInclude

        protected boolean checkInclude​(boolean isDirectory,
                               java.lang.String fileName,
                               org.apache.manifoldcf.core.interfaces.Specification documentSpecification)
                        throws org.apache.manifoldcf.core.interfaces.ManifoldCFException
        Check if a file or directory should be included, given a document specification.
        Parameters:
        isDirectory - is true if the file is a directory.
        fileName - is the canonical file name.
        documentSpecification - is the specification.
        Returns:
        true if it should be included.
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException

      • wouldFileBeIncluded

        protected boolean wouldFileBeIncluded​(java.lang.String fileName,
                                      org.apache.manifoldcf.core.interfaces.Specification documentSpecification,
                                      boolean pretendIndexable)
                               throws org.apache.manifoldcf.core.interfaces.ManifoldCFException
        Pretend that a file is either indexable or not, and return whether or not it would be ingested. This is only ever called for files.
        Parameters:
        fileName - is the canonical file name.
        documentSpecification - is the specification.
        pretendIndexable - should be set to true if the document's contents would be fingerprinted as "indexable", or false otherwise.
        Returns:
        true if the file would be ingested given the parameters.
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException

      • checkNeedFileData

        protected boolean checkNeedFileData​(java.lang.String fileName,
                                    org.apache.manifoldcf.core.interfaces.Specification documentSpecification)
                             throws org.apache.manifoldcf.core.interfaces.ManifoldCFException
        Check to see whether we need the contents of the file for anything. We do this by assuming that the file is indexable, and assuming that it's not, and seeing if the same thing would happen.
        Parameters:
        fileName - is the name of the file.
        documentSpecification - is the document specification.
        Returns:
        true if the file needs to be fingerprinted.
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException

      • checkIngest

        protected boolean checkIngest​(java.io.File localFile,
                              java.lang.String fileName,
                              org.apache.manifoldcf.core.interfaces.Specification documentSpecification,
                              org.apache.manifoldcf.crawler.interfaces.IFingerprintActivity activities)
                       throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                              org.apache.manifoldcf.agents.interfaces.ServiceInterruption
        Check if a file should be ingested, given a document specification and a local copy of the file. It is presumed that only files that passed checkInclude() and were also flagged as needing file data by checkNeedFileData() will be checked by this method.
        Parameters:
        localFile - is the file.
        fileName - is the JCIFS file name.
        documentSpecification - is the specification.
        activities - are the activities available to determine indexability.
        Returns:
        true if the file should be ingested.
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException
        org.apache.manifoldcf.agents.interfaces.ServiceInterruption

      • matchSubPath

        protected static int matchSubPath​(java.lang.String subPath,
                                  java.lang.String fullPath)
        Match a sub-path. The sub-path must match the complete starting part of the full path, in a path sense. The returned value should point into the file name beyond the end of the matched path, or be -1 if there is no match.
        Parameters:
        subPath - is the sub path.
        fullPath - is the full path.
        Returns:
        the index of the start of the remaining part of the full path, or -1.

      • checkMatch

        protected static boolean checkMatch​(java.lang.String sourceMatch,
                                    int sourceIndex,
                                    java.lang.String match)
        Check a match between two strings with wildcards.
        Parameters:
        sourceMatch - is the expanded string (no wildcards)
        sourceIndex - is the starting point in the expanded string.
        match - is the wildcard-based string.
        Returns:
        true if there is a match.

      • processCheck

        protected static boolean processCheck​(boolean caseSensitive,
                                      java.lang.String sourceMatch,
                                      int sourceIndex,
                                      java.lang.String match,
                                      int matchIndex)
        Recursive worker method for checkMatch. Returns 'true' if there is a path that consumes both strings in their entirety in a matched way.
        Parameters:
        caseSensitive - is true if file names are case sensitive.
        sourceMatch - is the source string (w/o wildcards)
        sourceIndex - is the current point in the source string.
        match - is the match string (w/wildcards)
        matchIndex - is the current point in the match string.
        Returns:
        true if there is a match.

      • getForcedAcls

        protected static java.lang.String[] getForcedAcls​(org.apache.manifoldcf.core.interfaces.Specification spec)
        Grab forced acl out of document specification.
        Parameters:
        spec - is the document specification.
        Returns:
        the acls.

      • getForcedShareAcls

        protected static java.lang.String[] getForcedShareAcls​(org.apache.manifoldcf.core.interfaces.Specification spec)
        Grab forced share acls out of document specification.
        Parameters:
        spec - is the document specification.
        Returns:
        the acls.

      • getForcedParentFolderAcls

        protected static java.lang.String[] getForcedParentFolderAcls​(org.apache.manifoldcf.core.interfaces.Specification spec)
        Grab forced parent folder acls out of document specification.
        Parameters:
        spec - is the document specification.
        Returns:
        the acls.

      • mapToIdentifier

        protected java.lang.String mapToIdentifier​(java.lang.String path)
                                    throws java.net.MalformedURLException,
                                           java.net.UnknownHostException
        Map a "path" specification to a full identifier.
        Throws:
        java.net.MalformedURLException
        java.net.UnknownHostException

      • getFileCanonicalPath

        protected static java.lang.String getFileCanonicalPath​(jcifs.smb.SmbFile file)
        Get canonical path

      • fileExists

        protected static boolean fileExists​(jcifs.smb.SmbFile file)
                             throws jcifs.smb.SmbException
        Check for file/directory existence
        Throws:
        jcifs.smb.SmbException

      • fileIsDirectory

        protected static boolean fileIsDirectory​(jcifs.smb.SmbFile file)
                                  throws jcifs.smb.SmbException
        Check if file is a directory
        Throws:
        jcifs.smb.SmbException

      • fileLastModified

        protected static long fileLastModified​(jcifs.smb.SmbFile file)
                                throws jcifs.smb.SmbException
        Get last modified date for file
        Throws:
        jcifs.smb.SmbException

      • fileLength

        protected static long fileLength​(jcifs.smb.SmbFile file)
                          throws jcifs.smb.SmbException
        Get file length
        Throws:
        jcifs.smb.SmbException

      • fileListFiles

        protected static jcifs.smb.SmbFile[] fileListFiles​(jcifs.smb.SmbFile file,
                                                   jcifs.smb.SmbFileFilter filter)
                                            throws jcifs.smb.SmbException
        List files
        Throws:
        jcifs.smb.SmbException

      • getFileInputStream

        protected static java.io.InputStream getFileInputStream​(jcifs.smb.SmbFile file)
                                                 throws java.io.IOException
        Get input stream for file
        Throws:
        java.io.IOException

      • getFileSecurity

        protected static jcifs.ACE[] getFileSecurity​(jcifs.smb.SmbFile file,
                                             boolean useSIDs)
                                      throws java.io.IOException
        Get file security
        Throws:
        java.io.IOException

      • getFileShareSecurity

        protected static jcifs.ACE[] getFileShareSecurity​(jcifs.smb.SmbFile file,
                                                  boolean useSIDs)
                                           throws java.io.IOException
        Get share security
        Throws:
        java.io.IOException

      • getFileType

        protected static int getFileType​(jcifs.smb.SmbFile file)
                          throws jcifs.smb.SmbException
        Get file type
        Throws:
        jcifs.smb.SmbException

      • equivalentSmbExceptions

        protected static boolean equivalentSmbExceptions​(jcifs.smb.SmbException e1,
                                                 jcifs.smb.SmbException e2)
        Check if two SmbExceptions are equivalent

      • equivalentIOExceptions

        protected static boolean equivalentIOExceptions​(java.io.IOException e1,
                                                java.io.IOException e2)
        Check if two IOExceptions are equivalent

      • outputConfigurationHeader

        public void outputConfigurationHeader​(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext,
                                      org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                      java.util.Locale locale,
                                      org.apache.manifoldcf.core.interfaces.ConfigParams parameters,
                                      java.util.List<java.lang.String> tabsArray)
                               throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                      java.io.IOException
        Output the configuration header section. This method is called in the head section of the connector's configuration page. Its purpose is to add the required tabs to the list, and to output any javascript methods that might be needed by the configuration editing HTML.
        Specified by:
        outputConfigurationHeader in interface org.apache.manifoldcf.core.interfaces.IConnector
        Overrides:
        outputConfigurationHeader in class org.apache.manifoldcf.core.connector.BaseConnector
        Parameters:
        threadContext - is the local thread context.
        out - is the output to which any HTML should be sent.
        parameters - are the configuration parameters, as they currently exist, for this connection being configured.
        tabsArray - is an array of tab names. Add to this array any tab names that are specific to the connector.
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException
        java.io.IOException

      • outputConfigurationBody

        public void outputConfigurationBody​(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext,
                                    org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                    java.util.Locale locale,
                                    org.apache.manifoldcf.core.interfaces.ConfigParams parameters,
                                    java.lang.String tabName)
                             throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                    java.io.IOException
        Output the configuration body section. This method is called in the body section of the connector's configuration page. Its purpose is to present the required form elements for editing. The coder can presume that the HTML that is output from this configuration will be within appropriate <html>, <body>, and <form> tags. The name of the form is "editconnection".
        Specified by:
        outputConfigurationBody in interface org.apache.manifoldcf.core.interfaces.IConnector
        Overrides:
        outputConfigurationBody in class org.apache.manifoldcf.core.connector.BaseConnector
        Parameters:
        threadContext - is the local thread context.
        out - is the output to which any HTML should be sent.
        parameters - are the configuration parameters, as they currently exist, for this connection being configured.
        tabName - is the current tab name.
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException
        java.io.IOException

      • processConfigurationPost

        public java.lang.String processConfigurationPost​(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext,
                                                 org.apache.manifoldcf.core.interfaces.IPostParameters variableContext,
                                                 java.util.Locale locale,
                                                 org.apache.manifoldcf.core.interfaces.ConfigParams parameters)
                                          throws org.apache.manifoldcf.core.interfaces.ManifoldCFException
        Process a configuration post. This method is called at the start of the connector's configuration page, whenever there is a possibility that form data for a connection has been posted. Its purpose is to gather form information and modify the configuration parameters accordingly. The name of the posted form is "editconnection".
        Specified by:
        processConfigurationPost in interface org.apache.manifoldcf.core.interfaces.IConnector
        Overrides:
        processConfigurationPost in class org.apache.manifoldcf.core.connector.BaseConnector
        Parameters:
        threadContext - is the local thread context.
        variableContext - is the set of variables available from the post, including binary file post information.
        parameters - are the configuration parameters, as they currently exist, for this connection being configured.
        Returns:
        null if all is well, or a string error message if there is an error that should prevent saving of the connection (and cause a redirection to an error page).
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException

      • viewConfiguration

        public void viewConfiguration​(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext,
                              org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                              java.util.Locale locale,
                              org.apache.manifoldcf.core.interfaces.ConfigParams parameters)
                       throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                              java.io.IOException
        View configuration. This method is called in the body section of the connector's view configuration page. Its purpose is to present the connection information to the user. The coder can presume that the HTML that is output from this configuration will be within appropriate <html> and <body>tags.
        Specified by:
        viewConfiguration in interface org.apache.manifoldcf.core.interfaces.IConnector
        Overrides:
        viewConfiguration in class org.apache.manifoldcf.core.connector.BaseConnector
        Parameters:
        threadContext - is the local thread context.
        out - is the output to which any HTML should be sent.
        parameters - are the configuration parameters, as they currently exist, for this connection being configured.
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException
        java.io.IOException

      • outputSpecificationHeader

        public void outputSpecificationHeader​(org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                      java.util.Locale locale,
                                      org.apache.manifoldcf.core.interfaces.Specification ds,
                                      int connectionSequenceNumber,
                                      java.util.List<java.lang.String> tabsArray)
                               throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                      java.io.IOException
        Output the specification header section. This method is called in the head section of a job page which has selected a repository connection of the current type. Its purpose is to add the required tabs to the list, and to output any javascript methods that might be needed by the job editing HTML. The connector will be connected before this method can be called.
        Specified by:
        outputSpecificationHeader in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector
        Overrides:
        outputSpecificationHeader in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector
        Parameters:
        out - is the output to which any HTML should be sent.
        locale - is the locale the output is preferred to be in.
        ds - is the current document specification for this job.
        connectionSequenceNumber - is the unique number of this connection within the job.
        tabsArray - is an array of tab names. Add to this array any tab names that are specific to the connector.
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException
        java.io.IOException

      • outputSpecificationBody

        public void outputSpecificationBody​(org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                    java.util.Locale locale,
                                    org.apache.manifoldcf.core.interfaces.Specification ds,
                                    int connectionSequenceNumber,
                                    int actualSequenceNumber,
                                    java.lang.String tabName)
                             throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                    java.io.IOException
        Output the specification body section. This method is called in the body section of a job page which has selected a repository connection of the current type. Its purpose is to present the required form elements for editing. The coder can presume that the HTML that is output from this configuration will be within appropriate <html>, <body>, and <form> tags. The name of the form is always "editjob". The connector will be connected before this method can be called.
        Specified by:
        outputSpecificationBody in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector
        Overrides:
        outputSpecificationBody in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector
        Parameters:
        out - is the output to which any HTML should be sent.
        locale - is the locale the output is preferred to be in.
        ds - is the current document specification for this job.
        connectionSequenceNumber - is the unique number of this connection within the job.
        actualSequenceNumber - is the connection within the job that has currently been selected.
        tabName - is the current tab name. (actualSequenceNumber, tabName) form a unique tuple within the job.
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException
        java.io.IOException

      • processSpecificationPost

        public java.lang.String processSpecificationPost​(org.apache.manifoldcf.core.interfaces.IPostParameters variableContext,
                                                 java.util.Locale locale,
                                                 org.apache.manifoldcf.core.interfaces.Specification ds,
                                                 int connectionSequenceNumber)
                                          throws org.apache.manifoldcf.core.interfaces.ManifoldCFException
        Process a specification post. This method is called at the start of job's edit or view page, whenever there is a possibility that form data for a connection has been posted. Its purpose is to gather form information and modify the document specification accordingly. The name of the posted form is always "editjob". The connector will be connected before this method can be called.
        Specified by:
        processSpecificationPost in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector
        Overrides:
        processSpecificationPost in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector
        Parameters:
        variableContext - contains the post data, including binary file-upload information.
        locale - is the locale the output is preferred to be in.
        ds - is the current document specification for this job.
        connectionSequenceNumber - is the unique number of this connection within the job.
        Returns:
        null if all is well, or a string error message if there is an error that should prevent saving of the job (and cause a redirection to an error page).
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException

      • viewSpecification

        public void viewSpecification​(org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                              java.util.Locale locale,
                              org.apache.manifoldcf.core.interfaces.Specification ds,
                              int connectionSequenceNumber)
                       throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                              java.io.IOException
        View specification. This method is called in the body section of a job's view page. Its purpose is to present the document specification information to the user. The coder can presume that the HTML that is output from this configuration will be within appropriate <html> and <body>tags. The connector will be connected before this method can be called.
        Specified by:
        viewSpecification in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector
        Overrides:
        viewSpecification in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector
        Parameters:
        out - is the output to which any HTML should be sent.
        locale - is the locale the output is preferred to be in.
        ds - is the current document specification for this job.
        connectionSequenceNumber - is the unique number of this connection within the job.
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException
        java.io.IOException

      • getShareNames

        public jcifs.smb.SmbFile[] getShareNames​(java.lang.String serverURI)
                                  throws org.apache.manifoldcf.core.interfaces.ManifoldCFException
        given a server uri, return all shares
        Parameters:
        serverURI - -
        Returns:
        an array of SmbFile
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException

      • validateFolderName

        public java.lang.String validateFolderName​(java.lang.String folder)
                                    throws org.apache.manifoldcf.core.interfaces.ManifoldCFException
        Given a folder path, determine if the folder is in fact legal and accessible (and is a folder).
        Parameters:
        folder - is the relative folder from the network root
        Returns:
        the canonical folder name if valid, or null if not.
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException

      • getChildFolderNames

        public java.lang.String[] getChildFolderNames​(java.lang.String folder)
                                       throws org.apache.manifoldcf.core.interfaces.ManifoldCFException
        given a smb uri, return all children directories
        Parameters:
        folder - is the relative folder from the network root
        Returns:
        array of child folder names
        Throws:
        org.apache.manifoldcf.core.interfaces.ManifoldCFException