Fetching Content and Metadata

This chapter describes fetching and explains how to implement it in your exteNd Director search applications.

The following topics are covered:

About fetching

Fetching content involves two processes:

Importing data from the Content Management (CM) repository
Indexing the data in the exteNd Director DRE

The default fetcher

exteNd Director comes with a data fetcher for the exteNd Director CM repository. The CM fetcher allows you to perform Autonomy-based conceptual searches only on data stored in the CM repository.

The CM fetcher initiates the following processes automatically:

Process	Description
Fetching	Imports data from the CM repository into the exteNd Director DRE database for indexing
Synchronization	Propagates updates made in the CM repository to the corresponding DRE database so they can be indexed

Fetching is initiated whenever synchronization occurs. You schedule synchronization to run as a real-time or batch process, as described in Synchronization mode.

For real-time—or immediate—synchronization, you can specify the actions that trigger the process, as described in Operations that trigger immediate synchronization.

By default, synchronization occurs as a real-time process in immediate mode and is triggered when any of the following CM operations occur:

Add
Update
Remove
Check in
Check out
Publish
Uncheckout
Unpublish
Unlock
Roll back

Fetchers for custom data sources

If you want to use a data source other than the CM repository, you must purchase additional data fetchers from Autonomy, Inc.

When you purchase other data fetchers from Autonomy, you must initiate the fetch process programmatically using the Search API, as demonstrated in Implementing fetching in your applications next. You must also create a descriptor for each data fetcher you use, as described in Data fetcher descriptors.

Implementing fetching in your applications

The exteNd Director Search API provides wrapper classes around Autonomy APIs that provide methods for fetching content.

Key fetcher classes and interfaces

Key classes and interfaces for fetching content include:

Class or interface	Description	Package
EbiDataFetcherDelegate	Delegate for accessing objects that implement the EbiDataFetcher interface, which provides methods for importing content from a specified data source into the exteNd Director DRE, where it is indexed	com.sssw.search.api
EboFactory	Factory class that provides methods for getting Search subsystem delegates such as EbiDataFetcherDelegate	com.sssw.search.client

Fetcher methods

This section describes Search API methods that you can use to perform data fetching in your exteNd Director applications.

Getting a fetcher delegate

Here is the method for getting a fetcher delegate:

  com.sssw.search.client.EboFactory.getDataFetcherDelegate()

This method returns an object that implements the EbiDataFetcherDelegate interface. Methods on this object can be used to invoke and manage the data fetching process.

For more information For information about why to use delegates, see Programming practices.

Initiating the fetch process

Here is the method for fetching data:

  com.sssw.search.API.EbiDataFetcherDelegate.fetchData()

This method fetches document data from the source repository into the destination query engine database.

Code example: fetching data

The following code segment shows how to initiate the fetch process using the Search API:

  ...
  //Instantiate a data fetcher delegate
  com.sssw.search.api.EbiDataFetcherDelegate fetcher = com.sssw.search.client.EboFactory.getDataFetcherDelegate(whichFetcher);
  
  //Fetch the data
  fetcher.fetchData(context, false);
  ...

The getDataFetcherDelegate() method takes a string argument that specifies which data fetcher to instantiate. Each data fetcher requires a descriptor, as described in Data fetcher descriptors next.

Data fetcher descriptors

You must create a descriptor for each data fetcher you purchase from Autonomy. exteNd Director provides a descriptor for the CM data fetcher.

You add data fetcher descriptors in the services configuration file services.xml for the Search subsystem.

For more information about where project files are located, see the section on exteNd Director project structure in Developing exteNd Director Applications.

Syntax

The syntax of data fetcher descriptors in services.xml sets up a mapping between the data fetcher interface in the Search API and the data fetcher implementation class.

This mapping is illustrated in the descriptor for the CM data fetcher, which is supplied with exteNd Director:

  <service>
  <interface>com.sssw.search.api.EbiDataFetcher.CM</interface>
  <impl-class>com.sssw.cm.core.EboDataFetcher</impl-class>
  <description>Data fetcher object for the Content Management subsystem</description>
  <max-instances>0</max-instances>
  <startup>M</startup>
  <namespaced>false</namespaced>
  </service>

The string CM in the <interface> element is an example of a data fetcher specifier that you pass as a string argument to the getDataFetcherDelegate() method to initiate the fetch process, as described in Code example: fetching data.

This is the syntax to use for any data fetcher descriptor. For example, if you purchase or create a custom data fetcher called MYCO that is defined by the class com.sssw.myco.Fetcher, you need to add a descriptor that looks like this:

  <service>
  <interface>com.sssw.search.api.EbiDataFetcher.MYCO</interface>
  <impl-class>com.sssw.myco.Fetcher</impl-class>
  <description>Data fetcher object for MYCO</description>
  <max-instances>0</max-instances>
  <startup>M</startup>
  <namespaced>false</namespaced>
  </service>

Associating data fetchers with query result objects

For each data fetcher you add, you can optionally add an associated query result object descriptor. Here is the descriptor in services.xml for the query result object already defined for the CM data fetcher:

  <service>
  <interface>com.sssw.search.api.EbiQueryResult.CM</interface>
  <impl-class>com.sssw.cm.core.EboQueryResult</impl-class>
  <description>Query result object for the Content Management subsystem</description>
  <max-instances>0</max-instances>
  <startup>M</startup>
  <namespaced>false</namespaced>
  </service>

To associate the CM query result object with your MYCO data fetcher, you need to add a descriptor that looks like this:

  <service>
  <interface>com.sssw.search.api.EbiQueryResult.MYCO</interface>
  <impl-class>com.sssw.cm.core.EboQueryResult</impl-class>
  <description>Query result object for MYCO</description>
  <max-instances>0</max-instances>
  <startup>M</startup>
  <namespaced>false</namespaced>
  </service>

Use this same syntax to create a descriptor for a custom query result object, as follows:

  <service>
  <interface>com.sssw.search.api.EbiQueryResult.MYCO</interface>
  <impl-class>com.sssw.myco.webapp.impl.MycoQueryResult</impl-class>
  <description>Query result object for MYCO</description>
  <max-instances>0</max-instances>
  <startup>M</startup>
  <namespaced>false</namespaced>
  </service>