Content Search Guide

CHAPTER 12

Search Query Types

This chapter describes the types of Autonomy-based queries supported by the exteNd Director Search subsystem and how to implement them.

The following queries are covered:

For more information For background information, see Querying Content and Metadata.

Boolean queries

Description

A boolean query uses logical operators to refine search criteria.

Syntax

  word1+LOGICAL OPERATOR+word2+LOGICAL OPERATOR+word3+LOGICAL OPERATOR+... wordN

This is the order of precedence of logical operators (in descending order):

NOT
AND, WNEAR, NEAR
OR, XOR, EOR

You can use parentheses to group operators and operands.

Examples

  effect+recession+AND+economic+AND+slowdown+AND+consumer+AND+spending+OR+buying

In this example parentheses are not needed, because AND has a higher precedence than OR. To change the order of evaluation use parentheses, as in this example:

  ((effect+recession+AND+economic)+OR+(+slowdown+AND+consumer+AND+spending))+OR+buying+OR+(effect+AND+economic+NOT+depression)

Implementation

In the getComponentData() method of your exteNd Director component, set up the boolean query like this:

  ...
  // Instantiate a blank query object
  com.sssw.search.api.EbiQuery query = com.sssw.search.factory.EboFactory.getQuery();
  // Set the query type to be "text"
  query.setQueryType(query.QUERY_TYPE_TEXT);
  // Specify the query
  query.setText("effect+AND+recession+AND+economic+AND+slowdown+AND+consumer+AND+spending+OR+buying");
  ...

Conceptual queries

Description

A conceptual query returns content that is related by meaning and ranked by relevance to the search criteria. The query string should be as specific as possible—usually a paragraph or at least a sentence.

Syntax

  word1+word2+word3+... wordN

Example

  The+effect+of+the+recession+on+consumer+spending

Implementation

In the getComponentData() method of your exteNd Director component, set up the conceptual query like this:

  ...
  // Instantiate a blank query object
  com.sssw.search.api.EbiQuery query = com.sssw.search.factory.EboFactory.getQuery();
  // Set query type to be "text"
  query.setQueryType(query.QUERY_TYPE_TEXT);
  // Specify the query
  query.setText("The+effect+of+the+recession+on+consumer+spending");
  ...

Field queries

Description

A field query allows you to search metadata in documents, or metadata and content in a single method call. Metadata refers to standard metadata and/or custom metadata. Before issuing a field query, make sure you configure your search environment to specify the types of metadata you want to search—standard metadata and/or custom metadata—as described in Setting Search Options.

Syntax

To specify the metadata to search, you must set up a field specifier list by calling the method setFieldSpecList() on an object that implements the EbiQuery interface. If you are searching more than one field, you must define a field boolean expression that specifies how the fields should be searched, then pass this expression as the second argument to setFieldSpecList(). If you are searching only one field, this argument should be null.

Field specifier list The syntax for the field specifier list is:

  fnameFieldname1=*value1*+fnameFieldname2=*value2*+... fnameFieldnameN=*value3*

You can issue field queries using the names of standard or custom metadata fields, as long as the metadata has been indexed.

Standard metadata field names are listed in DirectorDRE.cfg, located in:

  exteNd Director installation directory\autonomy\engine

Field boolean expression The syntax for the field boolean expression is:

  fnameFieldname1+LOGICAL OPERATOR+fnameFieldname2+LOGICAL OPERATOR+... fnameFieldnameN

Logical operators can be the keywords AND, OR, and NOT.

Example

Suppose that in your Content Management (CM) repository you define a document type called Research Study and a custom field called Topic. If you want to find all research studies whose topic is Economy, follow these steps:

Define a field specifier list, as follows:

  String fieldSpecList = "fnameDOCTYPENAME=*Research Study*+fnameTopic=*Economy*";

Define a boolean expression to direct the search, as follows:

  String fieldBooleanExpr = "fnameDOCTYPENAME+AND+fnameTopic";

Pass these two strings as arguments to the setFieldSpecList() method:
```
  query.setFieldSpecList(fieldSpecList, fieldBooleanExpr);
```

Implementation

In the getComponentData() method of your exteNd Director component, set up the field query like this:

  ...
  // Instantiate a blank query object
  com.sssw.search.api.EbiQuery query = com.sssw.search.factory.EboFactory.getQuery();
  // Set query type to be "text"
  query.setQueryType(query.QUERY_TYPE_TEXT);
  // Specify the query
  query.setText("effect+recession+economic+slowdown+consumer+spending");
  // Provide field specifiers (to find research studies about the
  // economy)
   String fieldSpecList = "fnameDOCTYPENAME=*Research Study*+fnameTopic=*Economy*"
   String fieldBooleanExpr = "fnameDOCTYPENAME+AND+fnameTopic";
   query.setFieldSpecList(fieldSpecList, fieldBooleanExpr);
  ...

Notice that this example defines criteria for searching content (using the setText() method) as well as criteria for searching metadata (using the setFieldSpecList() method) in a single query expression. This is a powerful feature, because it allows you to search both content and metadata with a single call to the runQuery() method.

Fuzzy queries

Description

A fuzzy query tries to match terms even when they are misspelled.

Implementation

Fuzzy queries are implemented in the same way as conceptual queries except that you must define the query type to be fuzzy. In the getComponentData() method of your exteNd Director component, set up the fuzzy query like this:

  ...
  // Instantiate a blank query object
  com.sssw.search.api.EbiQuery query = com.sssw.search.factory.EboFactory.getQuery();
  // Set query type to be "fuzzy"
  query.setQueryType(query.QUERY_TYPE_FUZZY);
  // Specify the query
  query.setText("The+efect+of+the+recession+on+consumer+spending");
  ...

Get-all queries

Description

A get-all query returns all documents.

Implementation

You do not specify query strings for get-all queries, but you must define the query type to be get all. In the getComponentData() method of your exteNd Director component, set up the get-all query like this:

  ...
  // Instantiate a blank query object
  com.sssw.search.api.EbiQuery query = com.sssw.search.factory.EboFactory.getQuery();
  // Set query type to be "get all"
  query.setQueryType(query.QUERY_TYPE_GETALL);
  ...

NOTE: The get-all query is provided for debugging purposes, but not recommended for production use—because returning all documents can impact performance.

Keyword search

Description

A traditional keyword search looks for occurrences of a search string.

Syntax

  word1:+word2:+word3:+... wordN:

Example

  effect:recession:+economic:+slowdown:+consumer:+spending:+buying:

Implementation

These are the steps for implementing keyword searches:

Make sure that content can be indexed and stored in the DRE, by enabling the property com.sssw.cm.fetch.store.content.repository name in config.xml for the CM subsystem.

For more information about where project files are located, see the section on exteNd Director project structure in Developing exteNd Director Applications.

For more information about this property, see Copy document contents into the DRE?.

In the getComponentData() method of your exteNd Director component, set up the keyword query like this:

  ...
  // Instantiate a blank query object
  com.sssw.search.api.EbiQuery query = com.sssw.search.factory.EboFactory.getQuery();
  // Set query type to be "text"
  query.setQueryType(query.QUERY_TYPE_TEXT);
  // Specify the query
  query.setText("effect:+recession:+consumer:+spending:+buying:");
  ...

Proper name search

Description

As its name suggests, a proper name search looks for proper names.

Syntax

  ProperName1+ProperName2+ProperName3+... ProperNameN

Example

  Ralph+Waldo+Emerson

Implementation

Proper name searches are implemented in the same way as conceptual queries, except that you must define the query type to be a name search. In the getComponentData() method of your exteNd Director component, set up the proper name search like this:

  ...
  // Instantiate a blank query object
  com.sssw.search.api.EbiQuery query = com.sssw.search.factory.EboFactory.getQuery();
  // Set query type to be "name search"
  query.setQueryType(query.QUERY_TYPE_NAMESEARCH);
  // Specify the query
  query.setText("Ralph+Waldo+Emerson");
  ...

Proximity queries

Description

A proximity query specifies that certain words in the query must occur close to each other.

Syntax

Put single quotes around the words that should occur close together.

  word1+`word2+word3'+... wordN

Example

  effect+recession+`economic+slowdown'+`consumer+spending'+`consumer+buying'

Usage

In the getComponentData() method of your exteNd Director component, set up the proximity query like this:

  ...
  // Instantiate a blank query object
  com.sssw.search.api.EbiQuery query = com.sssw.search.factory.EboFactory.getQuery();
  // Set the query type to be "text"
  query.setQueryType(query.QUERY_TYPE_TEXT);
  // Specify the query
  query.setText("effect+recession+`economic+slowdown'+`consumer+spending'+`consumer+buying'");
  ...

Suggest similar documents

Description

A suggest-similar query tries to find documents similar to other documents that were found to be relevant to your search criteria.

Syntax

  document-identifier1+document-identifier2+document-identifier3+... document-identifierN

The document identifier is either the document ID assigned to the document by the query engine (exteNd Director DRE) or its URL (DRE reference).

Example

In the query expression, you can specify document identifiers as document IDs or document URL references.

Implementation

To implement the suggest-similar query, follow these steps:

Set the query type to suggest, using the setQueryType() method.
Define suggest options, using the setSuggestOptions() method:
1. Indicate whether the document identifiers you specify in the query expression should be treated as document IDs (generated by the DRE) or document references.
2. Indicate whether the documents you specify in the query expression should be included or excluded from the query results.
Construct the query expression.

In the getComponentData() method of your exteNd Director component, set up the suggest similar query like this:

  ...
  //Get the query engine delegate
  EbiQueryEngineDelegate qe = com.sssw.search.factory.EboFactory.getQueryEngineDelegate();
  
  //Instantiate a blank query object
  com.sssw.search.api.EbiQuery query = com.sssw.search.factory.EboFactory.getQuery();
  
  query.selectAll();
  
  //Set query type to "text" to get documents whose references
  //or IDs can be passed on to the suggest-similar query
  query.setQueryType(com.sssw.search.api.EbiQuery.QUERY_TYPE_TEXT);
  
  //Set query string
  query.setText("economic+recession+effects");
  
  String ids = "";
  
  //Search for documents that meet the original search criteria
  
  try
  {
     Iterator results = qe.runQuery(context, query, null, false).iterator();
  	 //Put together the list of engine document IDs that constitute
  	 //the suggest-similar query
     boolean first = true;
     while (results.hasNext())
     {
        com.sssw.search.api.EbiQueryResult res = (com.sssw.search.api.EbiQueryResult) results.next();
        String engineDocID = res.getProperty(EbiQueryResult.PROP_ENGINE_DOC_ID);
        if (!first)
           ids += "+";
        else
           first = false;
        ids += engineDocID;
     }
  }
  catch (Exception e)
  {
     if (m_log.isError())
        m_log.error(e);
  }
  
  if (!"".equals(ids))
  {
     //Set query type to "suggest"
     query.setQueryType(EbiQuery.QUERY_TYPE_SUGGEST);
     query.setSuggestOptions(true, true);
     query.setText(ids);
     //Run the suggest-similar query
     Iterator results = qe.runQuery(context, query, null, false);
     ...
  }

In this example, the setSuggestOptions() method specifies that:

Identifiers in the query expression be interpreted as document IDs
Documents referenced in the query expression suggestQuery be excluded from the query results

To use document URL references (instead of document IDs) in this example, modify the following statements:

Change:

  String engineDocID = res.getProperty(EbiQueryResult.PROP_ENGINE_DOC_ID);

to:

  String engineDocID = res.getProperty(EbiQueryResult.PROP_ENGINE_DOC_REF);

Change:

  query.setSuggestOptions(true, true);

to:

  query.setSuggestOptions(false, true);

Thesaurus queries

Description

A thesaurus query analyzes not only the terms in the query expression, but also associated terms (or synonyms) that you define in a separate thesaurus.

To issue a thesaurus query, you follow these basic steps:

Decide what terms you want to search for.
For each term, create a thesaurus document that contains synonyms or words you want to associate with that term.
Construct your query expression.
Mark the query to run as a thesaurus query by calling the setIsThesaurusQuery() method on an object that implements the EbiQuery interface.
Get a repository descriptor object for each thesaurus DRE you create.
Set thesaurus options by calling the setThesaurus() method on an EbiQuery object.
Run the query.

Thesaurus DRE

For each term you plan to search, you need to create a thesaurus DRE (Step 2 above). This section explains how.

NOTE: This section is based on information adapted from the Autonomy Server 2.2 manual from Autonomy, Inc.

Procedure To create a thesaurus DRE:

Create a folder called ThesaurusDRE in the folder where the Autonomy integration files are installed.

By default, the Autonomy integration files are stored in the exteNd Director installation directory at:
```
  exteNd Director\autonomy
```
Copy all files from the existing exteNd Director DRE to the ThesaurusDRE folder.

These files are stored in the exteNd Director installation directory at:
```
  exteNd Director\autonomy\engine
```
In the ThesaurusDRE folder, rename the following files:

Change this

To this

DirectorDRE.exe

ThesaurusDRE.exe

DirectorDREadmin.exe

ThesaurusDREadmin.exe

DirectorDRE.cfg

ThesaurusDRE.cfg
Edit ThesaurusDRE.cfg to point to different ports, as follows:

Change this

To this

The QUERYPORT setting

QUERYPORT=8000

The INDEXPORT setting

INDEXPORT=8001
In your favorite text editor, create a thesaurus document.

The thesaurus document should contain a list of associated words, separated by carriage returns, as in this example:
Save the thesaurus document.
Use the DRE Administration console to import and index your thesaurus document in the thesaurus DRE. Follow these steps:
1. Double-click ThesaurusDRE.exe.
2. Connect the DRE Administration console to the thesaurus DRE by double-clicking ThesaurusDREadmin.exe in the ThesaurusDRE folder.
  
  Make sure the connection was successful by checking for the check mark symbol on the DRE Administration console window:
3. Create a new DRE database by entering ThesaurusDB in the New Database Name text box and clicking the Create New Database button.
4. If an alert box appears, click Yes to confirm that you want to create the database.
5. Select the Import-Index tab.
6. Click the Import Files into IDX format button.
  
  The Main Import Settings dialog opens.
7. Click the Add Files button.
8. Browse to your thesaurus document, select it, and click Open.
  
  The thesaurus document should appear in the list of input files to import.
9. Select ThesaurusDB as the destination database and click OK.
10. Click Yes to confirm that you want to add the thesaurus document's IDX file to the list of documents to index.
11. Click Index into DRE now! to index the thesaurus document.

Change this	To this
DirectorDRE.exe	ThesaurusDRE.exe
DirectorDREadmin.exe	ThesaurusDREadmin.exe
DirectorDRE.cfg	ThesaurusDRE.cfg

Change this	To this
The QUERYPORT setting	QUERYPORT=8000
The INDEXPORT setting	INDEXPORT=8001

Now you are ready to implement a thesaurus query.

Implementation

In the getComponentData() method of your exteNd Director component, set up the thesaurus query like this:

  ...
  //Instantiate a blank query object
  com.sssw.search.api.EbiQuery query = com.sssw.search.factory.EboFactory.getQuery();
  
  //Construct your query expression
  query.setText("feline");
  
  //Mark the query to run as a thesaurus query
  query.setIsThesaurusQuery(true);
  
  //Get a repository descriptor for the thesaurus DRE
  EbiRepositoryDesc thesaurus = com.sssw.search.factory.EboFactory.getRepositoryDesc("141.155.166.181", 8000, 8001, "ThesaurusDB");
  
  //Set thesaurus options
  query.setThesaurus(thesaurus);
  
  //Run the query
  Collection results = queryEngine.runQuery(context, query, repositories, true);
  ...