The Joomla! ® Community Magazine

Developing a Smart Search Plugin

Written by | Wednesday, 01 February 2012 00:00 | Published in 2012 February
With the Joomla! 2.5 release, a new search component has been included to improve the search engine in Joomla! sites. Based on JXtended's Finder, Smart Search is quite the powerful tool and can be used to improve search results and show the user exactly what they want. Out of the box, Smart Search will work for most core data, but third party developers will have to create their own Finder plugins to interact with Smart Search. Not to worry, I'll help you get started with that now!

What do the Finder plugins do?

The Finder plugins are where the logic is built to search for items and automatically update them during normal site use. Without these plugins, data from a component cannot be indexed and processed during searches from the Smart Search component. This same requirement exists for the "classic" Search component to enable data to be searched.

What do the Finder plugins need?

Each plugin will have its own unique requirements, but there is some code that each plugin requires to allow for proper functioning. In lieu of posting numerous code snippets, I'll instead link you to the Finder plugin for com_content articles at GitHub: https://github.com/joomla/joomla-cms/blob/master/plugins/finder/content/content.php. I encourage you to follow along with the code as I describe it here.

Class Properties and External Dependencies

As you can see just below the file doc block, there are two files that are required to be loaded. The jimport call loads the JComponentHelper class, and this class is used in the index method. Next, the require_once statement loads the Finder plugin's parent class, FinderIndexerAdapter. All Finder plugins must extend this class in order to function properly.

Now, I'll explain each of the class properties set in this plugin, as well as a few properties in the parent class to be aware of:

  • $context - This is an identifier for the plugin; in all core use, this is the part of the class name after plgFinder
  • $extension - The extension the plugin supports, i.e. com_content
  • $layout - The name of the sublayout that will be used when rendering search results; in core use, this is the name of the single item view on the component's front end (to further explain this, by default, the front end logic will search for a default_article.php template file in the search view of com_finder, and if not present, defaults to the default_result.php file)
  • $type_title - The title of the content type being processed by the plugin; this will be used to display the name of the "Type" filtering option when the filters are available (note, this is tied to a language string as well; see the appropriate language file for details on the structure of the key)
  • $table - The database table that the component's data is stored in (note: must use the #__ prefix here)

Additional properties in the parent FinderIndexerAdapter class are:

  • $mime - If indexing various file types (such as PDF files), this property should to be set to the correct MIME of that data - defaults to NULL
  • $db - The database object, all plugins should interact with the database using
    $this->db
  • $state_field - This is the field in which the published state is stored in in the database (usually state or field) - defaults to state

Class Functions and Parent Information

Next, I'll describe each of the functions in this class as well as important methods to be aware of in the parent FinderIndexerAdapter class. Except for the Finder plugin for working with com_categories, each of these functions feeds into a parent class which contains generic code to update the index properties of an item. Developers who wish to override these functions should carefully review what the parent classes do so that their data is properly handled in all instances.

  • public function __construct - Standard constructor class required by most plugins in order to load the plugin's language files from their default locations
  • public function onFinderCategoryChangeState - New to Joomla! 2.5, this method is triggered at the same time as the onCategoryChangeState event, which is triggered when a category's published state is changed in the admin list view, used to update index properties of child items of a category (obviously, not needed for components which do not have a category->item structure)
  • public function onFinderAfterDelete - This event acts at the same time as onContentAfterDelete and by default, removes the index data for items that have been removed either in the component or in the Smart Search administrator section.
  • public function onFinderAfterSave - This event acts at the same time as onContentAfterSave and by default, updates the access level information for items when the item's access level is changed or when its parent category's access level is changed. The context check for the single item should include both the backend and frontend single item edit views to work properly.
  • public function onFinderBeforeSave - This event acts at the same time as onContentBeforeSave and by default, queries for and stores the item or parent category's access level; used in conjunction with onFinderAfterSave to change the index data when the access level is changed
  • public function onFinderChangeState - This event acts at the same time as onContentChangeState and by default, updates the published state information for items when the published state is changed in the administrator list view. Also, this method catches when the plugin is disabled and removes index data handled by the now disabled plugin.
  • protected function index - This is the method that handles all processing to add items to the search index. This is where things such as the item's raw URL, routed URL, and metadata are set.
  • protected function setup - This method is run prior to the index method and prepares the indexer to process the plugin's data. In core use, each extension's route helper is included so that the routing can be properly set during index.
  • protected function getListQuery - This method sets a JDatabaseQuery object with a query to pull the data that should be indexed. The JDatabase and JDatabaseQuery APIs must be used here otherwise the indexer will fail.

Methods that developers should be aware of from the parent FinderIndexerAdapter class are:

  • protected function getStateQuery - This method sets a JDatabaseQuery object with a query to pull the published state and access levels of an item and its parent category. Components not using a category->item structure will need to override this method in their plugins.
  • protected function getUpdateQueryByTime - This method sets a JDatabaseQuery object with a query to pull the update time from a database table's "modified" field. Components which either don't store this data or use a different field name will need to override this method in their plugins.
  • protected function getURL - This method returns the non-SEF URL of an item. Using com_content with an article ID of 1, the default return is "index.php?option=com_content&view=article&id=1. Developers who need to add additional information to the URL will need to override this method in their plugins.

Note that this is by no means a complete listing of the full API for the Smart Search component, but just a highlight of methods that developers need to be aware of at a minimum to write functioning plugins for their components.

I Have Questions!!!

Undoubtedly, there will be a question or three that will be asked about Smart Search. Obviously, the best resources to use will be the Joomla! Forums (http://forum.joomla.org) and Joomla! WIKI (http://docs.joomla.org). Several pages already exist on the WIKI with information about Smart Search, and I personally frequent the forums and read many of the threads and am willing to help answer questions about Smart Search. So, don't be shy, ask away!

Read 23051 times
Tagged under Developers
Michael Babker

Michael Babker

Michael has been very active within the Joomla! Community since building his first site in February 2010.  He has contributed hundreds of hours supporting to the Joomla! project via the Bug Squad, spending time on the Joomla! Forums, and helping to enhance the Joomla! Platform.  One of his biggest contributions to date is working on migrating and integrating the Smart Search component into Joomla! 2.5.  His contributions to the community have been recognized with the 2012 J!OSCAR "GitHub Junkie" Award.  Michael joined the Production Leadership Team in November 2012 and is currently leading development on a new issue tracker system for the Joomla! CMS in addition to helping to maintain and improve the code in the CMS and Framework.

When he isn't working on core code, he can be found working on his own set of extensions, opinionating on Twitter, or out in the real world enjoying the sights and sounds of wherever he may be at the time.

Leave a comment

Make sure you enter the (*) required information where indicated.

[b] [i] [u] [s] [url] [quote] [code] [img]   

Comments (15)

  • avatar
    • 1
    • 0
    shahki

    Can Smart Search be directly connected to the enrtries in Sobipro and if so, how?

  • avatar
    • 0
    • 1
    Michael Babker

    If the developers create a Smart Search plugin that allows for Sobipro data to be indexed, then yes, Smart Search can be connected. Just like the old search component, in order to be used, there must be an appropriate plugin to retrieve the data.

  • avatar
    • 1
    • 0
    shahki

    Hello, thanx for the reply

    Do you have any suggestions on where to find the appropriate plugin?

  • avatar
    • 0
    • 1
    Michael Babker

    You would need to check with the extension developer to see if they have a plugin already or have plans to create one. I may be biased since I helped work on the code, but if you are following core conventions for database fields, then creating a plugin should not take very much time at all. Plus, we tried to make the adapter as flexible as possible for those who need to tweak some things so developers aren't locked into a strict API.

  • avatar
    • 0
    • 1
    Thomas Riis

    Very good article, thanks. However I am new to plugins - where should my extending plugin (which folder) reside after install? A sample manifest file could be a nice addition to this article.

    Riceman

  • avatar
    • 1
    • 1
    Thomas Riis

    I figured out that the manifest file is also in the svn : ) Should I add a folder for my own component like "content" or should i just place it into the content folder?

    Riceman

  • avatar
    • 0
    • 1
    Michael Babker

    When you package the plugin to install it, it will automatically place the files in the correct place for you. Using the plg_finder_content manifest as an example, at a minimum, all you would need to change is the name and the files listing. ZIP it all up, and you'll have an installable plugin.

    If you need another example of a non-core solution, I've developed a couple of plugins for one of my own components. That manifest is at https://github.com/mbabker/Podcast-Manager/blob/master/ plg_finder_podcastmanager_feeds/podcastmanager_feeds.xml

  • avatar
    • 1
    • 1
    y8 car

    Very good article, However I am new to plugins
    thanks.

  • avatar
    • 0
    • 1
    Anne

    hello. i just wanna ask if what appropriate plugins for joomla will i use for modules so i can access in the database?

  • avatar
    • 0
    • 1
    ralf

    Where could I find (beginner) step by step instructions on how to edit the code so that it works with joomla 2.5 and jvotesystem plugin. You're giving great insight above, but I don't know where to start to edit what, and then do what with the code...
    Sorry, but I'm a newbie and joomla is quite complex...
    thanks for any help and advice

  • avatar
    • 0
    • 1
    Olivia

    My Smart Search will not finish indexing. It will get stuck at 45% and will only gather 45% of the articles on the site I'm working with. I have looked up how to fix this, but every forum I have discovered asking the same question, the problem was never resolved. Or no one answered. This has to be a problem for more people. How do I fix this?

  • avatar
    • 0
    • 1
    Kathryn

    enter your message here...
    Hi! I just tried to do one of these plugins using your information here with a copy of the finder content plugin. First let me say I've never created a custom plugin before in my life. Second I haven't tried to use it as yet because I'm experiencing some (probably minor) problems with the install.

    1. for some reason when I upload it, although I renamed all the files, the files are going into a folder titled "content" and not into their own folder;
    2. when trying to open the plugin in the backend (via plugin manager) it is looking for content.xml, while I have renamed the manifest file to something else.

    It is for an auction website using wdc auction

    Is there something I'm doing wrong somewhere?

    This is the contents of the auction1.php file (copy of content.php which I changed everything I could see that might need changing according to your information):

    Code:
    loadLanguage();
    }

    /**
    * Method to update the item link information when the item category is
    * changed. This is fired when the item category is published or unpublished
    * from the list view.
    *
    * @param string $extension The extension whose category has been updated.
    * @param array $pks A list of primary key ids of the content that has changed state.
    * @param integer $value The value of the state that the content has been changed to.
    *
    * @return void
    *
    * @since 2.5
    */
    public function onFinderCategoryChangeState($extension, $pks, $value)
    {
    // Make sure we're handling com_auction categories
    if ($extension == 'com_auction')
    {
    $this->categoryStateChange($pks, $value);
    }
    }

    /**
    * Method to remove the link information for items that have been deleted.
    *
    * @param string $context The context of the action being performed.
    * @param JTable $table A JTable object containing the record to be deleted
    *
    * @return boolean True on success.
    *
    * @since 2.5
    * @throws Exception on database error.
    */
    public function onFinderAfterDelete($context, $table)
    {
    if ($context == 'com_auction.article')
    {
    $id = $table->id;
    }
    elseif ($context == 'com_finder.index')
    {
    $id = $table->link_id;
    }
    else
    {
    return true;
    }
    // Remove the items.
    return $this->remove($id);
    }

    /**
    * Method to determine if the access level of an item changed.
    *
    * @param string $context The context of the content passed to the plugin.
    * @param JTable $row A JTable object
    * @param boolean $isNew If the content has just been created
    *
    * @return boolean True on success.
    *
    * @since 2.5
    * @throws Exception on database error.
    */
    public function onFinderAfterSave($context, $row, $isNew)
    {
    // We only want to handle articles here
    if ($context == 'com_auction.product' || $context == 'com_auction.form')
    {
    // Check if the access levels are different
    if (!$isNew && $this->old_access != $row->access)
    {
    // Process the change.
    $this->itemAccessChange($row);
    }

    // Reindex the item
    $this->reindex($row->id);
    }

    // Check for access changes in the category
    if ($context == 'com_categories.category')
    {
    // Check if the access levels are different
    if (!$isNew && $this->old_cataccess != $row->access)
    {
    $this->categoryAccessChange($row);
    }
    }

    return true;
    }

    /**
    * Method to reindex the link information for an item that has been saved.
    * This event is fired before the data is actually saved so we are going
    * to queue the item to be indexed later.
    *
    * @param string $context The context of the content passed to the plugin.
    * @param JTable $row A JTable object
    * @param boolean $isNew If the content is just about to be created
    *
    * @return boolean True on success.
    *
    * @since 2.5
    * @throws Exception on database error.
    */
    public function onFinderBeforeSave($context, $row, $isNew)
    {
    // We only want to handle products here
    if ($context == 'com_auction.product' || $context == 'com_auction.form')
    {
    // Query the database for the old access level if the item isn't new
    if (!$isNew)
    {
    $this->checkItemAccess($row);
    }
    }

    // Check for access levels from the category
    if ($context == 'com_categories.category')
    {
    // Query the database for the old access level if the item isn't new
    if (!$isNew)
    {
    $this->checkCategoryAccess($row);
    }
    }

    return true;
    }

    /**
    * Method to update the link information for items that have been changed
    * from outside the edit screen. This is fired when the item is published,
    * unpublished, archived, or unarchived from the list view.
    *
    * @param string $context The context for the content passed to the plugin.
    * @param array $pks A list of primary key ids of the content that has changed state.
    * @param integer $value The value of the state that the content has been changed to.
    *
    * @return void
    *
    * @since 2.5
    */
    public function onFinderChangeState($context, $pks, $value)
    {
    // We only want to handle products here
    if ($context == 'com_auction.product' || $context == 'com_auction.form')
    {
    $this->itemStateChange($pks, $value);
    }
    // Handle when the plugin is disabled
    if ($context == 'com_plugins.plugin' && $value === 0)
    {
    $this->pluginDisable($pks);
    }
    }

    /**
    * Method to index an item. The item must be a FinderIndexerResult object.
    *
    * @param FinderIndexerResult $item The item to index as an FinderIndexerResult object.
    * @param string $format The item format
    *
    * @return void
    *
    * @since 2.5
    * @throws Exception on database error.
    */
    protected function index(FinderIndexerResult $item, $format = 'html')
    {
    // Check if the extension is enabled
    if (JComponentHelper::isEnabled($this->extension) == false)
    {
    return;
    }

    // Initialize the item parameters.
    $registry = new JRegistry;
    $registry->loadString($item->params);
    $item->params = JComponentHelper::getParams('com_auction', true);
    $item->params->merge($registry);

    $registry = new JRegistry;
    $registry->loadString($item->metadata);
    $item->metadata = $registry;

    // Trigger the onContentPrepare event.
    $item->summary = FinderIndexerHelper::prepareContent($item->summary, $item->params);
    $item->body = FinderIndexerHelper::prepareContent($item->body, $item->params);

    // Build the necessary route and path information.
    $item->url = $this->getURL($item->id, $this->extension, $this->layout);
    $item->route = ContentHelperRoute::getProductRoute($item->slug, $item->catslug);
    $item->path = FinderIndexerHelper::getContentPath($item->route);

    // Get the menu title if it exists.
    $title = $this->getItemMenuTitle($item->url);

    // Adjust the title if necessary.
    if (!empty($title) && $this->params->get('use_menu_title', true))
    {
    $item->title = $title;
    }

    // Add the meta-author.
    $item->metaauthor = $item->metadata->get('author');

    // Add the meta-data processing instructions.
    $item->addInstruction(FinderIndexer::META_CONTEXT, 'metakey');
    $item->addInstruction(FinderIndexer::META_CONTEXT, 'metadesc');
    $item->addInstruction(FinderIndexer::META_CONTEXT, 'metaauthor');
    $item->addInstruction(FinderIndexer::META_CONTEXT, 'author');
    $item->addInstruction(FinderIndexer::META_CONTEXT, 'created_by_alias');

    // Translate the state. Products should only be published if the category is published.
    $item->state = $this->translateState($item->state, $item->cat_state);

    // Add the type taxonomy data.
    $item->addTaxonomy('Type', 'Product');

    // Add the author taxonomy data.
    if (!empty($item->author) || !empty($item->created_by_alias))
    {
    $item->addTaxonomy('Vendor', !empty($item->created_by_alias) ? $item->created_by_alias : $item->author);
    }

    // Add the category taxonomy data.
    $item->addTaxonomy('Category', $item->category, $item->cat_state, $item->cat_access);

    // Add the language taxonomy data.
    $item->addTaxonomy('Language', $item->language);

    // Get content extras.
    FinderIndexerHelper::getContentExtras($item);

    // Index the item.
    FinderIndexer::index($item);
    }

    /**
    * Method to setup the indexer to be run.
    *
    * @return boolean True on success.
    *
    * @since 2.5
    */
    protected function setup()
    {
    // Load dependent classes.
    include_once JPATH_SITE . '/components/com_auction/helpers/route.php';

    return true;
    }

    /**
    * Method to get the SQL query used to retrieve the list of content items.
    *
    * @param mixed $sql A JDatabaseQuery object or null.
    *
    * @return JDatabaseQuery A database object.
    *
    * @since 2.5
    */
    protected function getListQuery($sql = null)
    {
    $db = JFactory::getDbo();
    // Check if we can use the supplied SQL query.
    $sql = $sql instanceof JDatabaseQuery ? $sql : $db->getQuery(true);
    $sql->select('a.id, a.title, a.alias, a.introtext AS summary, a.fulltext AS body');
    $sql->select('a.state, a.catid, a.created AS start_date, a.created_by');
    $sql->select('a.created_by_alias, a.modified, a.modified_by, a.attribs AS params');
    $sql->select('a.metakey, a.metadesc, a.metadata, a.language, a.access, a.version, a.ordering');
    $sql->select('a.publish_up AS publish_start_date, a.publish_down AS publish_end_date');
    $sql->select('c.title AS category, c.published AS cat_state, c.access AS cat_access');

    // Handle the alias CASE WHEN portion of the query
    $case_when_item_alias = ' CASE WHEN ';
    $case_when_item_alias .= $sql->charLength('a.alias');
    $case_when_item_alias .= ' THEN ';
    $a_id = $sql->castAsChar('a.id');
    $case_when_item_alias .= $sql->concatenate(array($a_id, 'a.alias'), ':');
    $case_when_item_alias .= ' ELSE ';
    $case_when_item_alias .= $a_id.' END as slug';
    $sql->select($case_when_item_alias);

    $case_when_category_alias = ' CASE WHEN ';
    $case_when_category_alias .= $sql->charLength('c.alias');
    $case_when_category_alias .= ' THEN ';
    $c_id = $sql->castAsChar('c.id');
    $case_when_category_alias .= $sql->concatenate(array($c_id, 'c.alias'), ':');
    $case_when_category_alias .= ' ELSE ';
    $case_when_category_alias .= $c_id.' END as catslug';
    $sql->select($case_when_category_alias);

    $sql->select('u.name AS author');
    $sql->from('#__auction_products AS a');
    $sql->join('LEFT', '#__categories AS c ON c.id = a.catid');
    $sql->join('LEFT', '#__users AS u ON u.id = a.created_by');

    return $sql;
    }
    }

    and this is the related auction.xml file:

    Code:


    plg_finder_auction1
    Joomla! Project
    August 2011
    (C) 2005 - 2012 Open Source Matters. All rights reserved.
    GNU General Public License version 2 or later; see LICENSE.txt
    admin@joomla.org
    www.joomla.org
    2.5.0
    PLG_FINDER_AUCTION1_XML_DESCRIPTION
    script.php

    auction1.php
    index.html


    language/en-GB/en-GB.plg_finder_auction1.ini
    language/en-GB/en-GB.plg_finder_auction1.sys.ini
  • avatar
    • 1
    • 0
    lucid

    Hello, This is my first finder plugin, in fact this is my first ever plugin. Im finding it really hard to work out how to edit the language file once the plugin is installed. Is this possible? I added a new language string to my file and have tried copying the file to both admin and root language folders to no avail.

    What am I doing wrong? Thanks :)))

  • avatar
    • 0
    • 1
    lucid

    Please ignore my last comment, so sorry - I foung a rogue character in my language file that was causing the problem.