The anatomy of smart search in Joomla 5 Part 2: Creating a plugin I.
In the previous article, we got acquainted with the capabilities of the Joomla smart search component, talked about the parameters and configuration of scheduled indexing using CRON. Let's start creating the code for our own plugin.
List of resources
Before starting the technical part, I will mention some articles that directly address the main topic. As well as articles that generally cover the creation and/or updating of a plugin for the modern architecture of Joomla 4 / Joomla 5. Next, I will assume that the reader has read them and generally has an idea of how to make a working plugin for Joomla:
- Creating a Smart Search plugin - official Joomla documentation. It is for Joomla 3, but most of the provisions remained true for Joomla 4 / Joomla 5
- Developing a Smart Search Plugin an article from Joomla Community Magazine in 2012.
- The book Joomla Extensions Development by Nicholas Dionysopoulos that covers the development of Joomla! extensions under Joomla versions 4 and 5.
- The Database section on the new documentation portal manual.joomla.org - for Joomla 4 and Joomla 5.
The technical part. Development of the Joomla 5 smart search plugin
The smart search component works with data provider plugins, whose main task remains the same - to select data and give it to the component for indexing. But over time, reindexing tasks also fell into the plugin's area of responsibility. In the article, we will assume that we run content indexing manually from the admin panel. The work from the CLI is visually different, but its essence remains the same.
For experienced developers, I will say that the search plugin extends the \Joomla\Component\Finder\Administrator\Indexer\Adapter
class, the class file is located in administrator/components/com_finder/src/Indexer/Adapter.php. Well, then they will figure it out for themselves. Also, as a sample, you can study the Joomla core smart search plugins - for articles, categories, contacts, tags, etc. - in the plugins/finder folder. I worked on a smart search plugin for JoomShopping and SW JProjects components, so the class names and some nuances will be associated with them. I will show most of it using the example of JoomShopping. The solution to the issue of multilingualism is based on the example of SW JProjects.
The file structure of the smart search plugin
The file structure of the smart search plugin for Joomshopping does not differ from the typical one:
File services/provider.php
The file provider.php allows you to register a plugin in a Joomla DI container and allows you to access plugin methods from the outside using MVCFactory.
<?php
/**
* @package Joomla.Plugin
* @subpackage Finder.Wtjoomshoppingfinder
*
* @copyright (C) 2023 Open Source Matters, Inc. <https://www.joomla.org>
* @license GNU General Public License version 2 or later; see LICENSE.txt
*/
\defined('_JEXEC') or die;
use Joomla\CMS\Extension\PluginInterface;
use Joomla\CMS\Factory;
use Joomla\CMS\Plugin\PluginHelper;
use Joomla\Database\DatabaseInterface;
use Joomla\DI\Container;
use Joomla\DI\ServiceProviderInterface;
use Joomla\Event\DispatcherInterface;
use Joomla\Plugin\Finder\Wtjoomshoppingfinder\Extension\Wtjoomshoppingfinder;
return new class () implements ServiceProviderInterface {
/**
* Registers the service provider with a DI container.
*
* @param Container $container The DI container.
*
* @return void
*
* @since 4.3.0
*/
public function register(Container $container)
{
$container->set(
PluginInterface::class,
function (Container $container) {
$plugin = new Wtjoomshoppingfinder(
$container->get(DispatcherInterface::class),
(array) PluginHelper::getPlugin('finder', 'wtjoomshoppingfinder')
);
$plugin->setApplication(Factory::getApplication());
// Our plugin uses DatabaseTrait, so the setDatabase() method appeared
// If it is not present, then we use only setApplication().
$plugin->setDatabase($container->get(DatabaseInterface::class));
return $plugin;
}
);
}
};
Plugin class file
This is the file that contains the main working code of your plugin. It should be located in the src/Extension folder. In my case, the plugin class \Joomla\Plugin\Finder\Wtjoomshoppingfinder\Extension\Wtjoomshoppingfinder
is in the file plugins/finder/wtjoomshoppingfinder/src/Extension/Wtjoomshoppingfinder.php. The namespace of the plugin is Joomla\Plugin\Finder\Wtjoomshoppingfinder\Extension
.
There is a minimal set of class properties and methods required for operation (they are accessed, including by the parent Adapter class).
The minimum required properties of the class
$extension
- is the name of your component, which defines the type of your content. For example, com_content. In my case, this iscom_jshopping
.$context
- is a unique identifier for the plugin, it sets the context of indexing, in which the plugin will be accessed. In fact, this is the name of the plugin class (element). In our case,Wtjoomshoppingfinder
.$layout
- is the name of the output layout for the search results element. This layout is used when displaying search results. For example, if the$layout
parameter is set toarticle
, then the default view mode will search for a layout file named default_article.php when you need to display a search result of this type. If such a file is not found, then a layout file with the name default_result.php will be used instead. The output layouts with HTML layout are located in components/com_finder/tmpl/search. However, we should place our layouts as overrides - in the html template folder - templates/YOUR_TEMPLATE/html/com_finder/search. In my case, I named the layoutproduct
, and the file is called default_product.php.$table
- is the name of the table in the database that we are accessing to get data, for example,#__content
. In my case, the main table with JoomShopping products is called#__jshopping_products
.$state_field
- is the name of the field in the database table that is responsible for whether the indexed element is published or not. By default, this field is calledstate
. However, in the case of JoomShopping, this field is calledproduct_publish
.
<?php
// So far, only the namespaces used in the example are listed here.
use Joomla\Component\Finder\Administrator\Indexer\Adapter;
use Joomla\Event\SubscriberInterface;
use Joomla\Database\DatabaseAwareTrait;
\defined('_JEXEC') or die;
final class Wtjoomshoppingfinder extends Adapter implements SubscriberInterface
{
// We want to use a setDatabase() method in provider.php and getDatabase() method in our plugin.
// So let's...
use DatabaseAwareTrait;
/**
* The unique identifier of the plugin. You can specify the name of the class.
*
* @var string
* @since 2.5
*/
protected $context = 'Wtjoomshoppingfinder';
/**
* For which component are we indexing the content
*
* @var string
* @since 2.5
*/
protected $extension = 'com_jshopping';
/**
*
* The name of the suffix for the search results output sublayout.
* If it is an "article", then the file name will be "default_article.php "
*
* @var string
* @since 2.5
*/
protected $layout = 'product';
/**
* The type of content being indexed. The user can
* search only among products, only among tags, only
* among articles, etc.
*
* @var string
* @since 2.5
*/
protected $type_title = 'Product';
/**
* A field in the database that stores the flag whether the item is published or not.
* Default is "state"
*
* @var string
* @since 1.0.0
*/
protected $state_field = 'product_publish';
/**
* The name of the database table.
*
* @var string
* @since 2.5
*/
protected $table = '#__jshopping_products';
/**
* Whether to load the plugin's language files when initializing the class.
*
* @var boolean
* @since 3.1
*/
protected $autoloadLanguage = true;
/**
* Language tag for JoomShopping products.
* Non-standard class property, only we need
* and only for JoomShopping.
*
* @var string
* @since 3.1
*/
protected string $languageTag = '';
}
The minimum required methods of the class
setup() : bool
- is a method for pre-configuring the plugin, connecting libraries, etc. The method is called during reindexing (thereindex()
method), on theonBeforeIndex
event. The method must return true, otherwise indexing will be interrupted.index() : void
- is the method to start indexing itself. It collects an object of the desired structure from raw SQL query data, which is then passed to the\Joomla\Component\Finder\Administrator\Indexer\Indexer
class for indexing. The method is run for each indexed element. The method argument is$item
- the result of a query to the database, formatted in the\Joomla\Component\Finder\Administrator\Indexer\Result
class.getListQuery() : Joomla\Database\DatabaseQuery
- is a method for getting a list of indexed items…
... and here we start to dive into the details, since the getListQuery()
method is not really mandatory, despite the fact that both the documentation and most articles talk about it.
Dive into the details. The data structure of the indexed element.
It's amazing how many times some information or idea sometimes passes by us in a circle before we notice and realize it! Many things, being in front of our eyes for more than one year, still do not reach awareness, and our attention focuses on them only after years of experience.
In connection with Joomla, for some reason, the vision does not immediately come that its components assume some kind of common architecture characteristic of Joomla (although this is an obvious fact). Including at the level of the database table structure. Let's look at some fields of the Joomla content table. I will make a reservation that specific column names are not so important to us (you can always query SELECT `name` as `title`
), how much is the data structure for one indexed element:
id
- autoincrementasset_id
- the id of the entry in the#__assets table
, where the access rights of groups and users are stored for each element of the site: articles, products, menus, modules, plugins, and everything else. Joomla uses the Access Control List (ACL) pattern.title
- the element titlelanguage
- the element languageintrotext
- introductory text or a brief visible description of the elementfulltext
- the full text of the item, the full description of the product, etc.state
- the logical flag responsible for the publication status: whether the item is published or not.catid
- the ID of the item category. Joomla doesn't just have "site pages" like in other CMS. There are content entities (articles, contacts, products, etc.) that must belong to some categories.created
- the date the item was created.access
- access rights group id (unauthorized site users (guests), all, registered, etc.)metakey
- meta keywords for the element. Yes, since 2009 they are not used by Google. But in Joomla, they historically remain, since this field is used in the similar articles module to search for actually similar articles using specified keywords.metadesc
- the element meta descriptionpublish_up
andpublish_down
- the date of the start of publication and de-publication of the element. This is more of an option, but it is found in many components.
If we compare the tables #__content
(Joomla articles), #__contact_details
(contact component), #__tags
(Joomla tags), #__categories
(Joomla category component), then we will find almost all the listed data types everywhere.
If the component for which smart search plugins are created followed the "Joomla way" and inherits its architecture, then you can do with a minimum of methods in the plugin class. If the developers decide not to look for easy ways and go their own way, then you will have to go the hard way, redefining almost all the methods of the
Adapter
class.
getListQuery() method
This method is called in 3 cases:
- The
getContentCount()
method of theAdapter
class is to get the number of indexed items (how many articles in total, how many products in total, etc.).
- The
getItem($id)
method of theAdapter
class is to get a specific indexed element by its id. ThegetItem()
method, in turn, is called in thereindex($id)
method - during reindexing. - The
getItems($offset, $limit, $query = null)
method of theAdapter
class is a method for getting a list of indexed elements. Offset and limit are set based on the component settings - how many indexed elements should be included in the "bundle".
Let's look at an example of implementation in Joomla core plugins:
<?php
// This is an example code from the Joomla smart search plugin for Joomla Articles.
use Joomla\Database\DatabaseQuery;
/**
* Method to get the SQL query used to retrieve the list of content items.
*
* @param mixed $query A DatabaseQuery object or null.
*
* @return DatabaseQuery A database object.
*
* @since 2.5
*/
protected function getListQuery($query = null)
{
$db = $this->getDatabase();
// Check if we can use the supplied SQL query.
$query = $query instanceof DatabaseQuery ? $query : $db->getQuery(true)
->select('a.id, a.title, a.alias, a.introtext AS summary, a.fulltext AS body')
->select('a.images')
->select('a.state, a.catid, a.created AS start_date, a.created_by')
->select('a.created_by_alias, a.modified, a.modified_by, a.attribs AS params')
->select('a.metakey, a.metadesc, a.metadata, a.language, a.access, a.version, a.ordering')
->select('a.publish_up AS publish_start_date, a.publish_down AS publish_end_date')
->select('c.title AS category, c.published AS cat_state, c.access AS cat_access');
// Handle the alias CASE WHEN portion of the query
$case_when_item_alias = ' CASE WHEN ';
$case_when_item_alias .= $query->charLength('a.alias', '!=', '0');
$case_when_item_alias .= ' THEN ';
$a_id = $query->castAsChar('a.id');
$case_when_item_alias .= $query->concatenate([$a_id, 'a.alias'], ':');
$case_when_item_alias .= ' ELSE ';
$case_when_item_alias .= $a_id . ' END as slug';
$query->select($case_when_item_alias);
$case_when_category_alias = ' CASE WHEN ';
$case_when_category_alias .= $query->charLength('c.alias', '!=', '0');
$case_when_category_alias .= ' THEN ';
$c_id = $query->castAsChar('c.id');
$case_when_category_alias .= $query->concatenate([$c_id, 'c.alias'], ':');
$case_when_category_alias .= ' ELSE ';
$case_when_category_alias .= $c_id . ' END as catslug';
$query->select($case_when_category_alias)
->select('u.name AS author')
->from('#__content AS a')
->join('LEFT', '#__categories AS c ON c.id = a.catid')
->join('LEFT', '#__users AS u ON u.id = a.created_by');
return $query;
}
The getListQuery()
method returns a DatabaseQuery
object, an object of the query constructor, where the name of the table and fields for selection are already specified. Work with it continues in the methods that call it.
If getListQuery()
is called from getContentCount()
in the DatabaseQuery $query
object, the set values for select are replaced with COUNT(*)
.
If getListQuery()
is called from getItem($id)
, the condition $query->where('a.id = ' . (int) $id)
and only a specific element is selected. And already here we see that the parent Adapter
class contains the table name in the query as a.*
. This means that we should also use these prefixes in our implementation of the getListQuery()
method.
In the case of calling getListQuery()
from getItems()
, $offset
and $limit
are added to the query that we have constructed in order to move through the list of elements for indexing.
Summary: getListQuery()
- must contain a "work piece" for three different SQL queries. And there is nothing particularly difficult about implementing Joomla here. But, if necessary, you can implement 3 methods yourself without creating getListQuery()
.
Non Joomla way: In the case of JoomShopping, I came across the fact that a product can have several categories and historically the category id (catid
) component for the product was stored in a separate table. At the same time, for many years it was not possible to specify the main category for the product. Upon receipt of the product category, a query was sent to the table with categories, where just the first query result was taken, sorted by default category id - i.e. ascending. If we changed categories when editing a product, then the main product category was the one with the lower id
number. The URL of the product was based on it and the product could jump from one category to another.
But, almost 2 years ago, this JoomShopping behavior was fixed. Since the component has a long history, a large audience and cannot just break backward compatibility, the fix was made optional. The ability to specify the main category for the product must be enabled in the component settings. Then the main_category_id
will be filled in the table with the products.
But this functionality is disabled by default. And in the smart search plugin, we need to get the parameters of the JoomShopping component, see if the option to specify the main product category is enabled (and it may be enabled recently and the main category for some products is not specified - also a nuance ...) and generate an SQL query to receive the product(s) based on the component parameters: either a simple query where we add the main_category_id
field, or a JOIN
request to get the category id
in the old wrong way.
Immediately, the nuance of multilingualism comes to the fore in this request. According to the Joomla way, a separate element is created for each language of the site and associations are set up between them. So, for the Russian language - one article. The same article in English is being created separately. Then we connect them with each other using language associations and when switching the language on the Joomla frontend, we will be redirected from one article to another.
This is not how it is done in JoomShopping: data for all languages is stored in the same table with products (Ok). Adding data for other languages is done by adding columns with the suffix of these languages (hmm...). That is, we do not have just a title or name field in the database. But there are fields name_ru-RU
, name_en-GB
, etc.
At the same time, we need to design a universal SQL query so that it can be indexed from both the admin panel and the CLI. At the same time, choosing the indexing language when launching the CLI using CRON is also a task. I admit, at the time of writing this article, I have postponed a full-fledged solution to this problem for the time being. The language is selected using our own getLangTag()
method, where we either take the main language from the JoomShopping parameters, or the default language of the site. That is, so far this solution is only for a monolingual site. The search in different languages will not work yet.
However, 3 months later I solved this problem, but already in the smart search plugin for SW JProjects component. I will tell you about the solution further.
In the meantime, let's look at what happened for JoomShopping
<?php
use Joomla\Database\DatabaseQuery;
/**
* Method to get the SQL query used to retrieve the list of content items.
*
* @param mixed $query A DatabaseQuery object or null.
*
* @return DatabaseQuery A database object.
*
* @since 2.5
*/
protected function getListQuery($query = null): DatabaseQuery
{
$db = $this->db;
$tag = $this->getLangTag();
// Check if we can use the supplied SQL query.
$query = ($query instanceof DatabaseQuery) ? $query : $db->getQuery(true);
$query->select(
[
'prod.product_ean',
'prod.manufacturer_code',
'prod.product_old_price',
'prod.product_price',
'prod.product_buy_price',
'prod.min_price',
'prod.product_weight',
]);
// Columns with ... AS
$query->select(
$db->quoteName(
[
'prod.product_id',
'prod.name_' . $tag,
'prod.alias_' . $tag,
'prod.description_' . $tag,
'prod.short_description_' . $tag,
'prod.product_date_added',
'prod.product_publish',
'prod.image',
'cat.name_' . $tag,
],
[ // ... AS ...
'slug',
'title',
'alias',
'body',
'summary',
'created',
'state',
'image',
'category',
]
)
);
$query->from($db->quoteName('#__jshopping_products', 'prod'))
->where($db->quoteName('prod.product_publish') . ' = ' . $db->quote(1))
->where($db->quoteName('cat.category_publish') . ' = ' . $db->quote(1));
/**
* If the JoomShopping option "Use the main category for the product" is available and enabled,
* then the product has the main_category_id field.
* If not, use the old JoomShopping approach - take the 1st category id from the table #__jshopping_products_to_categories
* to do this, we will make a subquery, since category_id should only be 1.
*/
if (property_exists($this, 'jshopConfig')
&& !empty($this->jshopConfig)
&& $this->jshopConfig->product_use_main_category_id == 1)
{
$query->select($db->quoteName('prod.main_category_id', 'catslug'));
$query->join('LEFT', $db->quoteName('#__jshopping_categories', 'cat') . ' ON ' . $db->quoteName('cat.category_id') . ' = ' . $db->quoteName('prod.main_category_id'));
}
else
{
$query->select($db->quoteName('cat.category_id', 'catslug'));
// Create a subquery for the sub-items list
$subQuery = $db->getQuery(true)
->select($db->quoteName('pr_cat.product_id'))
->select('MIN(' . $db->quoteName('pr_cat.category_id') . ') AS ' . $db->quoteName('catslug'))
->from($db->quoteName('#__jshopping_products_to_categories', 'pr_cat'))
->group($db->quoteName('product_id'));
$query->join('LEFT', '(' . $subQuery . ') AS ' . $db->quoteName('subquery') . ' ON ' . $db->quoteName('subquery.product_id') . ' = ' . $db->quoteName('prod.product_id'));
$query->join('LEFT', $db->quoteName('#__jshopping_categories', 'cat'), $db->quoteName('cat.category_id') . ' = ' . $db->quoteName('subquery.catslug'));
}
return $query;
}
Check point
We created a method to query the database from Joomla and learned a lot about how the smart search plugin works.
In the next article, we will create a method for indexing content and complete the creation of plugin. We will also get acquainted with how indexed items are stored in the database and understand why this is important and solve the problem of indexing content for multilingual components with a non-standard implementation of multilingualism.
Some articles published on the Joomla Community Magazine represent the personal opinion or experience of the Author on the specific topic and might not be aligned to the official position of the Joomla Project
By accepting you will be accessing a service provided by a third-party external to https://magazine.joomla.org/
Comments