Catalog post processing
Product catalog updates usually means some post jobs needs to be started, e.g. create product feeds used by affiliates. Mercury supports post processes to be configured which will start after a product catalog has been updated.
Design
Functional
Post processes can be configured for the catalog post process. The catalog post process retrieves all of the products of the site, loop through all products and will process each product with the configured post processes (see the diagram below).
Design choices
Catalog post processing can be started by invoking the /mercury/catalog/catalog/updated REST call. This starts an orchestrating Sitecore job that invokes the mercury.catalog.createPostCatalogImportJobs pipeline. Processors can be hooked into this pipeline and add Jobs to the PostProcessingJobs of the pipeline result.
Make sure the jobs are not started when you add them to the pipeline result, because they will be started sequentially by the coordinator.
The coordinating job will show up in the job viewer as: Post Catalog Import Job Runner. If it gets started multiple times, it will get queued.
The /mercury/catalog/catalog/updated REST call returns a processId that can be used to query the status of the catalog post processing job, by passing it into the mercury/catalog/catalog/status?processId=6f504b90-c3db-46b4-b4ba-4a5578d04a05 REST call.
There are two default processing jobs:
SiteSpecificIndexRebuildJob This rebuilds the indexes for the given site.
CatalogPostProcessingJob This job allows you to add a processor that operates on product level. This is mostly to improve performance. This Job reads all the products from the index and feeds them into the registered processors at once.
CatalogPostProcessingJob
Commerce catalog as data source
The search index will be used as the product data source instead of the Commerce catalog. With this choice, the data used for lister is equal to the data used in the post processing (which especially is important for the canonical mapping).
Note that the bottleneck isn't the retrieval of products from the index but the retrieval of the product item from Sitecore.
Abort process on error
Usually each post process will result, based on the catalog data, in output data that is used in the website or externally.Whenever an error occurred in one of the post processes the catalog process (or whole process) will be aborted. This to prevent inconsistent output data.
Model
After the productcatalog:updated event has been received, the CatalogPostProcessorHandler retrieves a list of post processors from the IProcessorRepository and starts a Sitecore job CatalogPostProcessorJob with the list of retrieved processors. This job uses the ICatalogPostProcessorFactory to create a catalog post processor of type ICatalogPostProcessor. Both the catalog post processor and the list of post processors can be configured in the configuration.
Catalog post processing class diagram
Catalog post processing sequence diagram
The CatalogPostProcessor is the core of the catalog post processing job. It retrieves all products, and processes each product with the list of processors.
A list of product ID's are retrieved using the ICommerceServerProductRepository. Using the product ID's the matching Sitecore items are retrieved. A ProductItemContainer is created to wrap the product item and its variants. The variants are from type ProductWithVariant which wraps both the variant and the product which it belongs to.
Before processing each product, the list products are filtered using the PostProcessFilterServiceProvider. This calls a Sitecore pipeline and checks if a product should be included or not.
Catalog post processor class diagram
Catalog post processor sequence diagram
Post processor high level overview
Each post processor has to implement the IProcessor interface. The post processor retrieves the product to be processed from the CatalogPostProcessor and uses a builder to process the product. The builder processes the product and maintains the resulting list of items. After all products are processed the resulting list of items are saved using a storage object.
Overview of processors in Mercury
Configuration
Mercury configures an event handler for the productcatalog:updated event which starts the catalog post processing.
The type of the catalog post processor type is configured in the following configuration:
<sitecore>
<mercury.Catalog>
<catalogPostProcessor resolve="true" type="Mercury.Foundation.Catalog.PostProcess.CatalogPostProcessor, Mercury.Foundation.Catalog">
<processors hint="skip">
</processors>
</catalogPostProcessor>
</mercury.Catalog>
</sitecore>
The list of post processors are configured in the catalogPostProcessor configuration section.
For example the processor for building product feeds is configured like:
<sitecore>
<mercury.Catalog>
<catalogPostProcessor resolve="true" type="Mercury.Foundation.Catalog.PostProcess.CatalogPostProcessor, Mercury.Foundation.Catalog">
<processors>
<processor resolve="true" type="Mercury.Feature.ProductFeeds.PostProcessors.BuildProductFeedProcessor, Mercury.Feature.ProductFeeds" />
</processors>
</catalogPostProcessor>
</mercury.Catalog>
</sitecore>
Extension points
Pipelines
The following pipelines are offered as extension points for customization:
mercury.catalog.createPostCatalogImportJobs.mercury.catalogpostprocessing.createSearchQuery.
mercury.catalog.createPostCatalogImportJobs
Allows you to register Jobs that need to be executed during catalog post processing.
mercury.catalogpostprocessing.createSearchQuery
This pipeline creates the query that retrieves the products. Extra processors can be added for additional filtering.
References
Developer guide SEO mapping
Developer guide canonical mapping
Developer guide sitemap
Developer guide product feeds