Have you ever thought about how many sales you are losing because of the slow page speed?
According to statistics, 57% of potential customers leave the website if it loads for more than 3 seconds. 80% of visitors never come back to slow websites.
But what to do if your catalogue is over 1 million products? How to make your customer journey in your online store fast and easy? Each time a visitor surfs your page, he or she puts the load on the database where all your products are stored. The default CS-Cart uses MySQL as a database to store data. Although it does have a great performance, in some cases, it is better to use for small or medium applications.
The integration of a database search engine could significantly enhance performance of a website with massive product counts. By equipping your database with a powerful search engine, the performance of your online store can dramatically increase.
Contemporary search engines provide sophisticated features like suggestions, full-text, faceted, fuzzy search, etc. for even more accurate and relevant results.
However, there are about 20 search engines to choose from. Which technology to choose? According to the Database Engines ranking, ElasticSearch is the most popular enterprise search engine followed by Apache Solr, also based on Lucene
So, if you are looking for a reliable and efficient solution for your web application, we would like to recommend one of the following three: Elasticsearch, Solr, or Sphinx.
What is an effective search?
The primary aim of the search is to retrieve the most relevant matches to the user’s queries, excluding other general content from the website. There are subtle differences between ElasticSearch, Solr, and Sphinx. They all serve their main purpose – to offer a fast and effective search.
Among the features that you can get along with the modern search engines, the most popular are:
- Fuzzy search – search with automatic correcting typos and misspellings.
- Geospatial search – search with automatic detection of the object location according to its latitude and longitude.
- Sorting feature. It can include:
- Search by ranges (price, dates, sizes, etc.),
- Filters (by popularity, date, price) and filtering (including only desirable parameters).
- Near real-time indexing – a search that considers frequent updates in the database like availability in stock, dynamic pricing, supply positioning, product description details, etc.
- Faceted search – count of attributes like size, or color (for example, brand, type, purpose, screen resolution, matrix, diagonal, HDD, capacity)
- Full-text – search by single words, multiple forms of a word, sentence, multiple forms of a sentence
- Highlighting – visual indication of the words entered in the search bar, aka snippets
- Scalability – ability to expand database while it gets bigger without hampering the search speed
- Storage – search engine database as an additional source of storage
- Visualization – a user interface with reports that visualize search in a form of charts, tables, and maps helping to understand how data is consumed. The visualization may be used for search trend analysis or a timely discovery of security breaches.
- Security – ability to identify threats: web server problems, broken links, unauthorized access attempts represented on a dashboard.
What you can get from implementing ElasticSearch?
ElasticSearch is a distributed open-source software and search-and-analytics engine able to resolve an increasing number of use cases. It can rapidly index frequently changing data almost instantly (in less than 1 sec). It is rational to implement it in projects where a database is constantly updating.
Uber applies ElasticSearch. It helps to aggregate business metrics on dynamic (surge) pricing and supply positioning in real-time. Thanks to ElasticSearch implementation, the Uber website is able to handle more than 1,000 queries per second at peak time.
Expedia, one of the biggest hotel and airline ticket aggregators, provides searching through up to 1TB a day with 300K events per second. With the help of ElasticSearch, they managed to improve their customers’ booking experience.
ElasticSearch has the ability to scale up while your database becomes bigger, so the search speed does not slow down.
However, there is a disadvantage with ElasticSearch: it actually does not have the “Did You Mean?” feature.
Getting Sphinx up and running
Sphinx has a vast experience with faceted search capabilities. Youku Tudou, China’s biggest video site, uses Sphinx for the faceted search for content delivered to over 400 million users per month, with peak volumes of 15,000 queries per second. With the help of Sphinx, Craigslist handles over 300 million queries a day. It has more than 50 billion page views per month.
Infegy uses Sphinx to index 22+ billion Twitter, Facebook, and assorted blog posts to serve insightful social media monitoring and analytic queries. One of the examples is Boardreader, where Sphinx indexes up to 16 billion documents across 37 machines.
It makes sense to integrate Sphinx in case you need only generic functions without additional features like data visualization and analysis. It’s quite fast and powerful for indexing and querying huge volumes of documents using limited computing resources, unlike ElasticSearch which consumes a lot of memory.
Hooking up Solr
Solr is not as fast as ElasticSearch and suits better the databases where static data that does not require frequent changing prevails. The reason lies in caches that are global in Solr meaning that even the slightest update would require re-indexing and refresh. This is usually a time-consuming process. Despite those disadvantages, Zappos effectively applies Solr on their website for search and navigation across 150,000 styles of their items.
Integrating a search engine with CS-Cart
We have tested many search engines during our practice with various projects. As of now, in most cases, we integrate ElasticSearch in CS-Cart, as this search engine has proven it has the best qualities for speedy scanning heavy databases. It is quick, flexible and simple to work with, and provides not only quick and relevant search capabilities but can also be a means of storage, by itself. It is convenient to use it for search in logs to quickly localize a problem with applications and provides a full visual picture of what is going on in your web page in real-time.
The table shows all the advantages and bottlenecks of the search engines. As you can see, all of them are quite similar to searching capabilities. If you already have a project on Sphinx or Solr, it is better to keep it on them and not rush into transferring it to Elasticsearch. Whatever your case is, a good practice is to rely on the experience of web developers who have the best knowledge of technical peculiarities of your project and have already worked with search engines. All eCommerce projects are individual, but we carefully analyze each request to come up with the most appropriate solution for your task.