September 17, 2014

Sphinx search performance optimization: multi-threaded search

Queries in MySQL, Sphinx and many other database or search engines are typically single-threaded. That is when you issue a single query on your brand new r910 with 32 CPU cores and 16 disks, the maximum that is going to be used to process this query at any given point is 1 CPU core and […]

Sphinx search performance optimization: attribute-based filters

One of the most common causes of a poor Sphinx search performance I find our customers face is misuse of search filters. In this article I will cover how Sphinx attributes (which are normally used for filtering) work, when they are a good idea to use and what to do when they are not, but […]

The Optimization That (Often) Isn’t: Index Merge Intersection

Prior to version 5.0, MySQL could only use one index per table in a given query without any exceptions; folks that didn’t understand this limitation would often have tables with lots of single-column indexes on columns which commonly appeared in their WHERE clauses, and they’d wonder why the EXPLAIN plan for a given SELECT would […]

Joining many tables in MySQL – optimizer_search_depth

Working on customer case today I ran into interesting problem – query joining about 20 tables (thank you ORM by joining all tables connected with foreign keys just in case) which would take 5 seconds even though in the read less than 1000 rows and doing it completely in memory. The plan optimizer picked was […]

Understanding Performance Optimization Terminology

There are few terms you need to have a good understanding if you’re working with high volume systems. I commonly see these mixed and people not understanding the difference between them. Performance – The performance comes down to performing up to users expectations (or expectations of your boss) which drills down to two important metrics […]

Goal driven performance optimization

When your goal is to optimize application performance it is very important to understand what goal do you really have. If you do not have a good understanding of the goal your performance optimization effort may well still bring its results but you may waste a lot of time before you reach same results as […]

Announce: Front End Performance Optimization

I guess many of you know us and so our company for MySQL related services. It is true this is majority of our business at this point but it is far from everything. Our goal in reality is to help people to build and operate quality systems, typically web sites, which means we help customers […]

Web Site Optimization: FrontEnd and BackEnd

I spent Monday and Tuesday this week on Velocity Conference It was quite interesting event worth attending and it was very good to see the problems in this are going beyond Apache, PHP, Memcache and MySQL. A lot of talks on this conference was focusing on what is called “FrontEnd”. The meaning of Frontend is […]

Using GROUP BY WITH ROLLUP for Reporting Performance Optimization

Quite typical query for reporting applications is to find top X values. If you analyze Web Site logs you would look at most popular web pages or search engine keywords which bring you most of the traffic. If you’re looking at ecommerce reporting you may be interested in best selling product or top sales people. […]

Sphinx: Going Beyond full text search

I’ve already wrote a few times about various projects using Sphinx with MySQL for scalable Full Text Search applications. For example on BoardReader we’re using this combination to build search against over 1 billion of forum posts totaling over 1.5TB of data handling hundreds of thousands of search queries per day. The count of forum […]