Why Index could refuse to work ?

Have you ever seen index which refused to be used even if there is every reason for it to work (from the glance view):

Why on the earth index would not be used you would think, even if MySQL is mentioning it in “possible keys” ? Should you try to force it ?

No Luck. Even Force Index can’t cure the problem. So what could it be ?

Lets take a look at article table:

As you can see article_id is VARCHAR and this is the problem. Comparing String to Number is not going to use the index. Lets check if your guess is right:

Looks much better does not it ?

So why you would define something as VARCHAR and when to refer to it as an INTEGER ? I see no good reason but It is quite frequently seen in applications. Might be designers just think – lets put it VARCHAR so it will fit strings if we need it to, but later decide to stick to numbers.

If you’re storing INTEGER it is much better to define your columns as INT for many reasons, but if you decided to go with VARCHAR (ie you need leading zeroes to be preserved) you should refer to it as a sting in your application by using “”.

You may ask why MySQL can’t use index in this case, simply by converting number to the string and performing index lookup ?
This can’t be done as it would result in wrong result for some queries. The thing is there are multiple strings possible for single integer value – for example for there is number 5 and strings “5”, “05” “0005” “5.0” which all have same numeric value but different strings – this is why simple coversion to the string does not work.

Interesing enough it works other way around – you can refer to integer column as a string in most cases and MySQL will use the index, as for any string there is only one number which matches it. I guess this causes a lot of confusions – having seen it working in one direction people assume it also works in reverse one.

To add confusion MySQL mentions key as “possible keys” while really it has no way to use it for lookup (it can do index scan though). I guess “possible keys” are calculated before type matching is checked.

So be careful to use matching reference types in your applications πŸ™‚

Share this post

Comments (20)

  • Daniel Schneller Reply

    Yes, this is really a great one. I have seen this in an application that performed searches on tables with millions of records and even when specifiying the exact primary key value, it took tens of seconds, not to mention joins.
    From my experience the “let’s take a varchar” approach is often taken by people who come from simpler systems, e. g. (in my example) Foxpro, where they stored nearly everything as character values.

    September 8, 2006 at 11:44 am
  • peter Reply

    I did not mention it in the article but a lot of such problems also come from comparing incompatible columns, ie varchar to integer which may require closer look to figure out. I would really like MySQL EXPLAIN to issue warning in such case or something similar so it is easier to track it down.

    Yes this mistake is usually done by people having limited experience, at least with MySQL.

    September 8, 2006 at 12:25 pm
  • parvesh Reply

    Sorry for commenting on an old post. But recently, I saw another interesting thing. You are right about indexes not being used in this case, but it works the other way round. If the column is an integer and the query is something like

    SELECT * FROM article WHERE article_id=”10″;

    My first intuition was that this will also fail, but MySQL is smart enough πŸ™‚

    July 3, 2007 at 5:14 am
  • peter Reply

    No this one is actually fine, because there is only one number for given string “10” is 10 but it is not the case other way around 10 can correspond to “10” “010” “10.0” and a lot of other strings.

    July 3, 2007 at 7:08 am
  • Jonathan Reply

    Thank you, this is a great post. It just caught me in a system!

    November 24, 2008 at 11:49 pm
  • Suman Reply

    What will happen if the table is:
    CREATE TABLE article (
    article_id int(5) NOT NULL,
    dummy varchar(255) NOT NULL DEFAULT ‘dummy’,
    PRIMARY KEY (article_id)
    And query is:
    SELECT * FROM article force INDEX (PRIMARY) WHERE article_id=’10’; (single quotes)
    Will the index be used?

    February 19, 2009 at 6:47 am
  • Suman Reply

    Sorry this has been answered above, I missed it.

    February 19, 2009 at 6:50 am
  • Peter Mouland Reply

    Hi, any other reasons why FORCE INDEX is ignored? I have a table with 100 columns which about 25% are integers and most of those have indexes on them. A varchar column is being used as to left join to another table which also has an (btree) index on it, but for some reason it is not being used. EXPLAIN also doesn’t tell me about any possible keys. The table has about 100,000 rows, so not huge but be steadily growing. any ideas?? thanks,pete

    June 15, 2009 at 3:11 am
  • peter Reply


    This may happen if MySQL can’t use this index for the query.
    FORCE INDEX only works if it is possible to use index but MySQL choses not to do that.

    June 15, 2009 at 8:55 pm
  • mylesmg Reply

    One reason not in this post, but in another post is that if you are joining tables, the join may not use an index if the tables are of different types (e.g. joining an InnoDB table to a Memory table). Simply changing the memory to innodb or vice versa will eliminate the issue.

    December 9, 2009 at 1:13 pm
  • Mark Cotner Reply

    This came up in a google search for a problem I was having where even if I forced I couldn’t get it to use an index. I quietly read the ENTIRE article, assuming this couldn’t possibly be it . . . surely we’re smarter than that.

    Went back and checked anyway, sure enough . . . varchar for a universal_id field. I guess some of them must have characters in them(we don’t assign them). Added quotes to the where clause and sped it up 300x.

    Thanks for posting this. It may seem obvious, but I surely didn’t catch it.

    January 13, 2012 at 1:33 pm
  • Pascal Reply

    The same problem i have, at the only difference, that my column is Already an INT….
    What should i do?


    July 11, 2012 at 12:21 pm
  • Yakovenko Stepan Reply

    Would you be so kind to take a look here: http://stackoverflow.com/questions/13691422/mysql-with-huge-tables-2-queries-faster-then-1-index-not-used ? Can this happen to bigint field?

    December 6, 2012 at 8:56 am
  • sivamurugan Reply

    Thanks a lot Peter Zaitsev.. It saves me in getting late of 6 million records…

    February 21, 2014 at 10:12 am
  • grandman Reply

    Thanks Peter.

    Example column field is type INT, name TestID, (it’s also an index). Is it then better to search as:
    WHERE table_name.TestID = ‘5’
    WHERE table_name.TestID = 5
    I know that you wrote:

    “Interesing enough it works other way around – you can refer to integer column as a string in most cases and MySQL will use the index”

    What’s best solution for this (what is working faster)? I have possibility to skip quotes. Should I?

    October 16, 2015 at 8:59 am
  • Pepijn Reply

    We are actually experiencing this with a Magento 1 shop right now.
    Magento stores the increment_id as string and with over 2 million orders in the sales_flat_order table queries were starting to take exceptionally long.

    time mysql -e “SELECT * FROM sales_flat_order WHERE increment_id=202405845”
    real 0m30.920s

    time mysql -e “SELECT * FROM sales_flat_order WHERE increment_id=’202405845′”
    real 0m0.006s

    Yes, that’s almost 31 SECONDS DIFFERENCE! There must be a bug somewhere in Magento/Zend Framework 1 which determines if the value needs quotes and decides against it, since it’s a number anyway, or we have some wonky third-party module doing this.

    Maybe a lesson learned: don’t use only integers in Magento order numbers! πŸ™‚

    November 17, 2017 at 6:18 pm
  • Tymoteusz Motylewski Reply

    Is it possible to make MySQL log queries passing wrong datatypes? Or maybe there is other way to find issues like that?

    November 22, 2017 at 3:40 am
  • OG Reply

    it’s an old post but let me get it right.
    MySQL does not use the index because of the unknown results: “for example for there is number 5 and strings β€œ5”, β€œ05” β€œ0005” β€œ5.0” which all have same numeric value but different strings”…
    but what is really happened is that the query run, scan the entire table (full table scan) and give me the “wrong” results anyway !
    so for my opinion, data conversion should not impact index comparison because it impact performance.

    February 13, 2019 at 6:37 am
  • Paolo Reply

    Thanks for this tip Peter, this will really help me !

    July 21, 2019 at 2:17 pm
  • Lee Crusher Reply

    This 13 year old article just saved my life. Thank you!

    August 23, 2019 at 9:42 am

Leave a Reply