How to monitor ALTER TABLE progress in MySQL

While working on a recent support issue as a Percona Support Engineer,  I got one question from a customer asking how to monitor ALTER TABLE progress. Actually, for MySQL 5.5 and prior versions, it’s quite difficult to ALTER the table in a running production environment especially for large tables (with millions of records). Because it will rebuild and lock the table affecting the performance as well as our users. Therefore even if we start ALTER it’s really important to know when it will finish. Even while creating the index, ALTER TABLE will not rebuild the table if fast_index_creation is ON but still it might lock the table.

fast_index_creation feature was introduced in MySQL 5.5 and higher versions. Also available in MySQL 5.1 with the InnoDB Plugin

From MySQL 5.6 and later, “Online DDL” feature has been introduced, which is enhancing many other types of ALTER TABLE operations to avoid “copying the table” and “locking.” It also allows SELECT, INSERT, UPDATE, and DELETE statements to run while the table is being altered. So in the latest version, we can control the copy of file and locking by using ALGORITHM and LOCK options. But even in MySQL 5.6, there are some ALTER operations which still required table rebuild i.e ADD/DROP column, change the data type, ADD/DROP primary key, etc. You can check the table here for more details.

Summary of Online Status for DDL Operations”

So in any case, if required, we can check the ALTER TABLE progress with below solutions. 

One of the solutions is to use the Percona Toolkit utility, pt-online-schema-change which will ALTER the table without locking it for a long time and show the progress. i.e

While using pt-online-schema-change, many times I get questions such as: “What will happen to those data changes (INSERT/UPDATE/DELETE) which are performing during the ALTER TABLE?”

Here, I would like to explain briefly about it. While running pt-online-schema-change, if we will check data dir,

We can see that it will create triggers (separate trigger for INSERT, UPDATE and DELETE) on the original table to update corresponding rows to the new table( _nil_test_new). So any modifications happened to the data in original tables during the copy, will be reflected in the new table.

NOTE: This tool will not work if any triggers are already defined on the table.

But what if we don’t want to use pt-online-schema-change and run regular ALTER TABLE on MySQL prompt? After some research, I found many ways to calculate the progress of ALTER TABLE especially with innodb_file_per_table is ON. (innodb_file_per_table is ON by default in MySQL 5.6.)

  • Calculate the progress by checking the size of temporary tablespace. 

With innodb_file_per_table=1,when we are running ALTER TABLE, InnoDB creates a temporary tablespace within the same data directory with random name starting from #sql ending with .ibd like #sql-1c80_27.ibd. i.e

While altering table, InnoDB reads original ibd file like nil_test.ibd and writes new pages to #sql-1c80_27.ibd. So with file size of nil_test.ibd and temporary #sql-1c80_27.ibd, we can check the ALTER TABLE progress with something like, 

When we run ALTER on mysql, we can simply run this script in data dir and check the progress like,

There are a couple of things which we need to consider while using this script.

  1. We have to change script with $TABLENAME as per your requirement. It will work for only one ALTER TABLE.
  2. Script should run from database dir (i.e /var/lib/mysql/nil)
  3. These calculations are an approximation because the new table can be a bit different in size. For example, deleted rows or fragmentation can cause accuracy problems. 

I would like to mention here that we cannot check the progress of every ALTER TABLE with the above formula. For example, with fast_index_creation, It will create the table without any secondary indexes, then adding the secondary indexes after the data is loaded so it will not create temp tablespace (.ibd file) while creating a secondary index. This process is very interesting, let me explain here.

When we add secondary index with ‘fast_index_creation‘, it will only create #sql***.frm file to update table structure but not temporary .ibd file. It will do some file sorts and then add directly index to the original table.ibd file. so when you monitor those files, initially you didn’t see that file size increasing, but after some time (after files sorts) file size starts increasing till the end of ALTER TABLE. i.e

Initially, you’ll see

Then suddenly .ibd file size starts increasing…till the end.

This will happen if there is only one secondary index. If there are multiple secondary indexes then for each index, the process will pause, do file sorts and then add index so the number of pauses in file size increases, will be equal to the number of secondary indexes.

  • Calculate the progress by checking the records in information_schema.GLOBAL_TEMPORARY_TABLES 

When the ALTER TABLE is running, we can also check GLOBAL_TEMPORARY_TABLES in information_schema and it will give you records count which are inserted in a temporary table. i.e

  •  Calculate the progress by checking the Handler_read_rnd_next status variable (global counters).  

While running ALTER TABLE, we can also check the handler_read_rnd_next status variable by “SHOW GLOBAL STATUS LIKE ‘Handler_read_rnd%’ OR mysqladmin extended. Check this 2008 post titled “How to estimate query completion time in MySQL” by Baron Schwartz. He describes it very well.

NOTE: This won’t need innodb_file_per_table = ON  but we can use this option only when there are no other parallel transactions running.

So normally in any case, it will become really difficult to find the ALTER TABLE progress unless MySQL itself will provide some feature like MariaDB is providing progress in “SHOW PROCESSLIST”

I’ve installed MariaDB 5.5 locally and tried to check, along with “SHOW PROCESSLIST”, you can also check progress by running a query against information_schema.processlist table. But I observed that progress time is different in both the output.  It looks like information_schema.processlist table gives accurate progress for ALTER TABLE.

Conclusion: As we know, some ALTER TABLE can cause problems like server hang or crash. So whenever we are planning to run it on productions servers, it has to be well planned. As MySQL doesn’t provide a “progress bar,” we can use the above-explained methods to check progress. If anyone knows of any other method aside from the above then please share in the comments and I would glad to add it in this post.

Share this post

Comments (8)

  • Kevin

    I’ve been working on this now for a little while, but is there any solution like the shell script for partitioned tables?

    I’ve been working on one in python but the tmp table ends up being far larger than the table I’m altering when using partitions.

    February 26, 2014 at 4:20 pm
  • Nilnandan Joshi

    Hi Kevin,

    I think, same solution you can use for partitioned tables as it will create temp tablespace for each partition. But yes, as I said in the post “This calculations are an approximation, because new table can be a bit different in size.” so it can end up with large temp table size but it should not become very large. Have you checked if there are any bugs related to this? Please let us know which MySQL/PS version, you are using.

    March 4, 2014 at 5:47 am
  • Kevin


    It seems I spoke too soon. The shell script doesn’t work for partitioned tables, but the python script I wrote does work… it just seems I had an alter die that left behind the temp table partitions of a different table. Sorry for the confusion.

    March 4, 2014 at 8:40 am
  • Amit Jain

    Any view on myisam table’s ALTER progress bar ?

    September 16, 2014 at 10:29 am
  • martin

    Is’t possible calculate required tmp (tmpdir) disk space for online DDL queries?
    As well in the case of online DDL with inplace operation (5.6 and higher) is used tmpdir too. But imho not within same size as whole table.

    September 25, 2014 at 6:45 am
  • Aaron Ross

    This is great. I used the temp table size trick to help me figure out if I need to start freaking out or not. And I didn’t! So thanks for sharing!

    September 25, 2014 at 8:18 am
  • Greg Woolsey

    I wrote this query to monitor progress via row count estimates using InnoDB locked rows as a proxy for progress, as indicated above. Useful on systems where you don’t have shell access.

    select info stmt, id process, time run_time, state,
    trx_rows_locked, table_rows,
    round(trx_rows_locked/table_rows*100,2) pct_done,
    round(trx_rows_locked/time) rows_per_sec,
    round(table_rows/(trx_rows_locked/time)/60/60,2) hours_remaining,
    round(table_rows/(trx_rows_locked/time)/60,2) minutes_remaining
    from information_schema.tables t
    join information_schema.processlist p
    on like concat(‘alter table ‘,t.table_name,’%’)
    and p.db = t.table_schema
    join information_schema.innodb_trx x
    on x.trx_mysql_thread_id =

    adjust the join criteria if your statements are more complex or have line breaks up front, of course.

    November 8, 2017 at 3:18 pm
    • Greg Woolsey

      argh, the remaining time calculations need to subtract the locked rows from the total rows before dividing:

      round((table_rows-trx_rows_locked)/(trx_rows_locked/time)/60/60,2) hours_remaining,
      round((table_rows-trx_rows_locked)/(trx_rows_locked/time)/60,2) minutes_remaining

      November 8, 2017 at 11:01 pm

Comments are closed.

Use Percona's Technical Forum to ask any follow-up questions on this blog topic.