Announcement

Announcement Module
Collapse
No announcement yet.

2GB on Win7, 64 bit?

Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • 2GB on Win7, 64 bit?

    Hi,

    Im developing an offline application with Visual Basic on Win7, 64bit. I use MS Visual Studio 2010 Express, MySQL Connector Net 6.3.5(32bit) and MySQL 5.1 (64bit).

    I have to insert a lot of records (+20.000.000) given by a XML streamer. To increase speed I already lock tables and disable keys.

    During testing I have recognized that MySQl doesn't seem to use more than 2GB. I have installed 8GB. How can I pass through the 2GB limit? Is the connector the limitation? Does it make sense to switch tothe MySQL ODBC Connector (64bit) to increase the memory usage? As far as I have heard the NET connector is platform indepandant and performance should be (more or less) same.

    Thank you very much for feedback on this...

    Markus

  • #2
    How exactly do you mean MySQL is limited to a 2GB barrier? How big is your working dataset? What storage engine does your tables use?
    Our documentation has a lot of answers about common questions on Percona software, have you checked there before posting that question here? http://www.percona.com/forums/core/i...lies/smile.png

    Join us at the annual Percona Live MySQL Users Conference - http://www.percona.com/live/mysql-conference-2014/

    Comment


    • #3
      Thanks for your feedback.

      Well, currently I use MyISAM. I've tried also INNO, but not seen a difference. For the current application I query 25.000.000 recordsets in total, which are placed in 5 tables. Each table has 1-5 coloumns only. No blobs. Biggest table is 13.000.000 recordsets.

      Does it help you?
      Thanks again.

      Greetings

      Markus

      Comment


      • #4
        This might help http://www.mysqlperformanceblog.com/2007/05/24/predicting-ho w-long-data-load-would-take/#comment-211056

        How long does loading 25M rows on InnoDB take for you? I just tested loading 34M on stock my-medium.cnf using the sakila.film sample database and it too 19mins average on Linux if this gives you an idea.

        I don't know much about .NET, but have you tried loading your data manually if possible?
        Our documentation has a lot of answers about common questions on Percona software, have you checked there before posting that question here? http://www.percona.com/forums/core/i...lies/smile.png

        Join us at the annual Percona Live MySQL Users Conference - http://www.percona.com/live/mysql-conference-2014/

        Comment


        • #5
          fre391 wrote on Tue, 23 November 2010 09:39


          Well, currently I use MyISAM. I've tried also INNO, but not seen a difference. For the current application I query 25.000.000 recordsets in total, which are placed in 5 tables. Each table has 1-5 coloumns only. No blobs. Biggest table is 13.000.000 recordsets.


          So 2GB/25.000.000 records = ~80 bytes per record.
          Is that in the vicinity of what you are loading?

          And if you are using MyISAM tables you should now that only the indexes can be cached within MySQL so all data will end up in the OS file cache instead so you will not see it that easily in the task manager.

          And I assume that you have changed the mysql server parameters to allow larger caches within MySQL?
          Otherwise the server will never use more memory than what is allowed in the my.ini file.

          Comment

          Working...
          X