•  

Announcement

Announcement Module
Collapse
No announcement yet.

2GB on Win7, 64 bit?

Page Title Module
Move Remove Collapse
Conversation Detail Module
Collapse
  • Page of 1
  • Filter
  • Time
  • Show
Clear All
new posts

  • 2GB on Win7, 64 bit?

    #1
    Hi,

    Im developing an offline application with Visual Basic on Win7, 64bit. I use MS Visual Studio 2010 Express, MySQL Connector Net 6.3.5(32bit) and MySQL 5.1 (64bit).

    I have to insert a lot of records (+20.000.000) given by a XML streamer. To increase speed I already lock tables and disable keys.

    During testing I have recognized that MySQl doesn't seem to use more than 2GB. I have installed 8GB. How can I pass through the 2GB limit? Is the connector the limitation? Does it make sense to switch tothe MySQL ODBC Connector (64bit) to increase the memory usage? As far as I have heard the NET connector is platform indepandant and performance should be (more or less) same.

    Thank you very much for feedback on this...

    Markus
    Tags: None

  • #2
    How exactly do you mean MySQL is limited to a 2GB barrier? How big is your working dataset? What storage engine does your tables use?

    Comment


    • #3
      Thanks for your feedback.

      Well, currently I use MyISAM. I've tried also INNO, but not seen a difference. For the current application I query 25.000.000 recordsets in total, which are placed in 5 tables. Each table has 1-5 coloumns only. No blobs. Biggest table is 13.000.000 recordsets.

      Does it help you?
      Thanks again.

      Greetings

      Markus

      Comment


      • #4
        This might help http://www.mysqlperformanceblog.com/2007/05/24/predicting-ho w-long-data-load-would-take/#comment-211056

        How long does loading 25M rows on InnoDB take for you? I just tested loading 34M on stock my-medium.cnf using the sakila.film sample database and it too 19mins average on Linux if this gives you an idea.

        I don't know much about .NET, but have you tried loading your data manually if possible?

        Comment


        • #5
          fre391 wrote on Tue, 23 November 2010 09:39


          Well, currently I use MyISAM. I've tried also INNO, but not seen a difference. For the current application I query 25.000.000 recordsets in total, which are placed in 5 tables. Each table has 1-5 coloumns only. No blobs. Biggest table is 13.000.000 recordsets.


          So 2GB/25.000.000 records = ~80 bytes per record.
          Is that in the vicinity of what you are loading?

          And if you are using MyISAM tables you should now that only the indexes can be cached within MySQL so all data will end up in the OS file cache instead so you will not see it that easily in the task manager.

          And I assume that you have changed the mysql server parameters to allow larger caches within MySQL?
          Otherwise the server will never use more memory than what is allowed in the my.ini file.

          Comment

          Working...
          X