[prev in list] [next in list] [prev in thread] [next in thread] 

List:       monetdb-users
Subject:    Re: [MonetDB-users] monetdb do not release the memory
From:       Martin Kersten <Martin.Kersten () cwi ! nl>
Date:       2012-06-15 9:18:29
Message-ID: 4FDAFDE5.90501 () cwi ! nl
[Download RAW message or body]



On 6/15/12 10:04 AM, Franck Routier wrote:
> Ouch...
Did not want to hurt you, thanks for responding to the question initially.
>
> Martin, could elaborate a bit or give me pointers ? I must admit that
> your last sentence lost me...
pointers... the code base ;)
>
> "The system keeps an eye on not over-stressing
> the RAM footprint of the OS platform using operator scheduling."
>
> By the system, do you mean Monetdb ? And is "operator scheduling" a
> defined concept ? I must admit my ignorance, and would gladly accept any
> pointer :-)
System is indeed MonetDB. If you look at EXPLAIN sqlquery you see
the plan being executed by the kernel. Multiple of such instructions
are executed concurrently. The kernel scheduler makes sure that it
does not overrun the memory resources, as this might cause page
thrashing (recognizable as a MonetDB server with 0% cpu use and 0% io)

regards, Martin
>
> Regards,
>
> Franck
>
> Le 15/06/2012 09:30, Martin Kersten a =E9crit :
>> To be a little more precise, MonetDB is designed from a main-memory
>> perspective and works happily with multiple GB databases.
>> It aggressively use the memory available to keep a hotset
>> readily available in RAM. During query processing, the virtual memory
>> allocated by MonetDB may be much larger then your RAM, because it
>> memory-maps columns. The system keeps an eye on not over-stressing
>> the RAM footprint of the OS platform using operator scheduling.
>>
>> regards, Martin
>>
>> On 6/15/12 8:44 AM, Franck Routier wrote:
>>> I think it is a good monetdb practice to manage the memory monetd is
>>> allocated at the OS level.
>>> Typically on Linux, you will have to use ulimit.
>>> Don't know how (if) your Windows flavour can handle that, but AFAIK,
>>> this is the way to go.
>>>
>>> Franck
>>>
>>>
>>> Le 15/06/2012 08:37, Fabian Groffen a =E9crit :
>>>> Hi liangmeng,
>>>>
>>>> On 15-06-2012 09:10:25 +0800, liangmeng wrote:
>>>>>     i'am new here, i install monetdb of windows 64bit version, after =
copy into
>>>>> table from several very large files(about 10G each), i found that mon=
etdb
>>>>> occupied too much momery, almost 100% of total 32G , and it keeps occ=
upying,
>>>>> after 24hours, it still don't release the memory, what's the matter?
>>>> MonetDB is a *main-memory* database.  It will basically keep as much in
>>>> memory as it can.
>>>>
>>>>
>>>> ----------------------------------------------------------------------=
--------
>>>> Live Security Virtual Conference
>>>> Exclusive live event will cover all the ways today's security and
>>>> threat landscape has changed and how IT managers can respond. Discussi=
ons
>>>> will include endpoint security, mobile security and the latest in malw=
are
>>>> threats.http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>>>
>>>>
>>>> _______________________________________________
>>>> MonetDB-users mailing list
>>>> MonetDB-users@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/monetdb-users
>>>
>>>
>>> -----------------------------------------------------------------------=
-------
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. Discussio=
ns
>>> will include endpoint security, mobile security and the latest in malwa=
re
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>>
>>>
>>>
>>> _______________________________________________
>>> MonetDB-users mailing list
>>> MonetDB-users@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/monetdb-users
>> ------------------------------------------------------------------------=
------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> MonetDB-users mailing list
>> MonetDB-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/monetdb-users
>
>
> -------------------------------------------------------------------------=
-----
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> MonetDB-users mailing list
> MonetDB-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/monetdb-users


---------------------------------------------------------------------------=
---
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and =

threat landscape has changed and how IT managers can respond. Discussions =

will include endpoint security, mobile security and the latest in malware =

threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
MonetDB-users mailing list
MonetDB-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/monetdb-users
[prev in list] [next in list] [prev in thread] [next in thread]