[prev in list] [next in list] [prev in thread] [next in thread] 

List:       ojb-user
Subject:    RE: Out of Memory bug
From:       "Guillaume Nodet" <guillaume.nodet () deliasystems ! com>
Date:       2004-05-28 13:32:25
Message-ID: NEBBJLFEPKFNNEEFBCIOIEANCLAA.guillaume.nodet () deliasystems ! com
[Download RAW message or body]

I had the same problem with the default cache implementation.
There are several cache implementations proposed in ojb:
  * ObjectCacheDefaultImpl
	This cache uses soft references for storing objects,
	but not for the keys and the objects stored so they
	stay in the map forever.
  * ObjectCacheSoftImpl
	This is a good cache implementation. It uses a lru map
	with a fixed number of hard references to objects and
	a hashmap containing soft references, cleaned with a
	queue associated to these soft references.
	This is only available with JDK 1.4.
	The only problem is that it has not been maintained and
	so does not have the proper constructor to use it.

So i used a modified ObjectCacheSoftImpl that implements the proper
constructor, added a way to customize the size of the hard references
map.

To use, you must remove the two constructors that are defined and add this
one

	public ObjectCacheSoftImpl(PersistenceBroker broker, Properties properties)
{
		if (cache == null) {
			cache = new SoftHashMap(1000);
		}
	}

It would be good if the properties parameter given to the constructor could
be valid, but it is always null, so the size must either be configured via
a static variable of the ObjectCacheSoftImpl class instead of using
properties
defined in OJB.properties file...

Guillaume

> -----Message d'origine-----
> De : AKakunin@pikos.net [mailto:AKakunin@pikos.net]
> Envoye : mercredi 26 mai 2004 18:07
> A : OJB Users List
> Objet : Re: Out of Memory bug
>
>
> Hello!
> Seems I found the solution for this problem: periodically (about after
> 100.000 writes) clear the broker cache:
> getDestBroker().clearCache();
>
> Now it works, but I'm not sure that it is good solution. In this simple
> example I can understand how often I should clear the cache, but in some
> more complex situation it will be not so easy.
> I think cache implementation in broker should be more inteligent
> and clear
> themself after cache will exids some critical size, so programm will not
> be interupted by out of memory error. (Maybe I should add it somethere in
> issue tracker? I do not know because I quite new in db-ojb)
>
> Best Regards,
> Alexey Kakunin
> Project Manager
> PIKOS GmbH
> Post Str. 24
> 53111 Bonn
>
> Tel: +49 30 43 72 92-30
> Fax: +49 30 43 72 92-30
>
>
>
>
> AKakunin@pikos.net
> 05/26/2004 01:43 PM
> Please respond to "OJB Users List"
>
>
>         To:     "OJB Users List" <ojb-user@db.apache.org>
>         cc:
>         Subject:        Re: Out of Memory bug
>
>
> Hello!
> Thank you for quick answer but:
> 1) I tried to put beginTransaction/commiTransaction into while loop -
> without success - same out of memory problem
> 2) I tried to comment beginTransaction/commiTransaction at all with same
> result
> 3) I tried to create one object Kvart before while and store
> readed values
>
> in this instance, but same without success :(
>
> Yes, I agree with you in case that seems it is probably a problem of
> Java/JDBC/Garbage Collector, but how it is possible to fix?
>
> Best Regards,
> Alexey Kakunin
> Project Manager
> PIKOS GmbH
> Post Str. 24
> 53111 Bonn
>
> Tel: +49 30 43 72 92-30
> Fax: +49 30 43 72 92-30
>
>
>
>
> Thomas Mahler <thma32@web.de>
> Sent by: thma32@web.de
> 05/26/2004 01:23 PM
> Please respond to "OJB Users List"
>
>
>         To:     OJB Users List <ojb-user@db.apache.org>
>         cc:
>         Subject:        Re: Out of Memory bug
>
>
> Hi Alexey,
>
> you are allocating a new Kvart instance for each Resultset row.
> You could change the code to save tons of ram as shown below.
> You could also try to commit blocks of say 10000 Kvarts. This will save
> a lot of JDBC resources.
>
> cheers,
> Thomas
>
> AKakunin@pikos.net wrote:
> > Hello everybody!
> > I made a investigate project (to see features of OJB). The current task
> of
> > this project is move data from MSAccess database into MySQL.
> > For getting data from MSAccess I'm using JDBC directly (because
> I wasn't
>
>
> > able to setup character set for MSAccess through OJB) and I'm using OJB
> > for writing data into MySQL.
> >
> > So, I got an "Out Of memory" error during copying a quite a big amount
> of
> > data (about 1.500.000 records), so only about 400.000 records was
> copied.
> > The generated table is quite easy: one integer fields and 2 strings
> > fields.
> >
> > Here is a code how I'm doing copying:
> >
> >                 //select data from source database
> >                 Statement stmt =
> getSourceConnection().createStatement();
> >                 ResultSet rs = stmt.executeQuery("SELECT * FROM KVART");
> >
> >                 //1. open transaction in destination database
> >                 getDestBroker().beginTransaction();
> >
>
>                                    app.MySQL.Kvart transferKvart =
>  >                                 new app.MySQL.Kvart();
>
> >                 while (rs.next()) {
> >
>                           transferKvart.setTel(rs.getInt("tel"));
>  transferKvart.setFio("fio");
>                           transferKvart.setVidTel("vid_tel");
> >
> >                        // save new phone
>                           getDestBroker().store(transferKvart);
> >
> >                 }
> >                 //commit transaction
> >                 getDestBroker().commitTransaction();
> >
> > So, why in such simple case it generates out of memory error and how it
> is
> > possible to fix it (may be I'm doing something wrong?)
> >
> > I'm using db-ojb 1.0.rc6 with MySQL Connector/J version 3.0.11
> >
> > Best Regards,
> > Alexey Kakunin
> > Project Manager
> > PIKOS GmbH
> > Post Str. 24
> > 53111 Bonn
> >
> > Tel: +49 30 43 72 92-30
> > Fax: +49 30 43 72 92-30
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
> For additional commands, e-mail: ojb-user-help@db.apache.org
>
>
>
>
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-user-help@db.apache.org

[prev in list] [next in list] [prev in thread] [next in thread]