[prev in list] [next in list] [prev in thread] [next in thread] 

List:       sqlite-users
Subject:    Re: [sqlite] Multithreading
From:       "Roger Binns" <rogerb () rogerbinns ! com>
Date:       2004-12-26 3:37:02
Message-ID: 002501c4eafc$29f5a880$3501a8c0 () rogersqyvr14d3
[Download RAW message or body]

> Well, currently SQLite seems to meet my performance needs. Currently I 
> don't think that brewing my own file format would pay off. I will 
> consider this only as a last solution.

I was being somewhat facetious mainly to prove that you don't
actually want best possible performance.  You are far better off
writing whatever code is simplest, especially to test, and then
profiling the results.  If performance is not acceptable then,
you can change how things work and will have prior code to test
against.  

  http://c2.com/cgi/wiki?PrematureOptimization

> The higher-level tasks will be performed by various callers in various
> threads. But in the end, each tasks requires several db commands to be
> executed, so the db will be the bottleneck; 

I did say that the work items should be higher level than a single
SQL command.

> adding more threads and CPUs
> won't help if I have to serialize all requests into a queue that is
> processed by a worker thread. 

Worker thread*s*.  You will generally want one more worker thread
than you have CPUs.  However until you actually profile you won't
know where your bottle neck is.  It could be disk, memory, CPU
or other factors.

> The thread switching at some point will limit scalability. 

And you know this how?  Given that operating systems and CPUs have
been tuning for thread switching for decades, actual contradictory
evidence would be useful first.

> The other option - opening the db in each thread -
> will lead to more memory overhead, also limiting the scalability.

The default settings use 2MB.  With a pragma you can change this
up or down.

As the page says:
 
  Make it work. 
  Make it right. 
  Make it fast.

Roger
[prev in list] [next in list] [prev in thread] [next in thread]