[prev in list] [next in list] [prev in thread] [next in thread] 

List:       openbsd-smp
Subject:    Re: Supporting 2nd Processors in Open BSD
From:       Robert Davies <rob_davies () ntlworld ! com>
Date:       2003-03-21 11:05:33
[Download RAW message or body]

On Friday 21 March 2003 10:03, Tobias Weingartner wrote:

> On Friday, March 21, Robert Davies wrote:
> > The issue that has held back development of SMP in Open BSD, is the large
> > amount of changes, in very many and complex areas of the OS kernel.  Some
> > a few years back did have processors booting and initialised (roughly
> > bout the 2.9 release or so), but core OpenBSD development continued a
> > pace without any regard to SMP support.
>
> You've almost hit the nail on the head.  The "problem" is that there are
> many more important things that need to be done, other than SMP.  Simply
> rewriting portions of the interrupt handling code for x86 would already
> be a good thing.  Or rewriting the x86 pmap to do PAE is more pressing to
> me than having an SMP box.  Tearing appart

Yes, but there's ppl interested in getting use of 2nd CPUs, the full SMP 
support has just gone backwards in the 3 years I've been on the list.  Time 
for a different approach?

NOTE the point this is NOT full SMP support, but something simpler and 
treating the 2nd CPU as a specialised encryption device, so that means only 
some basic initialisation code, and most of it being a device driver.

> > It would be much more realistic to cut back on the ambitions, and have a
> > project deliver something more self contained with a low impact on
> > current source.  It strikes me that treating the 2nd CPU as a device
> > allows simpler goals like :
>
> Some of this may not be easily possible.  For example:
>
> Using the second (or whatever) cpu on an x86 box means you will have
> to support the APIC devices.  These things route interrupts.  In other
> words, now you're dealing with converting the old interrupt controller
> code to deal with the APIC.  

No, it does not need general APIC support, that's the whole point.  Forget 
the I/O and real SMP, just get the processor coordination part done (IIRC 
they're IPI's).  I'm writing this on a Linux SMP system, it's a BP6 board and 
requires 'noapic' kernel option to run reliably, which disables the APIC for 
I/O.

/home/rob> cat /proc/interrupts
           CPU0       CPU1
  0:      70255          0          XT-PIC  timer
  1:          6          0          XT-PIC  keyboard
  2:          0          0          XT-PIC  cascade
  5:          0          0          XT-PIC  eth1
  8:         17          0          XT-PIC  rtc
  9:       4593          0          XT-PIC  usb-uhci, EMU10K1
 10:        844          0          XT-PIC  eth0
 11:       5043          0          XT-PIC  ide2, ide3
 12:      14179          0          XT-PIC  PS/2 Mouse
 14:      12634          0          XT-PIC  ide0
 15:         10          0          XT-PIC  ide1
NMI:          0          0
LOC:      70177      70176
ERR:          9
MIS:          0
/home/rob> < /proc/cpuinfo egrep 'proc|model name'
processor       : 0
model name      : Celeron (Mendocino)
processor       : 1
model name      : Celeron (Mendocino)

See just the same I/O handled by one processor, no APIC.

> > 1)  Intitialise 2nd Processor as Slave during boot up
>
> In order to do this, usually many other things need to be operating.
> For example (and I'm not sure if the sparc64 code deals or not), on
> the sparc64 arch I believe you even need to have special code in the
> boot code to catch the "extra" CPU's, such that you only run one on
> a non SMP system.

So that's SPARC, but device support does not have to be all embracing.  If an 
X86 crypto CPU for SMP hardware was available, it does not harm SPARC users, 
if it's useful the device would get ported.

> > 2)  Locking primitives for synchronisation, between Master & Slave
> > processor
>
> Including TLB shootdown, and other nasty things.  On the x86, you're
> going to need the APIC stuff...

Can't the 2nd CPU have a block of memory allocated, bit like a graphics card 
which the Master CPU then does not touch?

> > 3)  2nd CPU Crypto driver, where a process can punt calculations to the
> > unused CPU, treating it much like a hardware encryption device.
>
> The cost of a hifn is small, and will trounce your second CPU without even
> breaking a sweat.  The cost/perfomance ratio is not really worth it.  IE:
> I can spend 1 year trying to get my second CPU to do crypto work, or I can
> spend $200 (or whatever), and have a dedicated crypto engine working on the
> issue today.

Hyper-threading means soon the SMP cost for many users will be zero.

> > Actually getting some code contributions into the core Open BSD kernel,
> > without significantly increasing the complexity of that kernel, would be
> > implementable.  As there is already support for hardware devices for such
> > tasks, a "2nd CPU software emulation" ought not to be disruptive.
> >
> > Without the support and help of the core kernel developers, there is no
> > chance of things changing.  The objective "Produce a working SMP kernel"
> > is simply too big for a cinderella side-project.
>
> Instead of trying to solve the problem in this way, I would recommend that
> you look at the big picture.  Find the places in the kernel that prevent
> SMP from working, and that need biglock or other forms of locking.  If at
> all possible, rewrite them to be re-entrant, and otherwise lock-free, but
> SMP correct.  In this way, you are re-writing older code, will start to
> appreciate the complexity of the problem involved, and contribute new and
> better working code, hopefully moving things closer to a kernel that can
> deal with NCPU > 1.

The fact is in 3 or 5 years, OpenBSD has had this list, and there has not 
been any progress.  To get SMP going using the method you suggest, means a 
small team of ppl, practically re-writing the kernel, touching every single 
module.  It is only feasible if the OpenBSD core developers were committed to 
SMP as an objective, which they aren't for reasons you discuss.

> bad, contention parts of the kernel just more congested.  As such, I
> will be more excited seeing UVM, UBC, and other things like that going
> into the kernel, than SMP.  Of course, each has their own fetish...  :-)

Well this list is 'smp@openbsd.org' ...

Rob

[prev in list] [next in list] [prev in thread] [next in thread]