[prev in list] [next in list] [prev in thread] [next in thread] 

List:       netbsd-tech-kern
Subject:    Re: RFC: IRQ affinity (aka interrupt routing)
From:       Kengo NAKAHARA <k-nakahara () iij ! ad ! jp>
Date:       2014-07-28 8:55:31
Message-ID: 53D61003.8030802 () iij ! ad ! jp
[Download RAW message or body]

Hi Thor,

Fist, I am *not* going to implement automatic balancing interrupts
in the near future. As you point out, it must require more discussion
about automatic balancing interrupts. However, manual routing
interrupts must be needed, such as separating HDD interrupts' CPU
from NIC interrupts' CPU, eliminating some CPUs from CPUs assigned
interrupts to reserve for user processes, and so on. In these cases,
it is no need to balance interrupts automatically, it is enogh to
set affinity by administrators at the boot time or the rare
maintenance time. So, I want to implement manually routing interrupts
at the first step for these usage.

(2014/07/28 12:49), Thor Lancelot Simon wrote:
> On Mon, Jul 28, 2014 at 12:20:07PM +0900, Kengo NAKAHARA wrote:
>>
>> I think analyzing interrupt rates, deciding which IRQ's interrupts
>> shuld be moved and deciding which CPU is assigned the moved interrupts
>> are complex.
>
> I think this may prove counterproductive.  For one reason, it may cause
> harmful cache behavior as the datastructures associated with the same
> network flow move from one CPU to another as packets are received on
> different interfaces for the transmit and receive directions.

Yes, there are such cases which interrupts shold be not moved, so balancing
interrupts is too complex. I ought to have said full automatic balancing
interrupts daemon is far-off future work. In the immediate future, I want to
route interrupts occasionally by hand.

> I think you really don't want to do anything that could impede the use
> of hardware classifiers to put the packets for the same flow on the
> same CPUs' queues -- and balancing interrupts in userspace rather than in
> the kernel seems to me likely to do just that.

If NetBSD support multiqueue (and MSI-X), it may be correct for the NIC
supporting multiqueue.


Thanks,

-- 
//////////////////////////////////////////////////////////////////////
Internet Initiative Japan Inc.

Device Engineering Section,
Core Product Development Department,
Product Division,
Technology Unit

Kengo NAKAHARA <k-nakahara@iij.ad.jp>
[prev in list] [next in list] [prev in thread] [next in thread]