[prev in list] [next in list] [prev in thread] [next in thread] 

List:       csa
Subject:    Re: [Lse-tech] Re: Proposal for Enhanced Accounting HOWTO
From:       Guillaume Thouvenin <guillaume.thouvenin () bull ! net>
Date:       2004-09-06 8:06:27
Message-ID: 20040906080627.GA5714 () frec ! bull ! fr
[Download RAW message or body]

On Fri, Sep 03, 2004 at 10:48:33AM -0700, Jay Lan wrote:
> Support of project concept is certainly one thing we need to consider.
> We have received requests to support project in CSA.
> 
> Cheers,
>  - jay
> >
> >Can/will this support project based accounting ? 
> >
> >i.e. Users being member of several projects. Each project having
> >multiple users as members. And the user is allowed to decide which
> >project-id each container is going to run under / be accounted to ?

Here is an update of the Enhanced Accounting HOWTO. 

Here is the ChangeLog:
6/09/2004: Guillaume Thouvenin
	- Add a section concerning projects based accounting 
	  proposed by Jan-Frode Myklebust. 
	- Add a section concerning the EAC (Enhanced Accounting
	  Core) part.

And here is the proposal:

                   Enhanced Linux accounting Howto?
                   ================================
 
  According to discussion on the lse-tech mailing-list, here are some 
stages that are required to improve accounting. 

1) Improve accounting structure
   ----------------------------

   The current BSD-accounting structure doesn't have enough informations. 
Metrics computed by CSA module can be added to BSD accounting. According to
other discussion (like Andi Kleen's comment on the patch I wrote when I wanted
to add CSA IO values in the BSD accounting ( http://lkml.org/lkml/2004/8/2/70 )
the current method to get metrics about blocks/char read/write is not accurate 
since most writes can be accounted to pdflsuhd.


2) Group of processes management
   ------------------------------

   We need to be able to manage groups of processes as it's clear that a major
accounting improvement is the per-job accounting. I don't know if "job" is the
right noun. There are several implementation that already exist and some of them
are already in the kernel. The property needed here is that if a process is in
a container, its children will be in the same container. Different 
implementations can be:

	- PAGG + JOB (job)
	- ELSA (bank)
	- CKRM (class)
	- CPUSET (a cpuset of all CPUs can act as a container)

   The interface between kernel and user space application can be a new virtual
file system (like CKRM does with /rcfs) but it can also be a device driver with
ioctl operations (like ELSA does). Both solutions are interesting and need to
be discussed. We can notice that CPUSET is in 2.6.9-rc1-mm2 tree and PAGG has 
been removed from -mm tree because there is a lack of code that uses PAGG.


3) Project based accounting
   ------------------------

   It could be interesting to add a support for project based accounting. It 
means that a user can be a member of one or more projects and he is allowed 
to decide which project-id each container is going to run. This could be done 
inside the Enhanced Accounting Core module.


4) Data presentation
   -----------------
	
    We can have several different implementations. 


5) General overview
   ----------------

          KERNEL SPACE        |   USER SPACE (or MODULE)       	     
                              |
         ----------------     | 
        | BSD accounting |    | 
        |       +        |    |     ------------
        |      CSA       |<=======>|            |
         ----------------     |    |  Enhanced  |
                              |    | Accounting |
            -------------     |    |    Core    |
           |  group of   |<=======>|            |
           |  processes  |    |     ------------
           |  manager    |    |
            -------------     |
                              |
			       
  Communication between EAC (Enhanced Accounting Core) and kernel space can be
done via virtual fs or a device. The goal of EAC is to keep a trace of the
different groups of processes during the accounting period using the group of
processes management module. The idea is that the group of processes management
can send a message to EAC when a process ended to indicate the job
ID if any. The other thing that has to be done by EAC is the data presentation.

  Here is an example of what can be done:

  a) First we can add processes into containers using the GPM (group of 
     processes manager) module. For example we can add an ftp server with 
     pid #123 and a daemon ssh with pid #234. Thus, inside the GPM you have:
         container 1 -> 123
	 container 2 -> 234

  b) Now, a user can login via ssh, so, sshd will create new children. Thus, 
     inside the GPM you will have something like:
         container 1 -> 123
	 container 2 -> 234 333 334 335

  c) As soon as a process is terminated, the EAC must be aware of that fact 
     (using a signal from GPM for example). There is just this modification
     that is needed. With the signal, the GPM will transfer information like
     pid, jib (job ID), command, ... to the EAC. This will allow to keep trace
     about what happens in the system. Thus, in the EAC we keep information 
     about all processes that are finished. 

  d) If the sysadmin asks information about process 234, the EAC will know that
     it belongs to container 2 and there are other processes in this container.
     As all processes are exited, accounting information are written in the 
     accounting file (like it is currently with BSD accounting). With those 
     informations (accounting + job information) we can do per-job accounting.


6) Enhanced Accounting Core (EAC) implementation
   ---------------------------------------------

    The EAC can be implemented in the user space or as a kernel module.
We will see later differences between both implementation. The EAC has 
several goals. 

    o It will manage a group of processes historic. It means that it 
      will keep a trace concerning processes that belong to the same 
      container. We don't care about how container are managed, we just 
      need to know when a new process is created and if he is in a 
      container. It means that the job manager (or group of processes 
      manager) must provide a mechanism to share this information. If 
      we decide to implement EAC in user space, information can be pass 
      through a signal or a device. If it's a module, we can add an hook
      in the code of the job manager. 

    o It will provide routine to add/remove processes from container. It 
      will act as a wrapper for routine that already exist in the job 
      manager.

    o It will get information from the accounting file. Probably from 
      BSD accounting extended with CSA information. Those informations 
      will be used in conjunction with the group of process historic to 
      provide the enhanced accounting which is a per-job accounting 
      information.
    
    o Finally, EAC we will support the project based accounting as describe
      in section 3)



Any comments?

  I am working on the EAC implementation to check how we can make it work.

Guillaume

[prev in list] [next in list] [prev in thread] [next in thread]