[prev in list] [next in list] [prev in thread] [next in thread]
List: oprofile-list
Subject: Re: [PATCH] (Rivision2) Extended Feature and IBS documentation
From: Maynard Johnson <maynardj () us ! ibm ! com>
Date: 2009-05-26 21:39:11
Message-ID: 4A1C617F.6020500 () us ! ibm ! com
[Download RAW message or body]
Suravee Suthikulpanit wrote:
> Change from previous patch submission:
> - Grammar/spelling fixes as suggested by Maynard.
> - Add a note that "ext_init" is required by all features.
Suravee,
Sorry about the delay in reviewing this. Just a few minor fixes needed, identified \
below. Go ahead and commit the patch after making the fixes.
Thanks!
-Maynard
>
> This patch contains documentation for Extended Feature Interface and IBS.
>
> ChangeLog | 5 +
> doc/internals.xml | 215 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
> doc/oprofile.xml | 102 ++++++++++++++++++++++++-
> 3 files changed, 316 insertions(+), 6 deletions(-)
>
> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
>
> ---
>
> Index: oprofile-ibs-doc/ChangeLog
> ===================================================================
> RCS file: /cvsroot/oprofile/oprofile/ChangeLog,v
> retrieving revision 1.1871
> diff -u -r1.1871 ChangeLog
> --- oprofile-ibs-doc/ChangeLog 7 May 2009 14:20:14 -0000 1.1871
> +++ oprofile-ibs-doc/ChangeLog 7 May 2009 20:22:48 -0000
> @@ -1,3 +1,8 @@
> +2009-05-04 Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> +
> + * doc/oprofile.xml:
> + * doc/internals.xml: Add Extended-Interface and IBS documentation,
> +
> 2009-05-07 Andi Kleen <ak@linux.intel.com>
> * utils/ophelp.c:
> * libop/op_cpu_type.c:
> Index: oprofile-ibs-doc/doc/internals.xml
> ===================================================================
> RCS file: /cvsroot/oprofile/oprofile/doc/internals.xml,v
> retrieving revision 1.16
> diff -u -r1.16 internals.xml
> --- oprofile-ibs-doc/doc/internals.xml 15 Nov 2006 16:33:26 -0000 1.16
> +++ oprofile-ibs-doc/doc/internals.xml 7 May 2009 20:22:48 -0000
> @@ -900,6 +900,221 @@
>
> </chapter>
>
> +<chapter id="ext">
> +<title>Extended Feature Interface</title>
> +
> +<sect1 id="ext-intro">
> +<title>Introduction</title>
> +
> +<para>
> +The Extended Feature Interface is a standard callback interface
> +designed to allow extension to the OProfile daemon's sample processing.
> +Each feature defines a set of callback handlers which can be enabled or
> +disabled through the OProfile daemon's command-line option.
> +This interface can be used to implement support for architecture-specific
> +features or features not commonly used by general OProfile users.
> +</para>
> +
> +</sect1>
> +
> +<sect1 id="ext-name-and-handlers">
> +<title>Feature Name and Handlers</title>
> +
> +<para>
> +Each extended feature has an entry in the <varname>ext_feature_table</varname>
> +in <filename>opd_extended.cpp</filename>. Each entry contains a feature name,
> +and a corresponding set of handlers. Feature name is a unique string, which is
> +used to identify a feature in the table. Each feature provides a set
> +of handlers, which will be executed by the OProfile daemon from pre-determined
> +locations to perform certain tasks. At runtime, the OProfile daemon calls a \
> feature +handler wrapper from one of the pre-determined locations to check whether
> +an extended feature is enabled, and whether a particular handler exists.
No comma ^
> +Only the handlers of the enabled feature will be executed.
> +</para>
> +
> +</sect1>
> +
> +<sect1 id="ext-enable">
> +<title>Enabling Features</title>
> +
> +<para>
> +Each feature is enabled using the OProfile daemon (oprofiled) command-line
> +option "--ext-feature=<extended-feature-name>:[args]". The
> +"extended-feature-name" is used to determine the feature to be enabled.
> +The optional "args" is passed into the feature-specific initialization handler
> +(<function>ext_init</function>). Currently, only one extended feature can be
> +enabled at a time.
> +</para>
> +
> +</sect1>
> +
> +<sect1 id="ext-types-of-handlers">
> +<title>Type of Handlers</title>
> +
> +<para>
> +Each feature is responsible for providing its own set of handlers.
> +Types of handler are:
> +</para>
> +
> +<sect2 id="ext_init">
> +<title>ext_init Handler</title>
> +
> +<para>
> +"ext_init" handles initialization of an extended feature. It takes
> +"args" parameter which is passed in through the "oprofiled --ext-feature=<
> +extended-feature-name>:[args]". This handler is executed in the function
> +<function>opd_options()</function> in the file <filename>daemon/oprofiled.c
> +</filename>.
> +</para>
> +
> +<note>
> +<para>
> +The ext_init handler is required for all features.
> +</para>
> +</note>
> +
> +</sect2>
> +
> +<sect2 id="ext_print_stats">
> +<title>ext_print_stats Handler</title>
> +
> +<para>
> +"ext_print_stats" handles the extended feature statistics report. It adds
> +a new section in the OProfile daemon statistics report, which is normally
> +outputed to the file
> +<filename>/var/lib/oprofile/samples/oprofiled.log</filename>.
> +This handler is executed in the function <function>opd_print_stats()</function>
> +in the file <filename>daemon/opd_stats.c</filename>.
> +</para>
> +
> +</sect2>
> +
> +<sect2 id="ext_sfile_handlers">
> +<title>ext_sfile Handler</title>
> +
> +<para>
> +"ext_sfile" contains a set of handlers related to operations on the extended
> +sample files (sample files for events related to extended feature).
> +These operations include <function>create_sfile()</function>,
> +<function>sfile_dup()</function>, <function>close_sfile()</function>,
> +<function>sync_sfile()</function>, and <function>get_file()</function>
> +as defined in <filename>daemon/opd_sfile.c</filename>.
> +An additional field, <varname>odb_t * ext_file</varname>, is added to the
> +<varname>struct sfile</varname> for storing extended sample files
> +information.
> +
> +</para>
> +
> +</sect2>
> +
> +</sect1>
> +
> +<sect1 id="ext-implementation">
> +<title>Extended Feature Reference Implementation</title>
> +
> +<sect2 id="ext-ibs">
> +<title>Instruction-Based Sampling (IBS)</title>
Add a sentence here something to the effect:
"An example extended feature implementation can be seen by examining the AMD \
Instruction-Based Sampling support."
> +
> +<sect3 id="ibs-init">
> +<title>IBS Initialization</title>
> +
> +<para>
> +Instruction-Based Sampling (IBS) is a new performance measurement technique
> +available on AMD Family 10h processors. Enabling IBS profiling is done simply
> +by specifying IBS performance events through the "--event=" options.
> +</para>
> +
> +<screen>
> +opcontrol --event=IBS_FETCH_XXX:<count>:<um>:<kernel>:<user>
> +opcontrol --event=IBS_OP_XXX:<count>:<um>:<kernel>:<user>
> +
> +Note: * Count and unitmask for all IBS fetch events must be the same.
> + So does event and count for IBS op.
This is still a bit clumsy, and I don't think it's a complete sentence. How about:
"Count and unitmask for all IBS fetch events must be the same, as do those for IBS \
op."
> +</screen>
> +
> +<para>
> +IBS performance events are listed in <function>opcontrol --list-events</function>.
> +When users specify these events, opcontrol verifies them using ophelp, which
> +checks for the <varname>ext:ibs_fetch</varname> or <varname>ext:ibs_op</varname>
> +tag in <filename>events/x86-64/family10/events</filename> file.
> +Then, it configures the driver interface (/dev/oprofile/ibs_fetch/... and
> +/dev/oprofile/ibs_op/...) and starts the OProfile daemon as follows.
> +</para>
> +
> +<screen>
> +oprofiled \
> + --ext-feature=ibs:\
> + fetch:<IBS_FETCH_EVENT1>,<IBS_FETCH_EVENT2>,...,:<IBS fetch \
> count>:<IBS Fetch um>|\ \
> + op:<IBS_OP_EVENT1>,<IBS_OP_EVENT2>,...,:<IBS op count>:<IBS \
> op um> +</screen>
> +
> +<para>
> +Here, the OProfile daemon parses the <varname>--ext-feature</varname>
> +option and checks the feature name ("ibs") before calling the
> +the initialization function to handle the string
> +containing IBS events, counts, and unitmasks.
> +Then, it stores each event in the IBS virtual-counter table
> +(<varname>struct opd_event ibs_vc[OP_MAX_IBS_COUNTERS]</varname>) and
> +stores the event index in the IBS Virtual Counter Index (VCI) map
> +(<varname>ibs_vci_map[OP_MAX_IBS_COUNTERS]</varname>) with IBS event value
> +as the map key.
> +</para>
> +</sect3>
> +
> +<sect3 id="ibs-data-processing">
> +<title>IBS Data Processing</title>
> +
> +<para>
> +During a profile session, the OProfile daemon identifies IBS samples in the
> +event buffer using the <varname>"IBS_FETCH_CODE"</varname> or
> +<varname>"IBS_OP_CODE"</varname>. These codes trigger the handlers
> +<function>code_ibs_fetch_sample()</function> or
> +<function>code_ibs_op_sample()</function> listed in the
> +<varname>handler_t handlers[]</varname> vector in
> +<filename>daemon/opd_trans.c </filename>. These handlers are responsible for
> +processing IBS samples and translate them into IBS performance events.
> +</para>
> +
> +<para>
> +Unlike traditional performance events, each IBS sample can be derived into
> +multiple IBS performance events. For each event that the user specifies,
> +a combination of bits from Model-Specific Registers (MSR) are checked
> +against the bitmask defining the event. If the condition is met, the event
> +will then be recorded. The derivation logic is in the files
> +<filename>daemon/opd_ibs_macro.h</filename> and
> +<filename>daemon/opd_ibs_trans.[h,c]</filename>.
> +</para>
> +
> +</sect3>
> +
> +<sect3 id="ibs-sample-file">
> +<title>IBS Sample File</title>
> +
> +<para>
> +Traditionally, sample file information <varname>(odb_t)</varname> is stored
> +in the <varname>struct sfile::odb_t file[OP_MAX_COUNTER]</varname>.
> +Currently, <varname>OP_MAX_COUNTER</varname> is 8 on non-alpha, and 20 on
> +alpha-based system. Event index (the counter number on which the event
> +is configured) is used to access the corresponding entry in the array.
> +Unlike the traditional performance event, IBS does not use the actual
> +counter registers (i.e. <filename>/dev/oprofile/0,1,2,3</filename>).
> +Also, the number of performance events generated by IBS could be larger than
> +<varname>OP_MAX_COUNTER</varname> (currently upto 13 IBS-fetch and 46 IBS-op
Add space ^
> +events). Therefore IBS requires a special data structure, and sfile
Remove comma ^
> +handlers (<varname>struct opd_ext_sfile_handlers</varname>) for managing
> +IBS sample files. IBS-sample-file information is stored in a memory
> +allocated by handler <function>ibs_sfile_create()</function>, which can
> +be accessed through <varname>struct sfile::odb_t * ext_files</varname>.
> +</para>
> +
> +</sect3>
> +
> +</sect2>
> +
> +</sect1>
> +
> +</chapter>
> +
> <glossary id="glossary">
> <title>Glossary of OProfile source concepts and types</title>
>
> Index: oprofile-ibs-doc/doc/oprofile.xml
> ===================================================================
> RCS file: /cvsroot/oprofile/oprofile/doc/oprofile.xml,v
> retrieving revision 1.170
> diff -u -r1.170 oprofile.xml
> --- oprofile-ibs-doc/doc/oprofile.xml 3 Feb 2009 20:22:11 -0000 1.170
> +++ oprofile-ibs-doc/doc/oprofile.xml 7 May 2009 20:22:48 -0000
> @@ -129,7 +129,13 @@
>
> <note>Attempting to profile SPEs with kernel versions older than 2.6.22 may cause \
> the system to crash.</note>
> - </para></listitem>
> + </para>
> +
> + <para>
> + Instruction-Based Sampling (IBS) profile on AMD family10h processors requires
> + kernel version 2.6.28-rc2 or later.
> + </para>
> + </listitem>
> </varlistentry>
> <varlistentry>
> <term>modutils 2.4.6 or above</term>
> @@ -145,7 +151,7 @@
> required. In marketing terms this translates to anything
> between an Intel Pentium Pro (not Pentium Classics) and
> a Pentium 4 / Xeon, including all Celerons. The AMD
> - Athlon, and Duron CPUs are also supported. Other IA32
> + Athlon, Opteron, Phenom, and Turion CPUs are also supported. Other IA32
> CPU types only support the RTC mode of OProfile; please
> see later in this manual for details. Hyper-threaded Pentium IVs
> are not supported in 2.4. For 2.4 kernels, the Intel
> @@ -910,6 +916,8 @@
> <row><entry>Pentium 4 \
> (non-HT)</entry><entry>i386/p4</entry><entry>GLOBAL_POWER_EVENTS:100000:1:1:1</entry></row>
> <row><entry>Pentium 4 \
> (HT)</entry><entry>i386/p4-ht</entry><entry>GLOBAL_POWER_EVENTS:100000:1:1:1</entry></row>
> <row><entry>Hammer</entry><entry>x86-64/hammer</entry><entry>CPU_CLK_UNHALTED:100000:0:1:1</entry></row>
> +<row><entry>Family10h</entry><entry>x86-64/family10</entry><entry>CPU_CLK_UNHALTED:100000:0:1:1</entry></row>
> +<row><entry>Family11h</entry><entry>x86-64/family11h</entry><entry>CPU_CLK_UNHALTED:100000:0:1:1</entry></row>
> <row><entry>Itanium</entry><entry>ia64/itanium</entry><entry>CPU_CYCLES:100000:0:1:1</entry></row>
> <row><entry>Itanium \
> 2</entry><entry>ia64/itanium2</entry><entry>CPU_CYCLES:100000:0:1:1</entry></row> \
> <row><entry>TIMER_INT</entry><entry>timer</entry><entry>None \
> selectable</entry></row> @@ -1011,8 +1019,8 @@
> </note>
> <para>
> The Intel hardware performance counters are detailed in the Intel IA-32 \
> Architecture Manual, Volume 3, available
> -from <ulink url="http://developer.intel.com/">http://developer.intel.com/</ulink>. \
> The AMD Athlon/Duron
> -implementation is detailed in <ulink
> +from <ulink url="http://developer.intel.com/">http://developer.intel.com/</ulink>. \
> +The AMD Athlon/Opteron/Phenom/Turion implementation is detailed in <ulink
> url="http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/22007.pdf">
> http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/22007.pdf</ulink>.
> For PowerPC64 processors in IBM iSeries, pSeries, and blade server systems, \
> processor documentation @@ -1119,7 +1127,7 @@
> operation. Unfortunately the relationship between these registers is quite complex; \
> they cannot all be used with one another at any time. There is, however, a subset \
> of 8 counters, 8 ESCRs, and 8 CCCRs which can be used independently of one another, \
> so OProfile only accesses those registers, treating them as a bank of 8 "normal" \
> counters, similar
> -to those in the P6 or Athlon families of CPU.
> +to those in the P6 or Athlon/Opteron/Phenom/Turion families of CPU.
> </para>
> <para>
> There is currently no support for Precision Event-Based Sampling (PEBS), nor any \
> advanced uses of the Debug Store @@ -1230,6 +1238,88 @@
>
> </sect2>
>
> +<sect2 id="amd-ibs-support">
> +<title>AMD64 (x86_64) Instruction-Based Sampling (IBS) support</title>
> +
> +<para>
> +Instruction-Based Sampling (IBS) is a new performance measurement technique
> +available on AMD Family 10h processors. Traditional performance counter
> +sampling is not precise enough to isolate performance issues to individual
> +instructions. IBS, however, precisely identifies instructions which are not
> +making the best use of the processor pipeline and memory hierarchy.
> +For more information, please refer to the "Instruction-Based Sampling:
> +A New Performance Analysis Technique for AMD Family 10h Processors" (
> +<ulink url="http://developer.amd.com/assets/AMD_IBS_paper_EN.pdf">
> +http://developer.amd.com/assets/AMD_IBS_paper_EN.pdf</ulink>).
> +There are two types of IBS profile types, described in the following sections:
Change colon to period ^
> +</para>
> +
> +<sect3 id="ibs-fetch">
> +<title>IBS Fetch</title>
> +
> +<para>
> +IBS fetch sampling is a statistical sampling method which counts completed
> +fetch operations. When the number of completed fetch operations reaches the
> +maximum fetch count (the sampling period), IBS tags the fetch operation and
> +monitors that operation until it either completes or aborts. When a tagged
> +fetch completes or aborts, a sampling interrupt is generated and an IBS fetch
> +sample is taken. An IBS fetch sample contains a timestamp, the identifier of
> +the interrupted process, the virtual fetch address, and several event flags
> +and values that described what happened during the fetch operation.
change to "describe" ^^
> +</para>
> +
> +</sect3>
> +
> +<sect3 id="ibs-op">
> +<title>IBS Op</title>
> +
> +<para>
> +IBS op sampling selects, tags, and monitors macro-ops as issued from AMD64
> +instructions. Two options are available for selecting ops for sampling:
> +</para>
> +
> +<itemizedlist>
> +<listitem>
> +Cycles-based selection counts CPU clock cycles. The op is tagged and monitored
> +when the count reaches a threshold (the sampling period) and a valid op is
> +available.
> +</listitem>
> +
> +<listitem>
> +Dispatched op-based selection counts dispatched macro-ops.
> +When the count reaches a threshold, the next valid op is tagged and monitored.
> +</listitem>
> +</itemizedlist>
> +
> +<para>
> +In both cases, an IBS sample is generated only if the tagged op retires.
> +Thus, IBS op event information does not measure speculative execution activity.
> +The execution stages of the pipeline monitor the tagged macro-op. When the
> +tagged macro-op retires, a sampling interrupt is generated and an IBS op
> +sample is taken. An IBS op sample contains a timestamp, the identifier of
> +the interrupted process, the virtual address of the AMD64 instruction from
> +which the op was issued, and several event flags and values that describe
> +what happened when the macro-op executed.
> +</para>
> +
> +</sect3>
> +
> +<para>
> +Enabling IBS profiling is done simply by specifying IBS performance events
> +through the "--event=" options. These events are listed in the
> +<function>opcontrol --list-events</function>.
> +</para>
> +
> +<screen>
> +opcontrol --event=IBS_FETCH_XXX:<count>:<um>:<kernel>:<user>
> +opcontrol --event=IBS_OP_XXX:<count>:<um>:<kernel>:<user>
> +
> +Note: * All IBS fetch event must have the same event count and unitmask. So does \
> IBS op.
See my comment above for the same Note in the internals doc. Change wording here to \
be the same.
> +</screen>
> +
> +</sect2>
> +
> +
> <sect2 id="misuse">
> <title>Dangerous counter settings</title>
> <para>
> @@ -2477,7 +2567,7 @@
> <title>Interrupt masking</title>
> <para>
> OProfile uses non-maskable interrupts (NMI) on the P6 generation, Pentium 4,
> -Athlon and Duron processors. These interrupts can occur even in section of the
> +Athlon, Opteron, Phenom, and Turion processors. These interrupts can occur even in \
> section of the Linux where interrupts are disabled, allowing collection of samples \
> in virtually all executable code. The RTC, timer interrupt mode, and Itanium 2 \
> collection mechanisms use maskable interrupts. Thus, the RTC and Itanium 2 data \
> collection mechanism have "sample
>
>
>
> ------------------------------------------------------------------------------
> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
> production scanning environment may not be a perfect world - but thanks to
> Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
> Series Scanner you'll get full speed at 300 dpi even with all image
> processing features enabled. http://p.sf.net/sfu/kodak-com
> _______________________________________________
> oprofile-list mailing list
> oprofile-list@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/oprofile-list
------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com
_______________________________________________
oprofile-list mailing list
oprofile-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oprofile-list
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic