[prev in list] [next in list] [prev in thread] [next in thread] 

List:       openjdk-serviceability-dev
Subject:    Bytecode generation, Source code mappings, JCov, Future (Patch)
From:       Jonathan.Gibbons () Sun ! COM (Jonathan Gibbons)
Date:       2008-04-30 16:04:08
Message-ID: 022EB04B-F20F-4600-9827-7C89C7FB8D65 () Sun ! COM
[Download RAW message or body]

Alex,

I think Jim's comments go the desired granularity of the information  
that could be available.  Although it is not *necessary* recompile on  
the fly, it may be *preferable*. to By compiling on demand and  
comparing byte codes, you can achieve as high a resolution as is  
desired, without having to have any static information at all, whether  
in class files or in "side files". In principle, you should be able to  
step through an expression, sub expression by sub expression.  By  
contrast, with statically generated information, you get what you get  
-- and line-based info is not necessarily very good, and more detailed  
information is potentially very big.

However, back on the side of static information, I guess there is  
potentially a performance issue.  In a debugger, you're mostly dealing  
with "line at a time" code at human speeds, as the user steps through  
code, or sets up breakpoints.  In profiling tools, you probably want  
info about the entire contents of the class file, "in batch time". But  
the problem remains with statically generated files -- how do you  
determine the resolution of the information that should be generated?

-- Jon


On Apr 29, 2008, at 11:41 PM, Alex Rau wrote:

> Hi Jim,
>
> thanks for the detailed info. Unfortunately I've not had much time  
> this week to investigate deeply on your proposal (compiler API /  
> debugger API). Here are some things I came up so far - please  
> correct me in case I got something wrong:
>
> 1) The debugger API is based on a design with two virtual machines  
> involved ( the debugger vm and the vm which gets debugged). While  
> this fits perfectly a debugging or profiling scenario where two  
> virtual machines are always involved this does not properly line up  
> with my scenario where only one instance of a virtual machine  
> exists. Our software is based on top of a readily available  
> (compiled) build. It performs modifications on the byte code of the  
> build, runs all unit tests and generates xml reports (all done in  
> the mentioned single vm in one shot). That's all. A second vm is  
> just not existing and would mean much more overhead to our design  
> just for getting column information.
>
> 2) I could not yet find my way through the compiler and debugger API  
> from a technical point f view to really have the column information  
> in the end. I've already had a look on the netbeans sources and  
> (probably) found the right code location but I have to investigate  
> on that in more detail. However this indicates somehow that it's  
> getting much more tricky compared to the variant where the compiler  
> itself outputs the column information into the byte code via  
> additional attributes. A question here: is is necessary to recompile  
> on the fly during debugging to get the line/column information ? If  
> yes then this would make it even more difficult and would mean that  
> we have to support an additional compilation process while up to now  
> we strictly rely on already performed compilations. We work on byte  
> code exclusively and the sources are only required for the report  
> generation.
>
> 3) I think that line numbers and column information are actually  
> "attributes" of the compiler ( result ) in a broader sense. It  
> always depends on the compiler what values these attributes will  
> have. Compared to for example a duration of a method invocation  
> (profiling) or a certain value of a variable (debugging) the latter  
> are *always* runtime-dependent values. What I'd like to say is:  
> there are static ( runtime-independent, "compiler only"-dependent )  
> attributes (line and column info) and dynamic attributes ( runtime  
> and execution dependent ) attributes (invocation duration, variable  
> value). I see a "natural" separation between those where static  
> attributes should be stored statically (e.g. in the byte code) and  
> dynamic attributes should be accessible dynamically (like the  
> debugger API allows). This does  imply as well that while we are  
> interested in static attributes of the compiler it's really not  
> necessary to reread these attributes with every modification on  
> bytecode level. Having these information at a single point of time  
> (after the compilation is finished) is totally sufficient compared  
> to getting the information during runtime every time.
>
>
> It looks to me that what I want to achieve belongs more to the  
> compiler than somewhere else. Any comments ?
>
>
> Best regards,
>
> Alex
>
>
> On 24.04.2008, at 04:53, Jim Holmlund wrote:
>
>> Just to summarize:
>> - jcov is an internal to Sun tool.
>> - to support jcov, a .class file attribute called the  
>> CharacterRangeTable attribute was
>> defined and javac was changed to output it in response to the - 
>> Xjcov(I think) command line option:
>> CharacterRangeTable_attribute {
>> u2 attribute_name_index;
>> u4 attribute_length;
>> u2 character_range_table_length;
>> { u2 start_pc;
>> u2 end_pc;
>> u4 character_range_start;
>> u4 character_range_end;
>> u2 flags;
>> } character_range_table[character_range_table_length];
>> }
>> The 'flags' field item describes the kind of range, eg statement,  
>> block, assignment,
>> flow_controller ..
>>
>> - the CharacterRangeTable was never added to the VM Spec.
>>
>> - jcov used the old JVMPI. Robert rewrote it to do byte code  
>> instrumentation
>> via java.lang.instrument. It still uses the CharacterRangeTable.
>>
>> As Robert mentioned, we have had requests from debuggers to include  
>> this kind of info in the .class file, for example to allow stepping  
>> thru terms of an expression, multiple statements on one line, etc.  
>> We planned to do something for this in JDK 6, eg, formalize the  
>> CharacterRangeTable attribute by adding it to the definition of the  
>> class file in the VM spec, and add functionality to JVM TI, JDWP,  
>> and JDI to allow debuggers to access this information.
>>
>> When Peter von der Ah? heard about this, he suggested that we not  
>> do this and instead proposed a solution that required no changes to  
>> be made to the JDK. His idea was that an IDE has the source code  
>> for a file in which fine grained stepping is desired, and the IDE  
>> can get the bytecodes from the debuggee VM via JDI  
>> (Method.bytecodes()). The IDE can then use the compiler APIs  
>> introduced in JDK 6
>> http://www.artima.com/lejava/articles/compiler_api.html
>> to match the source code to the bytecodes to find the bytecodes  
>> that correspond to source constructs of interest. This idea was  
>> investigated by the NetBeans debugger team and found to be  
>> effective, so it was implemented as the 'expression stepping'  
>> feature in NetBeans 6.0:
>> http://www.netbeans.org/features/java/debugger.html
>>
>> So, we ended up not needing character offset information in JPDA  
>> and so we didn't add the CharacterRangeTable attribute to the VM  
>> spec. Adding thisinformation to JPDA would be very low on our list  
>> of things to do, unless
>> some needs arise that can't be handled by Peter's technique.
>>
>> I wonder if Alex could also use Peter's idea. Alex did mention that  
>> the tools he is interested
>> in normally have the source code available so maybe he could.
>>
>> - jjh
>>
>> Jonathan Gibbons wrote:
>>> Hi Serviceability folk,
>>>
>>> The Subject line is from a thread on the compiler-dev list. You  
>>> might be interested to check it out here:
>>> http://mail.openjdk.java.net/pipermail/compiler-dev/2008-April/thread.html#300
>>>
>>> The thread concerns an interest in improving the information about  
>>> source location generated by the compiler, javac, and more  
>>> specifically, increasing the resolution of the info from line- 
>>> based coordinates to source-based coordinates. The submitter is  
>>> also talking about using side files for the info, which (if I  
>>> recall correctly) I have heard folk such as Jim discuss before now.
>>>
>>> What would be the interest from the serviceability group about any  
>>> such work? Is it "on your radar", "sometime eventually", or "it'll  
>>> never happen"? :-)
>>>
>>> -- Jon
>>>
>>> P.S. Warning: the submitter has provided a patch on the compiler- 
>>> dev thread but has not yet signed the SCA.
>>>
>>>
>



[prev in list] [next in list] [prev in thread] [next in thread]