[prev in list] [next in list] [prev in thread] [next in thread]
List: soot-list
Subject: Re: [Soot-list] Call graphs and excluding libraries
From: "Steven Arzt" <Steven.Arzt () cased ! de>
Date: 2015-06-12 9:34:07
Message-ID: 007901d0a4f2$ede345c0$c9a9d140$ () cased ! de
[Download RAW message or body]
This is a multipart message in MIME format.
[Attachment #2 (multipart/alternative)]
This is a multipart message in MIME format.
Hi John,
If the package names are freemarker.*, it's important to put the asterisk into the \
exclusion option on the Soot command line as well. Otherwise, Soot will assume that \
there is a single class called "freemarker" which you would like to exclude.
If your code under analysis references a class which is not on the Soot classpath and \
–allow-phantom-refs is used, this class will automatically be considered as a \
phantom. Nothing will be loaded, because Soot cannot know where to load it from. If \
you exclude a class, its structure (method signatures, etc.) will be loaded, but it \
will be marked as a library class. If you specify –no-bodies-for-excluded, Soot \
will not load any bodies for those classes, even if some code tries to access the \
body. Therefore, not putting the class on the classpath is the most rigorous option \
for guaranteeing that it will not be loaded.
Your analysis then needs to deal with such cases. If you have a call to a.foo() and \
the type of the variable "a" refers to a phantom class, there will not be a callgraph \
edge from the call site to anywhere and thus there will not be an IFDS call edge. \
There will, however, still be an IFDS call-to-return edge in which you can detect \
that you are dealing with such an exclusion case and apply e.g., an external library \
model to make up for the information loss. Depending on your analysis, you might also \
be able to simply ignore the effects of the method call on your data flow \
abstraction, that's something you as the analysis designer need to decide.
Best regards,
Steven
Von: soot-list-bounces@CS.McGill.CA [mailto:soot-list-bounces@CS.McGill.CA] Im \
Auftrag von John A Toman
Gesendet: Donnerstag, 11. Juni 2015 21:18
Cc: soot-list@CS.McGill.CA
Betreff: Re: [Soot-list] Call graphs and excluding libraries
Hi Steven,
The freemarker library actually does put all its classes in the top-level \
freemarker.* package so I believe that my -x usage is correct.
As for your first point, I have been including all of the libraries (including the \
JDK libraries) because my first attempts to use Soot without them failed with missing \
class errors; I inferred that Soot would not function properly unless I included all \
classes that could be potentially referenced by the program being analyzed. If I \
understand you correctly there is some combination of flags and classpaths that will \
instruct Soot to simply ignore library classes (including the JDK). If this is true, \
what is the behavior of, say, the IFDS solver in the presence of these ignored \
libraries? If an application class calls into library method foo() that has been \
excluded what does Soot do with that call?
With regards to the strange call resolution, that problematic call statement appears \
to be in a library class that I wish to exclude anyway so the problem may become moot \
if I can figure out the above problem. Although it may still be a bug :)
Thanks!
John
On Thu, Jun 11, 2015 at 2:04 AM, Steven Arzt <Steven.Arzt@cased.de> wrote:
Hi John,
That's strange, your command-line looks good to me in principle. Your exclusion is \
not correct, you have to provide package or class names which would probably be \
something like com.freemarker.*. Soot does not search for substrings here, but for \
packages. Another option would be not to explicitly exclude these classes, but simply \
not have them on the Soot classpath. In that case, they will automatically become \
phantom classes.
The other question is whether you really want to prepend the JVM's classpath to your \
Soot classpath using –pp if you are ok with ignoring libraries at some other place. \
Is there a specific reason for not excluding the JDK as well? As recommended above, I \
would just remove it from the Soot classpath.
The strange call edge you are experiencing doesn't sound right to me. Can you provide \
a minimal working example that provides the same kind of callgraph with similar wrong \
edges, but has a more manageable size for debugging?
Best regards,
Steven
Von: soot-list-bounces@CS.McGill.CA [mailto:soot-list-bounces@CS.McGill.CA] Im \
Auftrag von John A Toman
Gesendet: Donnerstag, 11. Juni 2015 02:17
An: soot-list@cs.mcgill.ca
Betreff: [Soot-list] Call graphs and excluding libraries
Hello!
I'm trying to use Soot for an interprocedural analysis but I'm having trouble getting \
a callgraph. I'm analyzing a web application that depends on several libraries, each \
of which have their own dependencies. These dependencies aren't interesting to my \
analysis but without them I can't seem to get a reliable call-graph.
I'm invoking Soot as follows:
java -cp ./soot-trunk.jar:./myanalysis -pp -soot-class-path \
/path/to/application/classes:/paths/to/app/library/jars -no-bodies-for-excluded -x \
'freemarker' -w -p cg.cha enabled:false -p cg.spark enabled:true,verbose:true -p \
wjtp.myanalysis on com.acme.DummyMain -allow-phantom-refs
where freemarker is the package name of the library I'm not interested in (I exclude \
the freemarker.jar from the soot class path in the above invocation too)[
With these options Spark builds a callgraph, but the results are strange. For \
instance, the possible types analysis dies because (as far I as can tell) it thinks \
that a call to <java.security.AccessController: java.lang.Object \
doPrivileged(java.security.PrivilegedAction)>
potentially resolves to <org.apache.commons.logging.LogFactory$1: java.lang.Object \
run()>.
Without the -no-bodies-for-excluded option Spark churns for a while until it reaches \
some part of the Freemarker library that relies on Jython. However, the Jython jar \
also has unmet dependencies which cause Spark to choke...
In short: how do I configure Soot to ignore missing/uninteresting library calls with \
the least effort possible?
Thanks!
[Attachment #5 (text/html)]
<html xmlns:v="urn:schemas-microsoft-com:vml" \
xmlns:o="urn:schemas-microsoft-com:office:office" \
xmlns:w="urn:schemas-microsoft-com:office:word" \
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" \
xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type \
content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 14 \
(filtered medium)"><style><!-- /* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Sprechblasentext Zchn";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.E-MailFormatvorlage17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.SprechblasentextZchn
{mso-style-name:"Sprechblasentext Zchn";
mso-style-priority:99;
mso-style-link:Sprechblasentext;
font-family:"Tahoma","sans-serif";
mso-fareast-language:DE;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=DE link=blue vlink=purple><div \
class=WordSection1><p class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi \
John,<o:p></o:p></span></p><p class=MsoNormal><span \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p \
class=MsoNormal><span lang=EN-US \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>If the \
package names are freemarker.*, it's important to put the asterisk into the exclusion \
option on the Soot command line as well. Otherwise, Soot will assume that there is a \
single class called "freemarker" which you would like to \
exclude.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p \
class=MsoNormal><span lang=EN-US \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>If your \
code under analysis references a class which is not on the Soot classpath and \
–allow-phantom-refs is used, this class will automatically be considered as a \
phantom. Nothing will be loaded, because Soot cannot know where to load it from. If \
you exclude a class, its structure (method signatures, etc.) will be loaded, but it \
will be marked as a library class. If you specify –no-bodies-for-excluded, Soot \
will not load any bodies for those classes, even if some code tries to access the \
body. Therefore, not putting the class on the classpath is the most rigorous option \
for guaranteeing that it will not be loaded.<o:p></o:p></span></p><p \
class=MsoNormal><span lang=EN-US \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p \
class=MsoNormal><span lang=EN-US \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Your \
analysis then needs to deal with such cases. If you have a call to a.foo() and the \
type of the variable "a" refers to a phantom class, there will not be a callgraph \
edge from the call site to anywhere and thus there will not be an IFDS call edge. \
There will, however, still be an IFDS call-to-return edge in which you can detect \
that you are dealing with such an exclusion case and apply e.g., an external library \
model to make up for the information loss. Depending on your analysis, you might also \
be able to simply ignore the effects of the method call on your data flow \
abstraction, that's something you as the analysis designer need to \
decide.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p \
class=MsoNormal><span lang=EN-US \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Best \
regards,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> \
Steven<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p \
class=MsoNormal><b><span \
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>Von:</span></b><span \
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> \
soot-list-bounces@CS.McGill.CA [mailto:soot-list-bounces@CS.McGill.CA] <b>Im Auftrag \
von </b>John A Toman<br><b>Gesendet:</b> Donnerstag, 11. Juni 2015 \
21:18<br><b>Cc:</b> soot-list@CS.McGill.CA<br><b>Betreff:</b> Re: [Soot-list] Call \
graphs and excluding libraries<o:p></o:p></span></p><p \
class=MsoNormal><o:p> </o:p></p><div><div><div><div><p class=MsoNormal>Hi \
Steven,<br><br>The freemarker library actually does put all its classes in the \
top-level freemarker.* package so I believe that my -x usage is \
correct.<o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p></div><div><p \
class=MsoNormal style='margin-bottom:12.0pt'>As for your first point, I have been \
including all of the libraries (including the JDK libraries) because my first \
attempts to use Soot without them failed with missing class errors; I inferred that \
Soot would not function properly unless I included all classes that could be \
potentially referenced by the program being analyzed. If I understand you correctly \
there is some combination of flags and classpaths that will instruct Soot to simply \
ignore library classes (including the JDK). If this is true, what is the behavior of, \
say, the IFDS solver in the presence of these ignored libraries? If an application \
class calls into library method foo() that has been excluded what does Soot do with \
that call?<o:p></o:p></p></div><div><p class=MsoNormal>With regards to the strange \
call resolution, that problematic call statement appears to be in a library class \
that I wish to exclude anyway so the problem may become moot if I can figure out the \
above problem. Although it may still be a bug :)<o:p></o:p></p></div><div><p \
class=MsoNormal><o:p> </o:p></p></div><div><p \
class=MsoNormal>Thanks!<o:p></o:p></p></div><div><p \
class=MsoNormal>John<o:p></o:p></p></div><div><p \
class=MsoNormal><o:p> </o:p></p></div></div></div><div><p \
class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>On Thu, Jun 11, 2015 at \
2:04 AM, Steven Arzt <<a href="mailto:Steven.Arzt@cased.de" \
target="_blank">Steven.Arzt@cased.de</a>> wrote:<o:p></o:p></p><div><div><p \
class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span \
lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi \
John,</span><o:p></o:p></p><p class=MsoNormal \
style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p \
class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span \
lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>That's \
strange, your command-line looks good to me in principle. Your exclusion is not \
correct, you have to provide package or class names which would probably be something \
like com.freemarker.*. Soot does not search for substrings here, but for packages. \
Another option would be not to explicitly exclude these classes, but simply not have \
them on the Soot classpath. In that case, they will automatically become phantom \
classes.</span><o:p></o:p></p><p class=MsoNormal \
style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p \
class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span \
lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>The \
other question is whether you really want to prepend the JVM's classpath to your Soot \
classpath using –pp if you are ok with ignoring libraries at some other place. Is \
there a specific reason for not excluding the JDK as well? As recommended above, I \
would just remove it from the Soot classpath.</span><o:p></o:p></p><p class=MsoNormal \
style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p \
class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span \
lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>The \
strange call edge you are experiencing doesn't sound right to me. Can you provide a \
minimal working example that provides the same kind of callgraph with similar wrong \
edges, but has a more manageable size for debugging?</span><o:p></o:p></p><p \
class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span \
lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p \
class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span \
lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Best \
regards,</span><o:p></o:p></p><p class=MsoNormal \
style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> \
Steven</span><o:p></o:p></p><p class=MsoNormal \
style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US \
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p \
class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b><span \
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>Von:</span></b><span \
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> <a \
href="mailto:soot-list-bounces@CS.McGill.CA" \
target="_blank">soot-list-bounces@CS.McGill.CA</a> [mailto:<a \
href="mailto:soot-list-bounces@CS.McGill.CA" \
target="_blank">soot-list-bounces@CS.McGill.CA</a>] <b>Im Auftrag von </b>John A \
Toman<br><b>Gesendet:</b> Donnerstag, 11. Juni 2015 02:17<br><b>An:</b> <a \
href="mailto:soot-list@cs.mcgill.ca" \
target="_blank">soot-list@cs.mcgill.ca</a><br><b>Betreff:</b> [Soot-list] Call graphs \
and excluding libraries</span><o:p></o:p></p><div><div><p class=MsoNormal \
style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div><div><div><div><div><div><div><p \
class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'>Hello!<o:p></o:p></p></div><p \
class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'>I'm trying to \
use Soot for an interprocedural analysis but I'm having trouble getting a callgraph. \
I'm analyzing a web application that depends on several libraries, each of which have \
their own dependencies. These dependencies aren't interesting to my analysis but \
without them I can't seem to get a reliable call-graph.<o:p></o:p></p></div><p \
class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'>I'm invoking \
Soot as follows:<o:p></o:p></p></div><p class=MsoNormal \
style='mso-margin-top-alt:auto;margin-bottom:12.0pt'>java -cp \
./soot-trunk.jar:./myanalysis -pp -soot-class-path \
/path/to/application/classes:/paths/to/app/library/jars -no-bodies-for-excluded -x \
'freemarker' -w -p cg.cha enabled:false -p cg.spark enabled:true,verbose:true -p \
wjtp.myanalysis on com.acme.DummyMain -allow-phantom-refs<o:p></o:p></p></div><div><p \
class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>where \
freemarker is the package name of the library I'm not interested in (I exclude the \
freemarker.jar from the soot class path in the above invocation \
too)[<o:p></o:p></p></div><div><p class=MsoNormal \
style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><p \
class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>With these \
options Spark builds a callgraph, but the results are strange. For instance, the \
possible types analysis dies because (as far I as can tell) it thinks that a call to \
<java.security.AccessController: java.lang.Object \
doPrivileged(java.security.PrivilegedAction)><o:p></o:p></p></div><p \
class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'>potentially \
resolves to <org.apache.commons.logging.LogFactory$1: java.lang.Object \
run()>.<o:p></o:p></p></div><div><p class=MsoNormal \
style='mso-margin-top-alt:auto;margin-bottom:12.0pt'>Without the \
-no-bodies-for-excluded option Spark churns for a while until it reaches some part of \
the Freemarker library that relies on Jython. However, the Jython jar also has unmet \
dependencies which cause Spark to choke...<o:p></o:p></p></div><div><p \
class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'>In short: how do \
I configure Soot to ignore missing/uninteresting library calls with the least effort \
possible?<o:p></o:p></p></div><p class=MsoNormal \
style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Thanks!<o:p></o:p></p></div></div></div></div></div></div><p \
class=MsoNormal><o:p> </o:p></p></div></div></body></html>
_______________________________________________
Soot-list mailing list
Soot-list@CS.McGill.CA
https://mailman.CS.McGill.CA/mailman/listinfo/soot-list
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic