[prev in list] [next in list] [prev in thread] [next in thread]
List: soot-list
Subject: Re: [Soot-list] [Soot] [CallGraph] [FlowDroid] Spark calligraph incomplete
From: Miguel Velez <mvelezce () cs ! cmu ! edu>
Date: 2019-10-29 17:16:33
Message-ID: 7872AEEF-9F5D-44C6-B50D-5146030FB3C7 () cs ! cmu ! edu
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Thanks for the hint on the getOrMakeFastHierarychy(). However, using \
PackManager.v().runPacks() did not work either. Also, the Scene.v().forceResolve() is \
also used in FlowDroid to create a call graph. That is why I am using it.
Thanks,
Miguel
> On Oct 29, 2019, at 11:13 AM, David Diepenbrock <ddiepenbrock@pjrcorp.com> wrote:
>
> I'm not an expert, but I think instead of applying specific packs you should use \
> PackManager.v().runPacks();.
> Also, there's no need to invoke getOrMakeFastHierarchy(), unless you want to use \
> the hierarchy. The hierarchy will be built by the packs if needed. \
> Scene.v().forceResolve() probably isn't required either.
> Best,
> David Diepenbrock
> From: Soot-list <soot-list-bounces@CS.McGill.CA \
> <mailto:soot-list-bounces@CS.McGill.CA>> on behalf of Miguel Velez \
> <mvelezce@cs.cmu.edu <mailto:mvelezce@cs.cmu.edu>>
> Sent: Monday, October 28, 2019 6:41 PM
> To: soot-list@cs.mcgill.ca <mailto:soot-list@cs.mcgill.ca> <soot-list@CS.McGill.CA \
> <mailto:soot-list@CS.McGill.CA>>
> Subject: [Soot-list] Spark calligraph incomplete
>
> Hi everyone,
>
> I am trying to build a CallGraph with Spark to called information about what \
> methods call other methods. I am trying to replicated how a CallGraph is built \
> using FlowDroid. However, the call graph that I generate for a program is \
> incomplete. I know it is incomplete since I compared the number of ClassNodes using \
> ASM (211) with the number of SootClasses from parsing the src and tat of the edges \
> from callGraph.listener() (80). I have spent several hours trying to debug why some \
> edges are missing, but I cannot understand why.
> Here is how I am building a call graph programmatically:
>
> ```
> public static CallGraph buildCallGraph(String entryPoint, String appPath) {
> long startTime = System.nanoTime();
> initializeSoot(entryPoint, appPath);
>
> PackManager.v().getPack("wjpp").apply();
> PackManager.v().getPack("cg").apply();
>
> Scene.v().getOrMakeFastHierarchy();
>
> CallGraph callGraph = Scene.v().getCallGraph();
>
> long endTime = System.nanoTime();
> System.out.println("Time to build call graph: " + ((endTime - startTime) / 1E9));
>
> return callGraph;
> }
>
> private static void initializeSoot(String entryPoint, String appPath) {
> soot.G.reset();
>
> Options.v().set_no_bodies_for_excluded(true);
> Options.v().set_allow_phantom_refs(true);
> Options.v().set_output_format(Options.output_format_none);
> Options.v()
> .set_soot_classpath(
> appPath
> + ":/Library/Java/JavaVirtualMachines/jdk1.8.0_202.jdk/Contents/Home/jre/lib/rt.jar: \
> /Library/Java/JavaVirtualMachines/jdk1.8.0_202.jdk/Contents/Home/jre/lib/jce.jar"); \
> setSparkOptions();
>
> Options.v().set_whole_program(true);
> Options.v().setPhaseOption("cg", "trim-clinit:false");
> Options.v().setPhaseOption("jb.ulp", "off");
> Options.v().set_src_prec(Options.src_prec_java);
>
> // User options
> Options.v().setPhaseOption("jb", "use-original-names:true");
>
> // Options needed for instrumentation
> Options.v().set_keep_line_number(true);
> Options.v().set_keep_offset(true);
> Options.v().set_coffi(true);
> Options.v().set_ignore_classpath_errors(true);
>
> loadClassesAndBodies(entryPoint);
> }
>
> private static void loadClassesAndBodies(String entryPoint) {
> Scene.v().addBasicClass(entryPoint, SootClass.BODIES);
> Scene.v().loadNecessaryClasses();
>
> boolean hasClasses = false;
>
> SootClass c = Scene.v().forceResolve(entryPoint, SootClass.BODIES);
> if (c != null) {
> c.setApplicationClass();
> if (!c.isPhantomClass() && !c.isPhantom()) hasClasses = true;
> }
>
> if (!hasClasses) {
> throw new RuntimeException("Only phantom classes loaded, skipping analysis...");
> }
> }
>
> private static void setSparkOptions() {
> Options.v().setPhaseOption("cg.spark", "on");
> Options.v().setPhaseOption("cg.spark", "string-constants:true");
> }
> ```
>
> Can anyone help me understand what am I doing wrong?
>
> For instance, this is a method that is part of the calligraphy:
>
> ```
> public List<PngPixel> getMergedColors(PngImage image, List<PngPixel> colors, long \
> start) { for (PngPixel pa : colors) {
> if (!pa.isDuplicate()) {
> ...
> }
> }
> }
> ```
>
> but the `isDuplicate()` method is not part of the calligraphy (i.e., the \
> `edgesOutOf(getMergedColors)` does not include `isDuplicate`). However, if I change \
> the class method to:
> ```
> public List<PngPixel> getMergedColors(PngImage image, List<PngPixel> colors, long \
> start) { PngPixel pa = new PngPixel();
> pa.isDuplicate();
> }
> ```
>
> The `isDuplicate()` method is part of the call graph. Why would creating a \
> constructor include the method in the call graph?
> Thanks,
>
> Miguel
> _______________________________________________
> Soot-list mailing list
> Soot-list@CS.McGill.CA <mailto:Soot-list@CS.McGill.CA>
> https://mailman.CS.McGill.CA/mailman/listinfo/soot-list \
> <https://mailman.cs.mcgill.ca/mailman/listinfo/soot-list>
[Attachment #5 (unknown)]
<html><head><meta http-equiv="Content-Type" content="text/html; \
charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: \
space; line-break: after-white-space;" class="">Thanks for the hint on the \
getOrMakeFastHierarychy(). However, using PackManager.v().runPacks() did not work \
either. Also, the Scene.v().forceResolve() is also used in FlowDroid to create a call \
graph. That is why I am using it.<div class=""><br class=""></div><div \
class="">Thanks,</div><div class=""><br class=""></div><div class="">Miguel<br \
class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Oct 29, \
2019, at 11:13 AM, David Diepenbrock <<a href="mailto:ddiepenbrock@pjrcorp.com" \
class="">ddiepenbrock@pjrcorp.com</a>> wrote:</div><br \
class="Apple-interchange-newline"><div class=""><div style="font-style: normal; \
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: \
start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: \
0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, \
Arial, Helvetica, sans-serif; font-size: 12pt;" class="">I'm not an expert, but I \
think instead of applying specific packs you should \
use PackManager.v().runPacks();. <span \
class="Apple-converted-space"> </span><br class=""><br class="">Also, there's no \
need to invoke getOrMakeFastHierarchy(), unless you want to use the \
hierarchy. The hierarchy will be built by the packs if needed. \
Scene.v().forceResolve() probably isn't required either.</div><div style="font-style: \
normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; \
text-align: start; text-indent: 0px; text-transform: none; white-space: normal; \
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; \
font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class=""><br \
class=""></div><div style="font-style: normal; font-variant-caps: normal; \
font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; \
text-transform: none; white-space: normal; word-spacing: 0px; \
-webkit-text-stroke-width: 0px; text-decoration: none; font-family: Calibri, Arial, \
Helvetica, sans-serif; font-size: 12pt;" class="">Best,</div><div style="font-style: \
normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; \
text-align: start; text-indent: 0px; text-transform: none; white-space: normal; \
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; \
font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">David \
Diepenbrock</div><div id="appendonsend" style="caret-color: rgb(0, 0, 0); \
font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: \
normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: \
0px; text-transform: none; white-space: normal; word-spacing: 0px; \
-webkit-text-stroke-width: 0px; text-decoration: none;" class=""></div><hr \
tabindex="-1" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: \
12px; font-style: normal; font-variant-caps: normal; font-weight: normal; \
letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; \
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; \
text-decoration: none; display: inline-block; width: 865.328125px;" class=""><span \
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; text-align: start; text-indent: 0px; text-transform: none; white-space: \
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; \
float: none; display: inline !important;" class=""></span><div id="divRplyFwdMsg" \
dir="ltr" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; text-align: start; text-indent: 0px; text-transform: none; white-space: \
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" \
class=""><font face="Calibri, sans-serif" style="font-size: 11pt;" class=""><b \
class="">From:</b><span class="Apple-converted-space"> </span>Soot-list <<a \
href="mailto:soot-list-bounces@CS.McGill.CA" \
class="">soot-list-bounces@CS.McGill.CA</a>> on behalf of Miguel Velez <<a \
href="mailto:mvelezce@cs.cmu.edu" class="">mvelezce@cs.cmu.edu</a>><br class=""><b \
class="">Sent:</b><span class="Apple-converted-space"> </span>Monday, October \
28, 2019 6:41 PM<br class=""><b class="">To:</b><span \
class="Apple-converted-space"> </span><a href="mailto:soot-list@cs.mcgill.ca" \
class="">soot-list@cs.mcgill.ca</a><span \
class="Apple-converted-space"> </span><<a \
href="mailto:soot-list@CS.McGill.CA" class="">soot-list@CS.McGill.CA</a>><br \
class=""><b class="">Subject:</b><span \
class="Apple-converted-space"> </span>[Soot-list] Spark calligraph \
incomplete</font><div class=""> </div></div><div class="BodyFragment" \
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; text-align: start; text-indent: 0px; text-transform: none; white-space: \
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: \
none;"><font size="2" class=""><span style="font-size: 11pt;" class=""><div \
class="PlainText">Hi everyone,<br class=""><br class="">I am trying to build a \
CallGraph with Spark to called information about what methods call other methods. I \
am trying to replicated how a CallGraph is built using FlowDroid. However, the call \
graph that I generate for a program is incomplete. I know it is incomplete since I \
compared the number of ClassNodes using ASM (211) with the number of SootClasses from \
parsing the src and tat of the edges from callGraph.listener() (80). I have spent \
several hours trying to debug why some edges are missing, but I cannot understand \
why.<br class=""><br class="">Here is how I am building a call graph \
programmatically:<br class=""><br class="">```<br class="">public static CallGraph \
buildCallGraph(String entryPoint, String appPath) {<br class=""> long startTime \
= System.nanoTime();<br class=""> initializeSoot(entryPoint, appPath);<br \
class=""><br class=""> PackManager.v().getPack("wjpp").apply();<br \
class=""> PackManager.v().getPack("cg").apply();<br class=""><br \
class=""> Scene.v().getOrMakeFastHierarchy();<br class=""><br class=""> \
CallGraph callGraph = Scene.v().getCallGraph();<br class=""><br class=""> long \
endTime = System.nanoTime();<br class=""> System.out.println("Time to build \
call graph: " + ((endTime - startTime) / 1E9));<br class=""><br class=""> \
return callGraph;<br class="">}<br class=""><br class="">private static void \
initializeSoot(String entryPoint, String appPath) {<br class=""> \
soot.G.reset();<br class=""><br class=""> \
Options.v().set_no_bodies_for_excluded(true);<br class=""> \
Options.v().set_allow_phantom_refs(true);<br class=""> \
Options.v().set_output_format(Options.output_format_none);<br class=""> \
Options.v()<br class=""> .set_soot_classpath(<br \
class=""> appPath<br \
class=""> \
+ ":/Library/Java/JavaVirtualMachines/jdk1.8.0_202.jdk/Contents/Home/jre/lib/rt.jar:/L \
ibrary/Java/JavaVirtualMachines/jdk1.8.0_202.jdk/Contents/Home/jre/lib/jce.jar");<br \
class=""><br class=""> setSparkOptions();<br class=""><br class=""> \
Options.v().set_whole_program(true);<br class=""> \
Options.v().setPhaseOption("cg", "trim-clinit:false");<br class=""> \
Options.v().setPhaseOption("jb.ulp", "off");<br class=""> \
Options.v().set_src_prec(Options.src_prec_java);<br class=""><br class=""> // \
User options<br class=""> Options.v().setPhaseOption("jb", \
"use-original-names:true");<br class=""><br class=""> // Options needed for \
instrumentation<br class=""> Options.v().set_keep_line_number(true);<br \
class=""> Options.v().set_keep_offset(true);<br class=""> \
Options.v().set_coffi(true);<br class=""> \
Options.v().set_ignore_classpath_errors(true);<br class=""><br class=""> \
loadClassesAndBodies(entryPoint);<br class="">}<br class=""><br class="">private \
static void loadClassesAndBodies(String entryPoint) {<br class=""> \
Scene.v().addBasicClass(entryPoint, SootClass.BODIES);<br class=""> \
Scene.v().loadNecessaryClasses();<br class=""><br class=""> boolean hasClasses \
= false;<br class=""><br class=""> SootClass c = \
Scene.v().forceResolve(entryPoint, SootClass.BODIES);<br class=""> if (c != \
null) {<br class=""> c.setApplicationClass();<br \
class=""> if (!c.isPhantomClass() && !c.isPhantom()) \
hasClasses = true;<br class=""> }<br class=""><br class=""> if \
(!hasClasses) {<br class=""> throw new RuntimeException("Only \
phantom classes loaded, skipping analysis...");<br class=""> }<br class="">}<br \
class=""><br class="">private static void setSparkOptions() {<br class=""> \
Options.v().setPhaseOption("cg.spark", "on");<br class=""> \
Options.v().setPhaseOption("cg.spark", "string-constants:true");<br class="">}<br \
class="">```<br class=""><br class="">Can anyone help me understand what am I doing \
wrong?<span class="Apple-converted-space"> </span><br class=""><br class="">For \
instance, this is a method that is part of the calligraphy:<br class=""><br \
class="">```<br class="">public List<PngPixel> getMergedColors(PngImage image, \
List<PngPixel> colors, long start) {<br class=""> for \
(PngPixel pa : colors) {<br class=""> if \
(!pa.isDuplicate()) {<br class=""> \
...<br class=""> }<br class=""> \
} <span class="Apple-converted-space"> </span><br \
class=""> } <span \
class="Apple-converted-space"> </span><br class="">```<br class=""><br \
class="">but the `isDuplicate()` method is not part of the calligraphy (i.e., the \
`edgesOutOf(getMergedColors)` does not include `isDuplicate`). However, if I change \
the class method to:<br class=""><br class="">```<br class="">public \
List<PngPixel> getMergedColors(PngImage image, List<PngPixel> colors, \
long start) {<br class=""> PngPixel pa = new PngPixel();<br \
class=""> pa.isDuplicate();<br \
class=""> } <span \
class="Apple-converted-space"> </span><br class="">```<br class=""><br \
class="">The `isDuplicate()` method is part of the call graph. Why would creating a \
constructor include the method in the call graph?<br class=""><br class="">Thanks,<br \
class=""><br class="">Miguel<br \
class="">_______________________________________________<br class="">Soot-list \
mailing list<br class=""><a href="mailto:Soot-list@CS.McGill.CA" \
class="">Soot-list@CS.McGill.CA</a><br class=""><a \
href="https://mailman.cs.mcgill.ca/mailman/listinfo/soot-list" \
class="">https://mailman.CS.McGill.CA/mailman/listinfo/soot-list</a></div></span></font></div></div></blockquote></div><br \
class=""></div></body></html>
_______________________________________________
Soot-list mailing list
Soot-list@CS.McGill.CA
https://mailman.CS.McGill.CA/mailman/listinfo/soot-list
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic