'Re: [cfe-dev] [CrossTU] [CTU] Status of Cross Translation Unit Static Analysis, and a new build serv'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       cfe-dev
Subject:    Re: [cfe-dev] [CrossTU] [CTU] Status of Cross Translation Unit Static Analysis, and a new build serv
From:       Adrian Prantl via cfe-dev <cfe-dev () lists ! llvm ! org>
Date:       2019-11-20 19:14:07
Message-ID: 21F8DD7E-A671-4EB0-9FD3-2D57B1305258 () apple ! com
[Download RAW message or body]

[Attachment #2 (multipart/alternative)]

> On Nov 20, 2019, at 11:10 AM, David Blaikie <dblaikie@gmail.com> wrote:
> 
> 
> 
> On Wed, Nov 20, 2019 at 2:47 AM Gábor Márton <martongabesz@gmail.com \
> <mailto:martongabesz@gmail.com>> wrote:
> > Any chance this could be wired up to the existing buildbot or jenkins \
> > infrastructure, rather than having another one?
> Parallel to the Jenkins job we started to work on a simplified buildbot instance, \
> see https://reviews.llvm.org/D61848 <https://reviews.llvm.org/D61848>. The process \
> of integrating that, however, seems to be stuck. Maybe Galina and Endre could \
> revive that process? Also, the Jenkins job is long running job, it takes 8 hours to \
> complete (8cores/64GB azure vm), but it analyzes 6 C/C++ projects. The buildbot is \
> quick, it takes roughly 1-2 hours, but analyzes only one simple C project. 
> About Jenkins in LLVM I have many infrastructural questions:
> Is there a way to integrate our Jenkins job to any LLVM Jenkins cluster? I know \
> about http://green.lab.llvm.org/green/ <http://green.lab.llvm.org/green/>,  but \
> that cluster seems to be OSX only. We need Linux. Is there a Jenkins cluster of \
> Linux executors we could use? Should we somehow connect our Azure vm into that \
> cluster as an additional executor? 
> Hey Adrian - is there any likelihood/interest in having Green Dragon include \
> non-apple-managed builders/scenarios in part to avoid proliferation of distinct CI \
> systems?

+Azhar manages Green Dragon and is in a better position to answer that question.

-- adrian

> 
> 
> Gabor
> 
> On Tue, Nov 19, 2019 at 8:48 PM David Blaikie <dblaikie@gmail.com \
> <mailto:dblaikie@gmail.com>> wrote: Any chance this could be wired up to the \
> existing buildbot or jenkins infrastructure, rather than having another one? 
> (I'm not 100% sure it's better for it to all be together, but figured I'd \
> ask/perhaps there are reasons that've already been considered/articulated) 
> On Mon, Nov 18, 2019 at 9:18 AM Gábor Márton via cfe-dev <cfe-dev@lists.llvm.org \
> <mailto:cfe-dev@lists.llvm.org>> wrote: Dear Clang Community,
> 
> TLDR; I'd like to encourage you to experiment with CTU static analysis because it \
> has evolved a lot this year! Usage documentation \
> <https://clang.llvm.org/docs/analyzer/user-docs/CrossTranslationUnit.html> is \
> available online. 
> This year, we have landed several ASTImporter patches that made it possible to \
> analyze even C++11/14 projects with reasonable stability. I can confidently assure \
> you that the upstream master of llvm/llvm-project is as stable as our downstream \
> fork. We've made enormous efforts to properly implement the error handling of \
> ASTImporter and this way we could dramatically improve the stability of CTU. We do \
> have a few more patches <https://github.com/Ericsson/clang/projects/2> that we \
> still want to land in the future, but they are not error handling related. From \
> Clang version 10, we no longer plan to maintain our CTU downstream fork. 
> We have set up a publicly available Jenkins build server \
> <http://codechecker-buildbot.eastus.cloudapp.azure.com:8080/job/ctu_pipeline_clang-master-monorepo/> \
>                 that continuously analyzes the following C and C++ projects:
> - Tmux (C)
> - Curl (C)
> - Redis (C)
> - Xerces (C++14)
> - Bitcoin (C++11)
> - Protobuf (C++11/C++14)
> CTU analysis results are compared to non-CTU results \
> <http://codechecker-buildbot.eastus.cloudapp.azure.com:8080/job/ctu_pipeline_clang-master-monorepo/CTU_20results_20on_20open_20projects_20_28C_29/>, \
> both for C projects and C++ projects. CTU always results in more findings, and the \
> false-positive/true-positive ratio remains roughly the same [1]. We are monitoring \
> the analysis job, and if an assertion/crash happens then we are going to get in \
> touch with the author of the commit that plausibly caused the error. We also have a \
> buildbot patch <https://reviews.llvm.org/D61848> in Phabricator which is ought to \
> analyse only one simple C project (Tmux). We decided to install  Jenkins because of \
> ownership and flexibility. The buildbot code is pretty convoluted and review and \
> communication are very slow. The buildbot also uses CodeChecker as a dependency, \
> changes to its workflow requires changes in the buildbot configuration as well. 
> Notes: 
> Please note that our primary target for CTU is Linux, and we encourage everybody to \
> use CodeChecker <https://github.com/Ericsson/codechecker> for CTU (scan-build is no \
> longer supported by CTU developers). Projects that use templates heavily may expect \
> an increase in analysis time. Also note that CTU can be very memory consuming, e.g. \
> in case of the analysis run on LLVM code we've seen 10GB of resident memory usage \
> for one process. Thus, it may be useful to set a maximum limit for the loaded \
> ASTUnits (e.g. -analyzer-config ctu-import-threshold=8, with CodeChecker you need \
> to edit an saargs file \
> <https://github.com/Ericsson/codechecker/blob/master/docs/analyzer/checker_and_analyzer_configuration.md>).
>  
> [1] 2017 EuroLLVM Developers' Meeting: G. Horvath "Cross Translational Unit \
> Analysis in Clang ..." 
> Cheers,
> Gabor Marton
> _______________________________________________
> cfe-dev mailing list
> cfe-dev@lists.llvm.org <mailto:cfe-dev@lists.llvm.org>
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev \
> <https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev>

[Attachment #5 (unknown)]

<html><head><meta http-equiv="Content-Type" content="text/html; \
charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; \
line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote \
type="cite" class=""><div class="">On Nov 20, 2019, at 11:10 AM, David Blaikie &lt;<a \
href="mailto:dblaikie@gmail.com" class="">dblaikie@gmail.com</a>&gt; wrote:</div><br \
class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div \
dir="ltr" class=""><br class=""></div><br class=""><div class="gmail_quote"><div \
dir="ltr" class="gmail_attr">On Wed, Nov 20, 2019 at 2:47 AM Gábor Márton &lt;<a \
href="mailto:martongabesz@gmail.com" class="">martongabesz@gmail.com</a>&gt; \
wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" \
class="">&gt; Any chance this could be wired up to the existing buildbot or jenkins \
infrastructure, rather than having another one?<div class="">Parallel to the Jenkins \
job we started to work on a simplified buildbot instance, see <a \
href="https://reviews.llvm.org/D61848" target="_blank" \
class="">https://reviews.llvm.org/D61848</a>. The process of integrating that, \
however, seems to be stuck. Maybe Galina and Endre could revive that process? Also, \
the Jenkins job is long running job, it takes 8 hours to complete (8cores/64GB azure \
vm), but it analyzes 6 C/C++ projects. The buildbot is quick, it takes roughly 1-2 \
hours, but analyzes only one simple C project.</div><div class=""><br \
class=""></div><div class="">About Jenkins in LLVM I have many infrastructural \
questions:</div><div class="">Is there a way to integrate our Jenkins job to any LLVM \
Jenkins cluster? I know about&nbsp;<a href="http://green.lab.llvm.org/green/" \
target="_blank" class="">http://green.lab.llvm.org/green/</a>,&nbsp; but that cluster \
seems to be OSX only. We need Linux. Is there a Jenkins cluster of Linux executors we \
could use? Should we somehow connect our Azure vm into that cluster as an additional \
executor?</div></div></blockquote><div class=""><br class="">Hey Adrian - is there \
any likelihood/interest in having Green Dragon include non-apple-managed \
builders/scenarios in part to avoid proliferation of distinct CI systems?<br \
class=""></div></div></div></div></blockquote><div><br class=""></div><div>+Azhar \
manages Green Dragon and is in a better position to answer that \
question.</div><div><br class=""></div><div>-- adrian</div><br class=""><blockquote \
type="cite" class=""><div class=""><div dir="ltr" class=""><div \
class="gmail_quote"><div class="">&nbsp;</div><blockquote class="gmail_quote" \
style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"><div dir="ltr" class=""><div class=""><div \
class=""><br class=""></div></div><div class="">Gabor</div></div><br class=""><div \
class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Nov 19, 2019 at 8:48 PM \
David Blaikie &lt;<a href="mailto:dblaikie@gmail.com" target="_blank" \
class="">dblaikie@gmail.com</a>&gt; wrote:<br class=""></div><blockquote \
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="">Any chance this could be \
wired up to the existing buildbot or jenkins infrastructure, rather than having \
another one?<br class=""><br class="">(I'm not 100% sure it's better for it to all be \
together, but figured I'd ask/perhaps there are reasons that've already been \
considered/articulated)</div><br class=""><div class="gmail_quote"><div dir="ltr" \
class="gmail_attr">On Mon, Nov 18, 2019 at 9:18 AM Gábor Márton via cfe-dev &lt;<a \
href="mailto:cfe-dev@lists.llvm.org" target="_blank" \
class="">cfe-dev@lists.llvm.org</a>&gt; wrote:<br class=""></div><blockquote \
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"><div dir="ltr" class=""><div dir="ltr" \
class=""><div style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;fo \
nt-variant-east-asian:inherit;font-stretch:inherit;font-size:15px;line-height:inherit;vertical-align:baseline;color:rgb(32,31,30)" \
class=""><div style="font-size: 11pt; font-family: Calibri, sans-serif; margin: 0px;" \
class=""><span style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; \
font-variant-ligatures: inherit; font-variant-position: inherit; font-variant-caps: \
inherit; font-variant-numeric: inherit; font-variant-alternates: inherit; \
font-variant-east-asian: inherit; font-weight: inherit; font-stretch: inherit; \
font-size: 12pt; line-height: inherit; font-family: inherit; vertical-align: \
baseline;" class="">Dear Clang Community,</span></div><div style="font-size: 11pt; \
font-family: Calibri, sans-serif; margin: 0px;" class=""><span style="margin: 0px; \
padding: 0px; border: 0px; font-style: inherit; font-variant-ligatures: inherit; \
font-variant-position: inherit; font-variant-caps: inherit; font-variant-numeric: \
inherit; font-variant-alternates: inherit; font-variant-east-asian: inherit; \
font-weight: inherit; font-stretch: inherit; font-size: 12pt; line-height: inherit; \
font-family: inherit; vertical-align: baseline;" class=""><br \
class=""></span></div><div style="font-size: 11pt; font-family: Calibri, sans-serif; \
margin: 0px;" class=""><span style="margin: 0px; padding: 0px; border: 0px; \
font-style: inherit; font-variant-ligatures: inherit; font-variant-position: inherit; \
font-variant-caps: inherit; font-variant-numeric: inherit; font-variant-alternates: \
inherit; font-variant-east-asian: inherit; font-weight: inherit; font-stretch: \
inherit; font-size: 12pt; line-height: inherit; font-family: inherit; vertical-align: \
baseline;" class="">TLDR;&nbsp;</span><span style="font-family: inherit; font-style: \
inherit; font-variant-ligatures: inherit; font-variant-caps: inherit; font-weight: \
inherit; font-size: 12pt;" class="">I'd like to encourage you</span><span \
style="font-family: inherit; font-style: inherit; font-variant-ligatures: inherit; \
font-variant-caps: inherit; font-weight: inherit; font-size: 12pt;" class="">&nbsp;to \
experiment with CTU static analysis because it has evolved a lot this year! Usage <a \
href="https://clang.llvm.org/docs/analyzer/user-docs/CrossTranslationUnit.html" \
target="_blank" class="">documentation</a> is&nbsp;</span><span style="font-family: \
inherit; font-style: inherit; font-variant-ligatures: inherit; font-variant-caps: \
inherit; font-weight: inherit; font-size: 12pt;" class="">available \
online.</span></div><div \
style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-eas \
t-asian:inherit;font-stretch:inherit;line-height:inherit;vertical-align:baseline" \
class=""><p style="font-size:11pt;font-family:Calibri,sans-serif;margin:0px" \
class=""></p></div></div><div \
style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-eas \
t-asian:inherit;font-stretch:inherit;font-size:15px;line-height:inherit;vertical-align:baseline;color:rgb(32,31,30)" \
class=""><p style="font-size:11pt;font-family:Calibri,sans-serif;margin:0px" \
class=""><span style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; \
font-variant-ligatures: inherit; font-variant-position: inherit; font-variant-caps: \
inherit; font-variant-numeric: inherit; font-variant-alternates: inherit; \
font-variant-east-asian: inherit; font-weight: inherit; font-stretch: inherit; \
font-size: 12pt; line-height: inherit; font-family: inherit; vertical-align: \
baseline;" class="">&nbsp;</span></p></div><div \
style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-eas \
t-asian:inherit;font-stretch:inherit;font-size:15px;line-height:inherit;vertical-align:baseline;color:rgb(32,31,30)" \
class=""><div style="font-size: 11pt; font-family: Calibri, sans-serif; margin: 0px;" \
class=""><span style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; \
font-variant-ligatures: inherit; font-variant-position: inherit; font-variant-caps: \
inherit; font-variant-numeric: inherit; font-variant-alternates: inherit; \
font-variant-east-asian: inherit; font-weight: inherit; font-stretch: inherit; \
font-size: 12pt; line-height: inherit; font-family: inherit; vertical-align: \
baseline;" class="">This year, we have landed several&nbsp;</span><span \
style="font-family: inherit; font-size: 12pt; font-style: inherit; \
font-variant-ligatures: inherit; font-variant-caps: inherit; font-weight: inherit;" \
class="">ASTImporter patches that made it possible to analyze even C++11/14 projects \
with&nbsp;</span><span style="font-family: inherit; font-size: 12pt; font-style: \
inherit; font-variant-ligatures: inherit; font-variant-caps: inherit; font-weight: \
inherit;" class="">reasonable stability. I can confidently assure you that the \
upstream master of&nbsp;</span><span style="font-family: inherit; font-size: 12pt; \
font-style: inherit; font-variant-ligatures: inherit; font-variant-caps: inherit; \
font-weight: inherit;" class="">llvm/llvm-project is as stable as our downstream \
fork.&nbsp;</span><span style="font-family: inherit; font-style: inherit; \
font-variant-ligatures: inherit; font-variant-position: inherit; font-variant-caps: \
inherit; font-variant-numeric: inherit; font-variant-alternates: inherit; \
font-variant-east-asian: inherit; font-weight: inherit; margin: 0px; padding: 0px; \
border: 0px; font-stretch: inherit; font-size: 12pt; line-height: inherit; \
vertical-align: baseline;" class="">We've made enormous efforts to properly implement \
the error handling of ASTImporter and this way we could dramatically improve the \
stability of CTU. We do have a <a href="https://github.com/Ericsson/clang/projects/2" \
target="_blank" class="">few more patches</a>&nbsp;that we still want to land in the \
future, but they are not error handling related.&nbsp;</span><span \
style="font-family: inherit; font-style: inherit; font-variant-ligatures: inherit; \
font-variant-caps: inherit; font-weight: inherit; font-size: 12pt;" class="">From \
Clang version 10,&nbsp;</span><span style="font-family: inherit; font-style: inherit; \
font-variant-ligatures: inherit; font-variant-caps: inherit; font-weight: inherit; \
font-size: 12pt;" class="">we no longer plan to maintain our CTU downstream \
fork.</span></div></div><div \
style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-eas \
t-asian:inherit;font-stretch:inherit;font-size:15px;line-height:inherit;vertical-align:baseline;color:rgb(32,31,30)" \
class=""><div style="font-size: 11pt; font-family: Calibri, sans-serif; margin: 0px;" \
class=""><span style="font-family: inherit; font-style: inherit; \
font-variant-ligatures: inherit; font-variant-caps: inherit; font-weight: inherit; \
font-size: 12pt;" class=""><br class=""></span></div><p \
style="font-size:11pt;font-family:Calibri,sans-serif;margin:0px" class=""><span \
style="font-family: inherit; font-style: inherit; font-variant-ligatures: inherit; \
font-variant-caps: inherit; font-weight: inherit; font-size: 12pt;" \
class=""></span></p></div><div \
style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-eas \
t-asian:inherit;font-stretch:inherit;font-size:15px;line-height:inherit;vertical-align:baseline;color:rgb(32,31,30)" \
class=""><div style="font-size: 11pt; font-family: Calibri, sans-serif; margin: 0px;" \
class=""><span style="font-family: inherit; font-style: inherit; \
font-variant-ligatures: inherit; font-variant-position: inherit; font-variant-caps: \
inherit; font-variant-numeric: inherit; font-variant-alternates: inherit; \
font-variant-east-asian: inherit; font-weight: inherit; margin: 0px; padding: 0px; \
border: 0px; font-stretch: inherit; font-size: 12pt; line-height: inherit; \
vertical-align: baseline;" class="">We have set up a <a \
href="http://codechecker-buildbot.eastus.cloudapp.azure.com:8080/job/ctu_pipeline_clang-master-monorepo/" \
target="_blank" class="">publicly available Jenkins build server</a> that \
continuously analyzes the&nbsp;</span><span style="font-family: inherit; font-style: \
inherit; font-variant-ligatures: inherit; font-variant-caps: inherit; font-weight: \
inherit; font-size: 12pt;" class="">following C and C++ projects:</span><br \
class=""></div></div><div \
style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-eas \
t-asian:inherit;font-stretch:inherit;font-size:15px;line-height:inherit;vertical-align:baseline;color:rgb(32,31,30)" \
class=""><div style="font-size: 11pt; font-family: Calibri, sans-serif; margin: 0px;" \
class=""><span style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; \
font-variant-ligatures: inherit; font-variant-position: inherit; font-variant-caps: \
inherit; font-variant-numeric: inherit; font-variant-alternates: inherit; \
font-variant-east-asian: inherit; font-weight: inherit; font-stretch: inherit; \
font-size: 12pt; line-height: inherit; font-family: inherit; vertical-align: \
baseline;" class="">- Tmux (C)</span></div></div><div \
style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-eas \
t-asian:inherit;font-stretch:inherit;font-size:15px;line-height:inherit;vertical-align:baseline;color:rgb(32,31,30)" \
class=""><div style="font-size: 11pt; font-family: Calibri, sans-serif; margin: 0px;" \
class=""><span style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; \
font-variant-ligatures: inherit; font-variant-position: inherit; font-variant-caps: \
inherit; font-variant-numeric: inherit; font-variant-alternates: inherit; \
font-variant-east-asian: inherit; font-weight: inherit; font-stretch: inherit; \
font-size: 12pt; line-height: inherit; font-family: inherit; vertical-align: \
baseline;" class="">- Curl (C)</span></div></div><div \
style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-eas \
t-asian:inherit;font-stretch:inherit;font-size:15px;line-height:inherit;vertical-align:baseline;color:rgb(32,31,30)" \
class=""><div style="font-size: 11pt; font-family: Calibri, sans-serif; margin: 0px;" \
class=""><span style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; \
font-variant-ligatures: inherit; font-variant-position: inherit; font-variant-caps: \
inherit; font-variant-numeric: inherit; font-variant-alternates: inherit; \
font-variant-east-asian: inherit; font-weight: inherit; font-stretch: inherit; \
font-size: 12pt; line-height: inherit; font-family: inherit; vertical-align: \
baseline;" class="">- Redis (C)</span></div></div><div \
style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-eas \
t-asian:inherit;font-stretch:inherit;font-size:15px;line-height:inherit;vertical-align:baseline;color:rgb(32,31,30)" \
class=""><div style="font-size: 11pt; font-family: Calibri, sans-serif; margin: 0px;" \
class=""><span style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; \
font-variant-ligatures: inherit; font-variant-position: inherit; font-variant-caps: \
inherit; font-variant-numeric: inherit; font-variant-alternates: inherit; \
font-variant-east-asian: inherit; font-weight: inherit; font-stretch: inherit; \
font-size: 12pt; line-height: inherit; font-family: inherit; vertical-align: \
baseline;" class="">- Xerces (C++14)</span></div></div><div \
style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-eas \
t-asian:inherit;font-stretch:inherit;font-size:15px;line-height:inherit;vertical-align:baseline;color:rgb(32,31,30)" \
class=""><div style="font-size: 11pt; font-family: Calibri, sans-serif; margin: 0px;" \
class=""><span style="margin: 0px; padding: 0px; border: 0px; font-style: inherit; \
font-variant-ligatures: inherit; font-variant-position: inherit; font-variant-caps: \
inherit; font-variant-numeric: inherit; font-variant-alternates: inherit; \
font-variant-east-asian: inherit; font-weight: inherit; font-stretch: inherit; \
font-size: 12pt; line-height: inherit; font-family: inherit; vertical-align: \
baseline;" class="">- Bitcoin (C++11)</span></div></div><div \
style="margin:0px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-eas \
t-asian:inherit;font-stretch:inherit;font-size:15px;line-height:inherit;vertical-align:baseline;color:rgb(32,31,30)" \
_______________________________________________<br class="">
cfe-dev mailing list<br class="">
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank" \
class="">cfe-dev@lists.llvm.org</a><br class=""> <a \
href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" \
target="_blank" class="">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><br \
class=""> </blockquote></div>
</blockquote></div>
</blockquote></div></div>
</div></blockquote></div><br class=""></body></html>

[Attachment #6 (text/plain)]

_______________________________________________
cfe-dev mailing list
cfe-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

[prev in list] [next in list] [prev in thread] [next in thread] 