[prev in list] [next in list] [prev in thread] [next in thread]
List: cfe-dev
Subject: [cfe-dev] Fwd: Proposal: Integrate CodeChecker analyzer infrastructure
From: Anna Zaks via cfe-dev <cfe-dev () lists ! llvm ! org>
Date: 2016-02-27 18:05:24
Message-ID: 37763513-285B-4FF5-AFAC-C52F60D7D862 () apple ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Hi Alex,
The infrastructure that the Ericsson team has built can and should be used for clang \
tidy reported issues as well as the static analyzer. Would be good if you or someone \
else working on clang tidy could chime in on this!
(I've been talking with them off line and asked them to send out this email to gather \
the feedback from the community.) Thanks,
Anna.
> Begin forwarded message:
>
> From: György Orbán via cfe-dev <cfe-dev@lists.llvm.org>
> Date: February 23, 2016 at 2:10:59 AM PST
> To: cfe-dev@lists.llvm.org
> Subject: [cfe-dev] Proposal: Integrate CodeChecker analyzer infrastructure
>
> Hi,
>
> We would like to add CodeChecker (https://github.com/Ericsson/codechecker) analyzer \
> infrastructure.
> This is an alternative tool to scan-build with extended functionality.
> Some of the main features are: track issues over time, suppress false positives, \
> detect new issues by comparing multiple analyzer run results, view and compare \
> results in a web browser or in the command line. A more detailed feature list can \
> be found below (*). The analyzer infrastructure is built in a way that integrating \
> a new analyzer can be easily done. We are developing a tool which can be used \
> easily by the developers or by automated continuous integration tools and view the \
> results from multiple analyzers in a common way. We think it would serve as a good \
> base for displaying and tracking bugs that can be detected by the other clang tools \
> such as clang-tidy which is already supported.
> For example, you can find the analysis results of the LLVM code 3.6.2 and 3.7.1 \
> here: http://modelserver.inf.elte.hu:5000
> Main questions to the community:
> 0. Does the Clang community like the idea?
> 1. CodeChecker has some 3rd party dependencies see below (**), are they acceptable?
> 2. Is the community satisfied with the CodeChecker name?
>
> Integration plan:
> 0. CodeChecker should use scan-build.py (OSX support) to generate the compilation \
> database instead of the current LD_PRELOAD technique 1. Migrate CodeChecker testing \
> infrastructure to the current LLVM testing infrastructure
> (*) Most notably it extends the current tool set with the following features:
> - stores the result of multiple large analysis run results efficiently (opposed to \
> scan-build/scan-view static htmls)
> - run multiple analyzers, currently Clang Static Analyzer and Clang-Tidy is \
> supported
> - dynamic web based defect viewer (instead of static html)
> - a SQLite/PostgreSQL based defect storage & management (both are optional, results \
> can be shown on standard output in quickcheck mode)
> - update analyzer results only for modified files (depends on the build system)
> - compare analysis results (new/resolved/unresolved bugs compared to a baseline)
> - filter analysis results (checker name, severity, source file name ...)
> - skip analysis in specific source directories if required
> - suppression of false positives (in config file or in the source)
> - Thrift API based server-client model for storing bugs and viewing results.
> - It is possible to connect multiple bug viewers. Currently a web-based viewer and \
> a command line viewer are provided. (command line client is the recommended way to \
> connect into Continuous Integration loops)
> Command line examples of usage can be found here: \
> https://github.com/Ericsson/codechecker/blob/master/docs/usage.md
> CodeChecker supports multiple use cases:
> - Small projects/several source files (quick feedback)
> No database is used, analysis results are shown in on the command line only
> - Medium size projects (~500 files)
> Results are stored in SQLite/PostgreSQL database and can be viewed from command \
> line or web viewer clients
> - Large size projects (>500 files)
> Results are stored in PostgreSQL database and can be viewed from command line or \
> web viewer clients
> There are currently discussions about analyzer tool support in multiple email \
> threads:
> http://clang-developers.42468.n3.nabble.com/Idea-for-better-invoking-static-analysis-via-command-line-td4049670.html
> http://clang-developers.42468.n3.nabble.com/Proposal-Integrate-static-analysis-test-suites-td4048967.html
>
> CodeChecker provides solutions for many problems discussed there:
>
> - Problem: Different analyzers provide different output formats (Clang Static \
> Analyzer provides plist/html/command line, Clang-tidy provides command line output \
> only)
> Solution: With Codechecker analyzer results from multiple analyzers can be viewed \
> in a common way for developers or other tools for further result processing.
> - Problem: CC environment variable overwriting by previous scan-build version \
> (written in perl) is not always a good solution.
> Solution: Compilation database is generated by CodeChecker (currently using the \
> LD_PRELOAD technique, later with scan-build.py for OSX support).
> - Problem: Analyzer has multiple command line arguments which could be changed by \
> time, the end users should not be affected.
> Solution: CodeChecker hides the clang analyzer specific options from the user. Many \
> options are preconfigured. But forwarding options without modifications to the \
> analyzers is supported.
> - Problem: Understanding analyzer results might be harder if only command line \
> results are available (currently generated static html sites do not scale and it is \
> hard to manage).
> Solution: Analysis steps can be viewed in command line with quickcheck or in the \
> web viewer (dynamically generated based on the database), which can help to \
> understand the analysis results.
> (**) 3rd party dependencies for various features:
> - Python 2.7.5 (Python Software Foundation) - required to run CodeChecker
> - SQLAlchemy (MIT) - Python SQL toolkit and Object Relational Mapper, for \
> supporting multiple database backends
> - Alembic (MIT) - required for database migration support which is only available \
> for PostgreSQL database
> - pg8000 (BSD) or psycopg2 (LGPL) - at least one database connector is required for \
> PostgreSQL database support (both are supported)
> - Thrift (Apache v2.0) - cross-language service building framework to handle data \
> transfer for report storage and result viewer clients
> - Codemirror (MIT) - view source code in the browser
> - Jsplumb (community edition, MIT) - draw bug paths
> - Marked (BSD) - view documentation for checkers written in markdown (generated \
> dynamically)
> - Dojotoolkit (BSD) - main framework for the web UI
> - Highlightjs (BSD) - required for highlighting the source code
>
> For further information check out our GitHub \
> (https://github.com/Ericsson/codechecker) page.
> Best Regards,
> Gyorgy Orban
> _______________________________________________
> cfe-dev mailing list
> cfe-dev@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
[Attachment #5 (unknown)]
<html><head><meta http-equiv="Content-Type" content="text/html \
charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; \
-webkit-line-break: after-white-space;" class="">Hi Alex,<div class=""><br \
class=""></div><div class="">The infrastructure that the Ericsson team has built can \
and should be used for clang tidy reported issues as well as the static analyzer. \
Would be good if you or someone else working on clang tidy could chime in on \
this!</div><div class=""><br class=""></div><div class="">(I've been talking with \
them off line and asked them to send out this email to gather the feedback from the \
community.)</div><div class="">Thanks,</div><div class="">Anna.<br class=""><div><br \
class=""><blockquote type="cite" class=""><div class="">Begin forwarded \
message:</div><br class="Apple-interchange-newline"><div style="margin-top: 0px; \
margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span \
style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; \
color:rgba(0, 0, 0, 1.0);" class=""><b class="">From: </b></span><span \
style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" \
class="">György Orbán via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" \
class="">cfe-dev@lists.llvm.org</a>><br class=""></span></div><div \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" \
class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, \
sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">Date: </b></span><span \
style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" \
class="">February 23, 2016 at 2:10:59 AM PST<br class=""></span></div><div \
style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" \
class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, \
sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">To: </b></span><span \
style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" \
class=""><a href="mailto:cfe-dev@lists.llvm.org" \
class="">cfe-dev@lists.llvm.org</a><br class=""></span></div><div style="margin-top: \
0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span \
style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; \
color:rgba(0, 0, 0, 1.0);" class=""><b class="">Subject: </b></span><span \
style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" \
class=""><b class="">[cfe-dev] Proposal: Integrate CodeChecker analyzer \
infrastructure</b><br class=""></span></div><br class=""><div class=""><div \
class="">Hi,<br class=""><br class="">We would like to add CodeChecker (<a \
href="https://github.com/Ericsson/codechecker" \
class="">https://github.com/Ericsson/codechecker</a>) analyzer infrastructure.<br \
class=""><br class="">This is an alternative tool to scan-build with extended \
functionality.<br class="">Some of the main features are: track issues over time, \
suppress false positives, detect new issues by comparing multiple analyzer run \
results,<br class="">view and compare results in a web browser or in the command \
line. A more detailed feature list can be found below (*).<br class="">The analyzer \
infrastructure is built in a way that integrating a new analyzer can be easily \
done.<br class="">We are developing a tool which can be used easily by the developers \
or by automated continuous integration tools and view the results from multiple \
analyzers in a common way.<br class="">We think it would serve as a good base for \
displaying and tracking bugs that can be detected by the other clang tools such as \
clang-tidy which is already supported.<br class=""><br class="">For example, you can \
find the analysis results of the LLVM code 3.6.2 and 3.7.1 here: <a \
href="http://modelserver.inf.elte.hu:5000" \
class="">http://modelserver.inf.elte.hu:5000</a><br class=""><br class="">Main \
questions to the community:<br class="">0. Does the Clang community like the idea?<br \
class="">1. CodeChecker has some 3rd party dependencies see below (**), are they \
acceptable?<br class="">2. Is the community satisfied with the CodeChecker name?<br \
class=""><br class="">Integration plan:<br class=""> 0. CodeChecker should use \
scan-build.py (OSX support) to generate the compilation database instead of the \
current LD_PRELOAD technique<br class=""> 1. Migrate CodeChecker testing \
infrastructure to the current LLVM testing infrastructure<br class=""><br \
class="">(*) Most notably it extends the current tool set with the following \
features:<br class=""> - stores the result of multiple large analysis run results \
efficiently (opposed to scan-build/scan-view static htmls)<br class=""> - run \
multiple analyzers, currently Clang Static Analyzer and Clang-Tidy is supported<br \
class=""> - dynamic web based defect viewer (instead of static html)<br class=""> - a \
SQLite/PostgreSQL based defect storage & management (both are optional, results \
can be shown on standard output in quickcheck mode)<br class=""> - update analyzer \
results only for modified files (depends on the build system)<br class=""> - compare \
analysis results (new/resolved/unresolved bugs compared to a baseline)<br class=""> - \
filter analysis results (checker name, severity, source file name ...)<br class=""> - \
skip analysis in specific source directories if required<br class=""> - suppression \
of false positives (in config file or in the source)<br class=""> - Thrift API based \
server-client model for storing bugs and viewing results.<br class=""> - It is \
possible to connect multiple bug viewers. Currently a web-based viewer and a command \
line viewer are provided.<br class=""> (command line client is the \
recommended way to connect into Continuous Integration loops)<br class=""><br \
class="">Command line examples of usage can be found here: <a \
href="https://github.com/Ericsson/codechecker/blob/master/docs/usage.md" \
class="">https://github.com/Ericsson/codechecker/blob/master/docs/usage.md</a><br \
class=""><br class="">CodeChecker supports multiple use cases:<br class=""> - Small \
projects/several source files (quick feedback)<br class=""> \
No database is used, analysis results are shown in on the \
command line only<br class=""> - Medium size projects (~500 files)<br class=""> \
Results are stored in SQLite/PostgreSQL database and can be \
viewed from command line or web viewer clients<br class=""> - Large size projects \
(>500 files)<br class=""> Results are stored in PostgreSQL \
database and can be viewed from command line or web viewer clients<br class=""><br \
class="">There are currently discussions about analyzer tool support in multiple \
email threads:<br class=""><br class=""><a \
href="http://clang-developers.42468.n3.nabble.com/Idea-for-better-invoking-static-analysis-via-command-line-td4049670.html" \
class="">http://clang-developers.42468.n3.nabble.com/Idea-for-better-invoking-static-analysis-via-command-line-td4049670.html</a><br \
class="">http://clang-developers.42468.n3.nabble.com/Proposal-Integrate-static-analysis-test-suites-td4048967.html<br \
class=""><br class="">CodeChecker provides solutions for many problems discussed \
there:<br class=""><br class=""> - Problem: Different analyzers provide different \
output formats (Clang Static Analyzer provides plist/html/command line, Clang-tidy \
provides command line output only)<br class=""> Solution: With \
Codechecker analyzer results from multiple analyzers can be viewed in a common way \
for developers or other tools for further result processing.<br class=""><br \
class=""> - Problem: CC environment variable overwriting by previous scan-build \
version (written in perl) is not always a good solution.<br class=""> \
Solution: Compilation database is generated by CodeChecker (currently \
using the LD_PRELOAD technique, later with scan-build.py for OSX support).<br \
class=""><br class=""> - Problem: Analyzer has multiple command line arguments which \
could be changed by time, the end users should not be affected.<br class=""> \
Solution: CodeChecker hides the clang analyzer specific options from the \
user. Many options are preconfigured. But forwarding options without modifications to \
the analyzers is supported.<br class=""><br class=""> - Problem: Understanding \
analyzer results might be harder if only command line results are available \
(currently generated static html sites do not scale and it is hard to manage).<br \
class=""> Solution: Analysis steps can be viewed in command line with \
quickcheck or in the web viewer (dynamically generated based on the database), which \
can help to understand the analysis results.<br class=""><br class="">(**) 3rd party \
dependencies for various features:<br class=""> - Python 2.7.5 (Python Software \
Foundation) - required to run CodeChecker<br class=""> - SQLAlchemy (MIT) - Python \
SQL toolkit and Object Relational Mapper, for supporting multiple database \
backends<br class=""> - Alembic (MIT) - required for database migration support which \
is only available for PostgreSQL database<br class=""> - pg8000 (BSD) or psycopg2 \
(LGPL) - at least one database connector is required for PostgreSQL database support \
(both are supported)<br class=""> - Thrift (Apache v2.0) - cross-language service \
building framework to handle data transfer for report storage and result viewer \
clients<br class=""> - Codemirror (MIT) - view source code in the browser<br \
class=""> - Jsplumb (community edition, MIT) - draw bug paths<br class=""> - Marked \
(BSD) - view documentation for checkers written in markdown (generated \
dynamically)<br class=""> - Dojotoolkit (BSD) - main framework for the web UI<br \
class=""> - Highlightjs (BSD) - required for highlighting the source code<br \
class=""><br class="">For further information check out our GitHub \
(https://github.com/Ericsson/codechecker) page.<br class=""><br class="">Best \
Regards,<br class="">Gyorgy Orban<br \
class="">_______________________________________________<br class="">cfe-dev mailing \
list<br class="">cfe-dev@lists.llvm.org<br \
class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev<br \
class=""></div></div></blockquote></div><br class=""></div></body></html>
[Attachment #6 (text/plain)]
_______________________________________________
cfe-dev mailing list
cfe-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic