From gtk-devel Thu Dec 20 16:51:43 2007 From: Tim Janik Date: Thu, 20 Dec 2007 16:51:43 +0000 To: gtk-devel Subject: Test reports and commit policies Message-Id: X-MARC-Message: https://marc.info/?l=gtk-devel&m=119816953323063 Hey All. A quick update on the unit test reports, a script for test report generation has now been comitted to GLib and will be used to generate HTML reports for the test report rules test-report, perf-report and full-report. The reports should render in all browsers and support colorization and collapsed/popup details for CSS and javascript enabled browsers. Example reports generated from today's SVN are provided here: http://people.imendio.com/timj/examples/glib-full-report.html http://people.imendio.com/timj/examples/gtk+-full-report.html Percentages are reported for the amount of tests that failed or succeeded, e.g. gtk+/tests/.libs/lt-objecttests is reported with a result of 80.39%, because ca. 20% of our automated widget property tests fail (generate some warnings or crash Gtk+, volounteers to look into those cases highly needed). It'd be nice if someone in the community could set up a build machine for jhbuild/modulesets/gtk.modules that builds, runs test-report, perf-report and full-report and uploads the resulting test-report-html, perf-report.html and full-report.html from GLib and Gtk+ to some public location. About commit policies, now that we have a test framework in place that distinguishes between slow (make full-report) and fast (make check) tests, and allows subcomponent related tests (e.g. make check -C gtk+/gtk/), it'd be nice to actually make ue of it. So i'd like to suggest that we establish some basic commit/test policies, e.g.: 1) Before committing changes in glib/ or gtk+/, a developer needs to ensure that make check -C glib/ (or make check -C gtk+/) passes. 2) For major changes (adding new components, changing lots of files, changing Makefiles, etc.), developers should run make full-report on the whole module and investigate the failing test cases before committing. The exact rules/process is open for discussion of course, the above is meant as some initial food for thought. Please discuss! ;-) --- ciaoTJ _______________________________________________ gtk-devel-list mailing list gtk-devel-list@gnome.org http://mail.gnome.org/mailman/listinfo/gtk-devel-list