[prev in list] [next in list] [prev in thread] [next in thread]
List: kde-commits
Subject: [kdev-ruby] parser: Better test suite based on assert_same and
From: Alexander Dymo <adymo () kdevelop ! org>
Date: 2011-08-11 21:53:33
Message-ID: 20110811215333.96AB7A60BE () git ! kde ! org
[Download RAW message or body]
Git commit 82fa991ad723e15dfa680cd52e95f5eb736c87de by Alexander Dymo.
Committed on 11/08/2011 at 23:52.
Pushed by dymo into branch 'master'.
Better test suite based on assert_same and test/unit instead of a our own suite.rb.
M +47 -18 parser/README
A +43 -0 parser/tests/parser_test.rb
D +0 -113 parser/tests/suite.rb
http://commits.kde.org/kdev-ruby/82fa991ad723e15dfa680cd52e95f5eb736c87de
diff --git a/parser/README b/parser/README
index a475c7e..d1c5d41 100644
--- a/parser/README
+++ b/parser/README
@@ -1,7 +1,7 @@
README
======
-Intro:
+== Intro ==
This is a bison-generated pure LALR parser. It's based on the MRI parser
but not in a "strong way". This parser has been designed to be small,
@@ -20,7 +20,7 @@ is used by the parser to match keywords quickly. All these files can be
generated by executing the tools/generate.rb script.
-The parser's interface:
+== The parser's interface ==
Now let's talk about what kind of interface can this parser provide
to the programmer in order to make things easier. In general, the header
@@ -37,21 +37,50 @@ as its second parameter. It will return the generated AST (RubyAst).
Once we don't need this AST anymore, we should call the function rb_free.
-Testing and Debugging the parser:
-
-When we execute the "ruby-parser" executable with a given ruby file,
-it will output a set of numbers. So, let's see what you should do to
-perform the tests. We're going to go inside the directory tools/
-and execute the script suite.rb. This executes the parser debugging
-utility ("ruby-parser") for each test inside the tests/ directory and
-finally it makes a diff between the expected output and the one given by
-the parser. We cannot execute a single test file, so if we really want to do
-it, we just have to do it by hand executing the parser with the test file
-as a parameter. As you will see, the output is a set of integers, and sometimes
-some names between parenthesis. To make it more human-readable, there's the
-script fancy.rb (also at the tools/ directory) that will translate those
-integers into strings, showing their meaning.
-But, what represents all those integers from the output ? In short,
+== Testing and Debugging the Parser ==
+
+ruby-parser:
+ command line tool to output the AST, verbatim from bison,
+ nodes are represented as numbers
+ usage:
+ cd <build directory>
+ parser/ruby-parser <ruby file>
+ to see the human-readable AST representation use fancy.rb
+ cd <build directory>
+ parser/tests/fancy.rb <test filename>
+
+To execute the test suite:
+ cd <build directory>
+ make && make install # tests do not currently run without installation
+ ruby parser/tests/parser_test.rb
+
+Test suite will show a diff for any test failure and ask whether you want to
+accept the change. If yes, the change will be written to the test file as
+reference output. If no, test will fail.
+
+Additional options to test suite:
+ruby parser/tests/parser_test.rb -- --no-interactive
+ skips all questions and just reports failures
+ruby parser/tests/parser_test.rb -- --autoaccept
+ prints diffs and automatically accepts all changes
+
+To run a single test:
+ cd <build directory>
+ make && make install # tests do not currently run without installation
+ ruby parser/tests/parser_test.rb -n test_<test file basename without extension>
+
+To add a new test:
+ - create a new test file under parser/tests/data/
+ - cd <build directory>
+ - make && make install
+ - rerun test suite
+ ruby parser/tests/parser_test.rb
+ - review and accept the test output
+ - copy test output (<testname>.txt file) from build dir to source dir
+ - commit both test file and test output
+
+Additional info on parser output:
+What represents all those integers from the output ? In short,
it's the representation of an AST printed in pre-order. As you
will see, the parser tries to beautify this output by telling you if the
expression is a condition inside of, for example, a for statement, or it will
@@ -62,7 +91,7 @@ and a complete mess. In those cases, experience and patience
will be our friends ;)
-Character encodings:
+== Character encodings ==
As stated before, this parser is meant to be simple and small. This means
that by now we only support UTF-8 encoding. This doesn't mean that
diff --git a/parser/tests/parser_test.rb b/parser/tests/parser_test.rb
new file mode 100644
index 0000000..75cbbb8
--- /dev/null
+++ b/parser/tests/parser_test.rb
@@ -0,0 +1,43 @@
+#
+# This file is part of KDevelop
+# Copyright (C) 2011 Alexander Dymo <adymo@kdevelop.org>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+
+require 'test/unit'
+require File.dirname(__FILE__) + '/assert_same/init'
+
+class AssertSameTest < Test::Unit::TestCase
+
+ Dir["#{File.dirname(__FILE__)}/data/*.rb"].each do |ruby_file|
+
+ define_method "test_#{File.basename(ruby_file, ".rb")}" do
+ log_file = ruby_file.gsub(/rb$/, 'txt')
+ ast = get_ast(ruby_file)
+ assert_same ast, :log => log_file
+ end
+
+ end
+
+private
+
+ def get_ast(filename)
+ # FIXME: adymo: properly find ruby-parser executable
+ `#{File.dirname(__FILE__)}/../ruby-parser #{filename}`
+ end
+
+ # Note, private section, don't add tests here
+end
diff --git a/parser/tests/suite.rb b/parser/tests/suite.rb
deleted file mode 100644
index 071876b..0000000
--- a/parser/tests/suite.rb
+++ /dev/null
@@ -1,113 +0,0 @@
-#
-# This file is part of KDevelop
-# Copyright (C) 2010 Miquel Sabaté <mikisabate@gmail.com>
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-#
-
-#!/usr/bin/env ruby
-
-##
-# This script executes "ruby-parser" to test every *.rb file
-# located in parser/tests/ and outputs in a file. Then it makes
-# a diff with the generated files against the expected results.
-# Finally it outputs the results and it cleans the parser/tests
-# directory.
-# You can also pass the name of a test as a parameter so you
-# can run it.
-
-require 'find'
-require 'fileutils'
-
-
-##
-# This is meant to return the name of the test from the given path.
-#
-# @param *String* path The path of the test.
-def sanitize(path)
- path.gsub /..\/tests\/?/, ''
-end
-
-##
-# This function does the dirty job. Basically it executes the parser to create
-# a file containing the output of the parser. The next step is to make a diff
-# between the expected results and the generated output. It detects this cases:
-# - Test OK if all was ok.
-# - Test Failed: if something went wrong
-# -> Bad Syntax if the parser "disliked" our code.
-#
-def do_diff(str)
- res = :fail
- pid = fork do
- exec("cd .. && ./ruby-parser tests/#{str} > tests/#{str}.out")
- end
- Process.waitpid(pid)
-
- expected = str.gsub(/\.rb$/, '.txt')
- unless File.exists?("../tests/#{expected}")
- puts "Missing expected test output"
- FileUtils::touch("../tests/#{expected}")
- end
-
- pid = fork do
- exec("cd ../tests && diff -swu #{expected} #{str}.out > output.txt")
- end
- Process.waitpid(pid)
- error = 0
- lines = []
- IO.foreach('../tests/output.txt') do |line|
- if line.include?('Error:')
- error = 1
- puts "Test Failed: #{str}. Bad Syntax."
- res = :fail
- break
- end
- lines << line
- end
- if error == 0
- if lines.size == 1
- puts "Test OK: #{sanitize(str)}"
- res = :ok
- else
- puts "Test Failed: #{sanitize(str)}. Generated diff:\n"
- puts lines
- res = :fail
- end
- end
- exec("rm -rf ../tests/#{str}.out ../tests/output.txt") if fork.nil?
- return res
-end
-
-# We just want to run a single test, so run it and then exit
-unless ARGV[0].nil?
- res = do_diff('../tests/' + ARGV[0])
- exit
-end
-
-# Counting the results obtained
-fails = 0
-oks = 0
-
-# Main procedure. It calls do_diff for all *.rb files located in parser/tests/
-Find.find('../tests') do |f|
- str = sanitize(f)
- unless str.nil?
- if str.end_with?('rb')
- res = do_diff(str)
- res == :fail ? fails += 1 : oks += 1
- end
- end
-end
-puts "Number of tests ok: #{oks}\nNumber of tests failed: #{fails}"
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic