[prev in list] [next in list] [prev in thread] [next in thread] 

List:       perl-qa
Subject:    Correctly testing external binaries
From:       "Cosimo Streppone" <cosimo () streppone ! it>
Date:       2013-01-31 7:44:26
Message-ID: op.wrrmgcxxs5ttvb () c0739
[Download RAW message or body]

Hi QA folks,

Net::Statsd::Server is a UDP server daemon packaged as CPAN module,
with a little 'bin/statsd' wrapper script.

In the latest release, 0.09, I tried to add some 'integration tests':

   https://metacpan.org/source/COSIMO/Net-Statsd-Server-0.09/t/integration-tests

implemented through the Test::Statsd module:

   https://metacpan.org/source/COSIMO/Net-Statsd-Server-0.09/lib/Test/Statsd.pm

Now, the idea behind this is that I can perform some server-independent
tests. The start_statsd() method forks off a given binary, with a
config file as only argument, and then tests are performed throwing
UDP packets at some local port.
When tests are done, stop_statsd() kills off the child process.

The goal is to be able to test the multitude of statsd server  
implementations
out there just given a binary and a config file.

Immediate problems:

1) binary can be an executable or a script that needs $^X,
    /nodejs/python/... How do I exec() reliably?

    https://metacpan.org/source/COSIMO/Net-Statsd-Server-0.09/t/integration-tests/counts-are-valid.t

2) Omitting $^X is not desirable, because I got a slew of
    CPAN testers fails. #!/usr/bin/perl is obviously a wrong choice
    in so many situations.

    Sample report:
    http://www.cpantesters.org/cpan/report/02669934-6a35-11e2-b5a4-ab61270fb964

Less immediate problems:

3) Should this generic test suite be an independent CPAN distribution?

I'd appreciate any advice you might have.

-- 
Cosimo
[prev in list] [next in list] [prev in thread] [next in thread]