[prev in list] [next in list] [prev in thread] [next in thread]
List: wsf-javascript-dev
Subject: [Dev] Automated Test Generation Feature for Language Server
From: Rasika Perera <rasikap () wso2 ! com>
Date: 2018-09-18 10:16:57
Message-ID: CA++WPhYFe+CH=4a8Y26uZUXkb2EAyujhoOLuXzDk76nNyYcBjQ () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/related)]
[Attachment #4 (multipart/alternative)]
Hi Devs,
I am working on "Automated Test Generation" feature for the Language
Server(LS). The initial plan is to support top-level constructs such
as functions,
HTTP services, Endpoints, WebSocket services, and WebSocket client services.
Target generated test content is as below;
*Source Content:*
function *calculate**(int n1, int n2)* returns int {
return 1000;
}
*Generated Test Content:*
import ballerina/test;
import ballerina/log;
@test:BeforeSuite
function beforeTest(){
log:printInfo("Running Test setup in beforeTest function");
}
@test:Config
function testCalculate() {
int actual = *calculate**(0, 0)*;
int expected = 0;
test:assertEquals(actual, expected, msg = "Return value should be equal
to the expected value");
}
@test:AfterSuite
function afterTest(){
log:printInfo("Running Test setup in afterTest function");
}
Since only the *variable type* is realizable during the compile time; all
the parameters passed are kept in its default values. The user needs to
modify them as required. Similarly, whenever there's an *HTTP Service*, a
Test HTTP client will be generated invoking all calculated HTTP resource
paths with default payloads.
Class Diagram Representation
Below is the Class Diagram representation for the target implementation;
[image: class-diagram.png]
When and How to generate tests
When generating a test file, adding a suffix "_test" can be used(eg.
main_test.bal). When a test file already exists, the current approach is to
add a counter at the end(eg. main1_test.bal).
Did a small PoC on how to integrate test generation in VSCode.
[image: ezgif.com-optimize.gif]
Moving forward; we need to finalize when and how to generate test
files. Supporting
future user modifications is another concern when generating test cases. There
are two approaches for the test generation;
1.
Generate Tests for the whole source file: Tests will be generated for
the all available constructs. Even in this approach, we may ask the user
what to generate by showing them a window(eg. TestNG test generation in
IntelliJ). However, when considering VSCode developer experience it
might not be the case.
2.
Continuously Generate Tests per top-level construct: For example; if a
user needs to add a test function for function foo(), he can locate the
foo function and trigger "Add a Test" code action. Similarly, he can
invoke "Add a Test" for the all supported constructs.
Since this test generation is supported through the *Language Server
protocol*, feature capabilities are *limited* to the LSP. Another
suggestion is to, *extend* the Language Client to provide a customized UX
for each IDEs. For instance, we might be able to get user input such as
test filename, methods to generate, adding "setup()/tearDown()"
methods...etc before generating the test file.
WDYT? Please let us know your suggestions and feedback!
--
With Regards,
*Rasika Perera*
Senior Software Engineer
Blogs: medium.com/@rasikaperera, tiriboy.blogspot.com
M: +9471 680 9060 | PGP Key: 56E03A48
LinkedIn: http://lk.linkedin.com/in/rasika90
<http://wso2.com/signature>
WSO2 Inc. www.wso2.com
lean.enterprise.middleware
[Attachment #7 (text/html)]
<div dir="ltr"><div class="gmail_default" \
style="font-family:tahoma,sans-serif;color:#073763"><p dir="ltr" \
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent; \
font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Hi \
Devs,</span></p><p dir="ltr" \
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent; \
font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br></span></p><p \
dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent; \
font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">I \
am working on "Automated Test Generation" feature for the Language Server(LS). The \
initial plan is to support top-level constructs such as </span><span \
style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent; \
font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">functions, \
HTTP services, Endpoints, WebSocket services, </span><span \
style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent; \
font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">and</span><span \
style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent; \
font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"> \
WebSocket client services</span><span \
style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent; \
font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">. \
Target generated test content is as below;</span></p><br \
style="color:rgb(34,34,34);font-family:Arial,Helvetica,sans-serif"><p dir="ltr" \
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent; \
font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><u>Source \
Content:</u></span></p><p dir="ltr" \
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;fo \
nt-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">function \
</span><span style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;fo \
nt-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><b>calculate</b></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;fo \
nt-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><b>(int \
n1, int n2)</b> returns int {</span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"> \
return 1000;</span><span style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;fo \
nt-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">}</span></p><u \
style="color:rgb(34,34,34);font-family:Arial,Helvetica,sans-serif"><br></u><p \
dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent; \
font-weight:700;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><u>Generated \
Test Content:</u></span></p><p dir="ltr" \
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;fo \
nt-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">import \
ballerina/test;</span><span style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;fo \
nt-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">import \
ballerina/log;</span><span style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;fo \
nt-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">@test:BeforeSuite</span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;fo \
nt-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">function \
beforeTest(){</span><span style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"> \
log:printInfo("Running Test setup in beforeTest function");</span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;fo \
nt-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">}</span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;fo \
nt-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">@test:Config</span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;fo \
nt-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">function \
testCalculate() {</span><span style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"> \
int actual = </span><span style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;fo \
nt-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><b>calculate</b></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;fo \
nt-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><b>(0, \
0)</b>;</span><span style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"> \
int expected = 0;</span><span style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"> \
test:assertEquals(actual, expected, msg = "Return value should be equal to the \
expected value");</span><span style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;fo \
nt-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">}</span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;fo \
nt-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">@test:AfterSuite</span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;fo \
nt-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">function \
afterTest(){</span><span style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"> \
log:printInfo("Running Test setup in afterTest function");</span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;fo \
nt-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">}</span><span \
style="font-size:11pt;font-family:"Courier \
New";color:rgb(0,0,0);background-color:transparent;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"><br \
class="gmail-m_-2479759050420295717gmail-kix-line-break"></span></p><p \
style="color:rgb(34,34,34);font-family:Arial,Helvetica,sans-serif;line-height:1.38;margin-top:0pt;margin-bottom:0pt"><font \
color="#000000"><span style="font-size:14.6667px;white-space:pre-wrap"><font \
face="arial, helvetica, sans-serif">Since only the <b>variable type</b> is \
realizable during the compile time; all the parameters passed are kept in its default \
values. The user needs to modify them as required. Similarly, whenever there's an \
<b>HTTP Service</b>, a Test HTTP client will be generated invoking all calculated \
HTTP resource paths with default payloads.</font></span></font></p><p dir="ltr" \
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><br></p><p dir="ltr" \
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span \
id="gmail-m_-2479759050420295717gmail-docs-internal-guid-ec6162c0-7fff-72f0-ca2b-46a7afd42c7c"><span \
style="font-size:14pt;font-family:Arial;color:rgb(67,67,67);background-color:transpare \
nt;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Class \
Diagram Representation</span></span><br></p><p dir="ltr" \
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:14pt;font-family:Arial;color:rgb(67,67,67);background-color:transpare \
nt;font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br></span></p><p \
dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent; \
font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap">Below \
is the Class Diagram representation for the target implementation;</span></p><p \
dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent; \
font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br></span></p><p \
dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent; \
font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"></span></p><div><img \
src="cid:ii_jm7izmy61" alt="class-diagram.png" width="955" height="586" \
style="margin-right: 0px;"> <br></div><p dir="ltr" \
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span \
style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent; \
font-variant-numeric:normal;font-variant-east-asian:normal;vertical-align:baseline;white-space:pre-wrap"><br></span></p><div \
style="color:rgb(34,34,34);font-family:Arial,Helvetica,sans-serif"><span \
id="gmail-m_-2479759050420295717gmail-docs-internal-guid-a5cddc43-7fff-2b1c-372d-a10644794186"><p \
dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span \
--0000000000003737a10576226e3f--
["ezgif.com-optimize.gif" (image/gif)]
["class-diagram.png" (image/png)]
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic