Skip to content

Development Testing

Jump to bottom
Leo Gassman edited this page Apr 28, 2017 · 14 revisions

Running Tests

Test suites are no longer necessary. To run all the tests, just right-click on org.uqbarproject.wollok.test project and run as JUnit tests.

Another way is to simply run

cd org.uqbar.project.wollok.releng/
mvn clean install

That will run all of our tests cases. That includes:

  • JUnit test cases
  • XPect tests

Writing Tests

There are a number of features that we might want to test in wollok, and depending on the nature of the feature there is a different strategy to write a test for it.

Here are some:

  • Runtime behaviour
  • Static Validations/Checks
  • Formatter
  • Autocomplete/Code Assist
  • Quick Fixes
  • Scoping
  • Debugger

Runtime Behaviour (interpreter execution)

This tests validates the expected behaviour of a wollok program execution. For example when if I call a method that changes the value of an instance variable, then I expect exactly that. After calling it, the instance variable must have changed.

The best way to test this is to extend from

org.uqbar.project.wollok.tests.interpreter.AbstractWollokInterpreterTestCase

Then the tests methods are basically wollok code as a rich string which will include this pattern:

   << some code to be tested >>
   << some code that executes the case >>
   << some code asserting expecting post execution state >>

Then basically the wollok program is already self-testeable so you just need to call a helper superclass method which will fail in case the wollok program fails or if the assertions fails.

Here is a sample method:

class ListTestCase extends AbstractWollokInterpreterTestCase {
	
	@Test
	def void testSize() {
		'''
		program p {
			const numbers = #{23, 2, 1}		
			assert.equals(3, numbers.size())
		}'''.interpretPropagatingErrors
	}
}

Also sometimes you need to define an scenario simulating more than one "wollok file". You can use xtend list literals and an similar extension method as the one we've seen before:

    @Test
    def void testSuperInvocation() { #['''
	class Golondrina {
		var energia = 100
		
		method energia() {
		    return energia
		}
		
		method volar(kms) {
		    energia = energia - kms
		}
	}
	''',
	'''
	program p {
	    val pepita = new Golondrina()
            pepita.volar(40)
            assert.equals(60,pepita.energia())
	}'''].interpretPropagatingErrors
    }

Static Validations/Checks

Wollok has a lot of static code analysis. Meaning that there are some validations that are performed against the code source without having to run it. For example, when you refer to an instance variable within a class method, then it can check wether that instance variable exists in the current class. And it will present an error in case it cannot be found.

To test those validations we use a framework called XPect which allows us to write the test in a declarative way and makes it really simple.

The idea is to write a file as if it was the wollok program (or library) itself (it is actually) but including some expectations within that code in the form of wollok comments.

To see examples of this you can checkout any file in the "tests" project with extension

   *.wlk.xt

Here is a really simple test for the class name validation

/* XPECT_SETUP org.uqbar.project.wollok.tests.xpect.WollokXPectTest END_SETUP */

	// XPECT errors --> "Class name must start with uppercase" at "carlota"
	class carlota {
	}

Formatter

We are currently adding them as test methods in

org.uqbar.project.wollok.tests.xpect.formatter.WollokFormatterTestCase

It provides a simple method that receives a piece of code, and a expected formatted text of the code.

Here is an example:

    @Test
    def void testSimpleProgramWithVariablesAndMessageSend() throws Exception {
    	assertFormatting('''program p { val a = 10 val b = 20 this.println(a + b) }''',
        '''
        
        program p {
        	val a = 10
        	val b = 20
        	this.println(a + b)
        }''')
    }

Autocomplete (CodeAssists / TemplateProposals)

This is also static so we use XPECT for this tests. We have an specific test class that extends XPECT to add a new check for proposals.

org.uqbar.project.wollok.tests.proposals.WollokContentAssistTest

Here is a sample test:

/* XPECT_SETUP org.uqbar.project.wollok.tests.proposals.WollokContentAssistTest END_SETUP */

class Golondrina {
	var energia = 100
	method getEnergia() {
		/* XPECT proposals at 'energia' ---
			"Value", #, (,
			Create Object - Create a new object element, [, false, if, new, null, object, return, super, self, throw, true, try, {
		--- */
		return energia
	}
}

This tells that when autocompleting at "energia" (just after the return) it expect a list with the following proposals:

  • Value

  • (
  • Create Object - Create a new object element,
  • [
  • false
  • if
  • new
  • null
  • object
  • return
  • super
  • self
  • throw
  • true
  • try
  • {

Quick Fixes

Tests the transformations done by quickfixes. This kinds of tests inherits from AbstractWollokQuickFixTestCase. They look like this (initial code -> fixed code)

@Test
def testCapitalizeName(){
		val initial = #['''
		class myClass{
			method someMethod(){
				return null
			}
		}
	''']

	val result = #['''
		class MyClass{
			method someMethod(){
				return null
			}
		}
	''']
	assertQuickfix(initial, result, Messages.WollokDslQuickfixProvider_capitalize_name)
}

The last parameter is the message of the quickfix to apply

Scoping

Scoping can also be checked with XPect Although we are not doing that already :( We should spent some time researching how to do this.

Debugger

There are many ways to test the debugger, since it is a complex architecture. When you actually debug a program using the IDE, it is launching a separated JVM so to be able to debug it the IDE "connects" to that wollok VM. This means that the debugger is implemeted as an interprocess comunication. In particular we use RMI. So there are several tests covering different parts of that whole architecture.

Tests classes

  • DebugWithoutThreadingTestCase: it tests the execution of a program without the sockets communication (all in the same VM) and actually without the real debugger implementation that handles multithreading to be able to pause, resume, etc. So basically is more like an "interpreter" test than a debugger test. It just assert program elements evaluation order.
  • InterpreterEvaluationOrderTestCase: our first test. Similar to the previously mentioned, just that it uses Mockito. It asserts over the way the interpreter notifies the debugger (which is a listener)
  • AbstractXDebuggingTestCase: this is the most complete kind of test for the debugger. It sets-up the communication between the debugger client and the wollok JVM just like in a real execution with RMI. It also uses the XDebuggerImpl which means multithreading and ability to pause, resume, step-into, etc.. execution. We are building a DSL like infrastructure for this kind of tests, since having such a power means the test could be complex to write. This is the recommended way to go if you want to add new tests to the debugger

Sample AbstractXDebuggingTestCase test:

        @Test	
	def void hittingABreakPointShouldRiseAndEvent() {
		'''
			program abc {
				console.println("hello")
				var a = 123
				console.println("a is" + a)
			}
		'''.debugSession [
			setBreakPoint(4)
		   	expect [ 
				on(suspended).thenDo[ resume ]
				on(breakPointHit(4))
					.checkThat [vm |
						val frames = vm.stackFrames
						// 1 stack level
						assertNotNull(frames)
						assertEquals(1, frames.size)
						
						// 1 variable (a = 123)
						assertEquals(1, frames.get(0).variables.size)
						frames.get(0).variables.get(0) => [
							assertEquals("a", variable.name)
							assertEquals("123", value.stringValue)	
						]
					]
					.thenDo [ resume ]
			]
		]		
	}

This will set a breakpoint on the last line of the program and assert that there's an "a" variable in the current stackframe with value "123". Then it will resume execution.

Notice this initial step

	on(suspended).thenDo[ resume ]

This will probably need to be done on each test, since when the wollok interpreter runs with the real debugger it pauses automatically just before starting to evaluate, waiting for the client to call "resume".

So here is a template for this kind of tests

    @Test	
	def void mydebuggertest() {
		'''
            // WOLLOK CODE
		'''.debugSession [
			setBreakPoint(4)
		   	expect [ 
				on(/* EVENT */)
                  .checkThat [vm | /* CHECK */ ]
                  .thenDo[ /* ACTION */ ]
                on(...)
			]
		]		
	}

Where EVENTs are provided by static methods from the superclass:

  • started()
  • suspended()
  • breakPointHit(nr)

CHECK is basically your code doing assertions. The "vm" parameter gives you the remote interface to the wollok interpreter so you can call "getStackFrames()" for example, to retrieve the actual state of the vm.

ACTIONs are also provided as methods of the closure single parameter (there avoided by using xtend's it implicit receptor). The type of this object is DebugCommandHandler. Some of the actions:

  • pause()
  • resume()
  • stepOver()
  • stepInto()
  • stepReturn()
  • exit()
  • setBreakpoint(URI, lineNumber)
  • clearBreakpoint(URI, lineNumber)
Clone this wiki locally