| Author |
Message
|
| AkaiRyujin |
 Posted: Fri Nov 09, 2012 9:12 am Post subject: JavaCompute Unit Testing Framework |
 |
|
Newbie
Joined: 06 Mar 2012
Posts: 7
|
Have despaired at the lack of java testing functionality for message broker, so here's a unit testing framework that I've written that allows mocking and testing of message broker java code.
http://github.com/ukcrpb6/wmb-mock
There are a couple of area's that need improving - (bitstream methods for example) - as well as a few new functions that v8 added over v7. |
|
| Back to top |
|
 |
| goffinf |
| Posted: Fri Nov 09, 2012 12:12 pm Post subject: |
|
|
Chevalier
Joined: 05 Nov 2005
Posts: 401
|
Nice, thanks for contributing, this is one area of the unit test framework that I created that was missing. Normally I tell my Dev's to put the majority of JCN functionality in simple POJOs since that makes them easy to unit test with standard JUnit, but the actual MB API has always been a niggling omission.
The rest of my unit test framework covers ESQL (individual Compute modules, Filter nodes, independent function and procedures) + and end-to-end testing over a range of transports and message type (e.g. HTTP, MQ, SMTP, TCP, XML, SOAP, MIME, SWA, Text, ....), driven by an XML based DSL and standard Java libraries (JUnit, XMLUnit, Restlet, Greenmail, and a few others) and a bit of bridging code (to allow remote invocation). We pull everything together with Jenkins. Works pretty well. One of these days I'll get around to putting together something for WUG or similar. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Nov 09, 2012 12:13 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
One unit tests message broker artifacts by creating test harness message flows.
I would not trust any code that has been 'proven' to work using a framework that runs *outside* of Message Broker.
Why not just write an SQL interpreter so you can unit test your database stored procedures? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Nov 09, 2012 12:14 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| goffinf wrote: |
| Nice, thanks for contributing, this is one area of the unit test framework that I created that was missing. Normally I tell my Dev's to put the majority of JCN functionality in simple POJOs since that makes them easy to unit test with standard JUnit, but the actual MB API has always been a niggling omission |
No, this is a horrible idea that proves nothing. |
|
| Back to top |
|
|
| goffinf |
| Posted: Fri Nov 09, 2012 1:59 pm Post subject: |
|
|
Chevalier
Joined: 05 Nov 2005
Posts: 401
|
| mqjeff wrote: |
One unit tests message broker artifacts by creating test harness message flows.
I would not trust any code that has been 'proven' to work using a framework that runs *outside* of Message Broker.
Why not just write an SQL interpreter so you can unit test your database stored procedures? |
Actually the majority of the test framework *is* written in broker itself (most of it in the form of helper flows), so the only thing running outside is the means to invoke flows over transports (much like the tools that Broker provides, just more integrated into standard testing tools) and to provide mocks for external dependencies. So everything that is testing is running in Broker.
Have been thru all of this with your guys in Hursleyon-site and at your gaff several times. They haven't come up with anything better yet. |
|
| Back to top |
|
|
| goffinf |
| Posted: Fri Nov 09, 2012 2:05 pm Post subject: |
|
|
Chevalier
Joined: 05 Nov 2005
Posts: 401
|
| mqjeff wrote: |
No, this is a horrible idea that proves nothing.
|
Well naturally I disagree. If a JCN calls other classes and methods with standard data types I see no issue at all in testing those methods using JUnit.
The missing piece was methods that use Broker's own objects, now perhaps Akai has plugged that gap. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Fri Nov 09, 2012 2:06 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| goffinf wrote: |
| mqjeff wrote: |
One unit tests message broker artifacts by creating test harness message flows.
I would not trust any code that has been 'proven' to work using a framework that runs *outside* of Message Broker.
Why not just write an SQL interpreter so you can unit test your database stored procedures? |
Actually the majority of the test framework *is* written in broker itself (most of it in the form of helper flows), so the only thing running outside is the means to invoke flows over transports (much like the tools that Broker provides, just more integrated into standard testing tools) and to provide mocks for external dependencies. So everything that is testing is running in Broker.
Have been thru all of this with your guys in Hursleyon-site and at your gaff several times. They haven't come up with anything better yet. |
Yes, okay, the stuff *you* are doing.
The stuff mentioned here appears very solidly to be a test framework for running JavaCompute node code *outside* of Message Broker, which *fails* to prove anything. |
|
| Back to top |
|
|
| goffinf |
| Posted: Fri Nov 09, 2012 2:14 pm Post subject: |
|
|
Chevalier
Joined: 05 Nov 2005
Posts: 401
|
| mqjeff wrote: |
Yes, okay, the stuff *you* are doing.
The stuff mentioned here appears very solidly to be a test framework for running JavaCompute node code *outside* of Message Broker, which *fails* to prove anything. |
Ah, I see what you mean. I haven't looked in detail at the JCN testing code yet (just about to). I had assumed it was going to test code deployed in the run-time, not outside. Might still provide some pointers to how to do that, so I don't want to be too judgemental until I've had a closer look.
Regardless I applaud anyone who takes unit testing seriously I see far too much code that is untested and frankly untestable. |
|
| Back to top |
|
|
| AkaiRyujin |
| Posted: Mon Nov 12, 2012 2:29 am Post subject: |
|
|
Newbie
Joined: 06 Mar 2012
Posts: 7
|
The majority of unit testing is performed outside the normal runtime - this is normal see the large proliferation of mock / spy frameworks available for java to abstract code from heavy untestable implementations such as JNI based service layers. Whilst not wholly indicative and to some extent results should be taken with a pinch of salt it should allow the testing of the java code at the smallest fundamental unit to prevent issues reoccurring and allow continuous integration to be performed.
Also unit testing is just part of a suite of tools that a developer should deploy to ensure code is sufficiently tested; I would always recommend testing with the internal tools provided by IBM to test at the component / flow level as part of normal development process (mostly integration as opposed to unit testing).
I, also, would rather have less custom code, my pet hate being that I can't make use of the IBM xpath engine and had to reimplement its functionality, but the native (C based) nature of the product makes this unfeasible or at least at this moment in time.
On another note; I have already had interest from IBM developers regarding this so perhaps we will see an IBM implementation that is closer to the native implementation in the near future. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Nov 12, 2012 6:27 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
|
| Back to top |
|
|
| AkaiRyujin |
| Posted: Mon Nov 12, 2012 6:46 am Post subject: |
|
|
Newbie
Joined: 06 Mar 2012
Posts: 7
|
| Quote: |
| Why is Junit not acceptable for your purpose? |
You will note that I use JUnit too; infact this is a JUnit runner implementation.
For most code standard JUnit / Java functions fine however pure JUnit cannot test all interactions with the Java code, anything that interacts with the JNI wrappers (MbElement; MbMessage; MbMessageAssembly etc) cannot be tested by JUnit.
So a test for example to assert that propagate is called on an MbNode instance cannot be handled (it is not possible to mock the necessary MbMessageAssembly instance that the propagate method gets passed). In point of fact any code within the evaluate() method of the java compute node is untestable - at the unit level - yes you can turn your JCN into a subflow (IN > JCN > OUT) and then create a test flow to just test the functionality of the JCN and fire test messages at it. However this cannot be tested using CI and requires alot of boilerplate for every JCN that needs to be tested in this fashion.
This is compounded if you try and write your own custom nodes (non - java compute nodes). Where you *have* to interact with the JNI interfaces. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Nov 12, 2012 6:48 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
We have no difficulty testing 100% of our interactions, both happy and non-happy path, using Junit. We unit test all WMB code fully and inject various mid-flow data to simulate various conditions. I'm not sure why you are not able to do the same.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| AkaiRyujin |
| Posted: Mon Nov 12, 2012 6:58 am Post subject: |
|
|
Newbie
Joined: 06 Mar 2012
Posts: 7
|
So this function would be testable?
| Code: |
private void createEmailAddressProperty(final MbElement emailInputHeader, final String propertyName,
final String headerName, Address[] addresses) throws MbException {
if (addresses != null) {
emailInputHeader.createElementAsLastChild(MbElement.TYPE_NAME_VALUE, propertyName, Joiner.on(", ")
.join(addresses));
} else if (Trace.userTraceIsOn) {
Trace.logNamedUserTraceData(this, "buildEmailInputHeader", Personality.getInstance().messageCatalogueName(),
2147488619L, "4971", new String[] { this.name, headerName });
}
}
|
That is you could assert, using JUnit alone, that the generated MbElement structure is correct? |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Mon Nov 12, 2012 7:08 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
Our Junit approach is to blackbox test a flow. Data injections are done prior to invoking the flow. We don't intercept any mid-flow interactions; rather, we test the output based on the input and injected data.
It's cleaner way to use Junit with WMB. Trying to simulate the Mb structures and test only the Java code is tedious and copious.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Nov 12, 2012 8:05 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| lancelotlinc wrote: |
| Trying to simulate the Mb structures and test only the Java code is tedious and copious. |
More importantly, it invalidates the results of the test.
The only thing you learn is that your emulation framework produced the correct result, but without extensive and fundamental tests to show that the emulation framework produces exactly the same results in all cases, you've not proved a single thing about the JCN code. |
|
| Back to top |
|
|
|
|
