Reporting With ReportNG

ReportNG is a plug-In for TestNG, using this we can generate HTML based report for our test.Here we learn how to setup ReportNG and how to build using ANT script.

•  ReportNg SetUP

               Step 1:

                     Dowload  ReportNG from here .

               Step 2:

                      Extract the downloaded folder after that it looks like that

                      The reportng-1.1.4.jar and velocity-dep-1.4.jar.jar files must be included into our project lib folder.

                Step 3:

       Write the Ant script as given below

<project name="TestNGTest" default="test" basedir=".">
    <!-- Define <testng> task -->
    <property name="testdir" location="test" />
    <property name="libdir" location="lib" />
    <property name="full-compile" value="true" />
    <path id="classpath.test">
        <fileset dir="${libdir}">
            <include name="**/*.jar" />
        </fileset>
        <pathelement location="${testdir}\class" />
    </path>
    <taskdef resource="testngtasks" classpathref="classpath.test"/>
    <target name="reportng" depends="testng">
        <mkdir  dir="${testdir}\ReportNgReport" />
        <testng outputdir="${testdir}\ReportNgReport" classpathref="classpath.test" useDefaultListeners="false" 
             haltonfailure="false"   listeners="org.uncommons.reportng.HTMLReporter,org.uncommons.reportng.JUnitXMLReporter">
        <xmlfileset dir="${testdir}\javaapplication3" includes="NewTestNGSuite.xml"/>
        <sysproperty key="org.uncommons.reportng.escape-output" value="false" />
        <sysproperty key="org.uncommons.reportng.velocity-log" value="true"/>
        <sysproperty key="org.uncommons.reportng.title" value="My Test Report"/>   
        <sysproperty key="org.uncommons.reportng.frames" value="true"/>   
        <sysproperty key="org.uncommons.reportng.failures-only" value="true"/>   
        <sysproperty key="org.uncommons.reportng.show-expected-exceptions" value="false"/>

  
        </testng>
    </target> 
</project> 

                 If we useDefaultListeners="false" then TestNg reporting files should not be created.


                 We need to set listner for HTML reporting
                    org.uncommons.reportng.HTMLReporter
    
                 We need to set listner for JUnit reporting (optional)
                     org.uncommons.reportng.JUnitXMLReporter

                  We can also set the various system property of the ReportNg like rendering screenshot for test failure,change the default title, logging,only failure test shown on the html report etc......

•    Some Error handling in ReportNG SetUP

                    Problem 1:   Exception in thread "main" java.lang.NoClassDefFoundError: com/google/inject/Module


                    Solution:Download guice-3.0 from here ,extract the files and add guice-3.0.jar in our project lib.

                    Problem 2: Velocity Log not created
            
                    Solution : Actually  if we set this <sysproperty key="org.uncommons.reportng.velocity-log" value="true"/>then log is created in system environment directory.So we need to set the working directory from TestNg.We can do this using workingdir attribute in <testng> tag like:                                         


<testng workingdir="${basedir}/ReportNG" outputdir="${basedir}/ReportNG" classpathref="masterclasspath"
                useDefaultListeners="false" listeners="org.uncommons.reportng.HTMLReporter,org.uncommons.reportng.JUnitXMLReporter">

Some Error and Solution For TestNG-XSLT

On our previous post we learn how to configure and generate different type of repot using TestNG XSLT. Now here we discuss some configuration related problem and how can we overcome this.

Problem 1:

Error! Syntax error in 'if ($testNgXslt.testDetailsFilter) then $testNgXslt.testDetailsFilter else 'FAIL,PASS,SKIP''

Solution: This error occur because we forget to add SaxonLiaison jar into our project or we forget to add processor="SaxonLiaison" as an attribute on <xslt> tag. Proper tag is given below.

<xslt in="${basedir}/build/test/results/testng-results.xml" style="${basedir}/testng-results.xsl" out="${basedir}/testng-xslt/index.html" processor="SaxonLiaison">

Problem 2:

Fatal Error! Failed to create output file file://///style.css Cause: java.io.IOException: The filename, directory name, or volume label syntax is incorrect

Solution: This error happen if we for get to mentioned <param expression="${basedir}/testng-xslt/" name="testNgXslt.outputDir" /> this is mandatory and must be an absolute path.This is mandatory and must be an absolute path.
 

Different Reporting Style With TestNG-XSLT

Before this we learn how to configure TestNG from eclipcs(By default TestNG framework comes with NetBeans) and if we run our tests TestNG report generated into the default location. Here we learn how we make more user friendly reporting using TestNG-XSLT.

We follow this Steps: 
     
       Step 1:
       
             Download TestNG-XSLT from here and going to File -> Download option

       Step 2:
         
             Extract all files from zip folder, after the unzipping it looks like

      Step 3:

              Just copy the testng-results.xsl from the testng-xslt folder(\testng-xslt-1.1.2-master\src\main\resources) to our own project folder.

      Step 4:

              Now copy the saxon library from (testng-xslt-1.1.2-master\lib\saxon-8.7.jar) to our own project lib folder.

      Step 5:

            Now we are do the main thing that is modify build.xml of ant and add the following target to it.
   

<project name="test" basedir=".">
    <property name="LIB" value="${basedir}/lib" />
    <property name="BIN" value="${basedir}/bin" />
    <path id="master-classpath">
        <pathelement location="${BIN}" />
        <fileset dir="${LIB}">
            <include name="**/*.jar" />
        </fileset>
    </path>
    
    <target name="testng-xslt-report">
        <delete dir="${basedir}/testng-xslt">
        </delete>
        <mkdir dir="${basedir}/testng-xslt">
        </mkdir>
        <xslt in="${basedir}/build/test/results/testng-results.xml" style="${basedir}/testng-results.xsl" out="${basedir}/testng-xslt/index.html" processor="SaxonLiaison">
            <param expression="${basedir}/testng-xslt/" name="testNgXslt.outputDir" />

            <param expression="true" name="testNgXslt.sortTestCaseLinks" />

            <param expression="FAIL,SKIP,PASS,CONF,BY_CLASS" name="testNgXslt.testDetailsFilter" />

            <param expression="true" name="testNgXslt.showRuntimeTotals" />
      

            <classpath refid="master-classpath">
            </classpath>
        </xslt>
    </target>
</project> 

**Please change the path which is marked Red as per directory structure.

Note:

1. testNgXslt.outputDir - Sets the target output directory for the HTML content. This is mandatory and must be an absolute path. 

    e.g : <param expression="${basedir}/testng-xslt/" name="testNgXslt.outputDir" />

2. testNgXslt.showRuntimeTotals - Boolean flag indicating if the report should display the aggregated information about the methods durations. The information is displayed for each test case and aggregated for the whole suite. Non-mandatory parameter, defaults to false. 

    e.g : <param expression="true" name="testNgXslt.showRuntimeTotals" />

3. testNgXslt.testDetailsFilter - Specified the default settings for the checkbox filters at the top of the test details page. Can be any combination (comma-separated) of: FAIL,PASS,SKIP,CONF,BY_CLASS 

   e.g : <param expression="FAIL,SKIP,PASS,CONF,BY_CLASS" name="testNgXslt.testDetailsFilter" />

4. testNgXslt.sortTestCaseLinks - Indicates whether the test case links (buttons) in the left frame should be sorted alphabetically. By default they are rendered in the order they are generated by TestNG so you should set this to true to change this behavior

   e.g : <param expression="true" name="testNgXslt.sortTestCaseLinks" />


5. testNgXslt.reportTitle - Use this setting to specify a title for your HTML reports. This is not a mandatory parameter and defaults to "TestNG Results".

   e.g : <param expression="TestName" name="testNgXslt.reportTitle" />

 
We need to provide the testng-xslt stylesheet the testng-results.xml , the path to the style sheet testng-results.xsl and the output index.html ( Those are here <xslt in="${basedir}/build/test/results/testng-results.xml" style="${basedir}/testng-results.xsl" out="${basedir}/testng-xslt/index.html" processor="SaxonLiaison"> ) path.

We also add the saxon library to our target classpath else we will get an error. In our case it is the master-classpath.

Now we run the ant target for report generation (in my case "testng-xslt-report") and check the ouput folder configured by us for testng-xslt report.

 

XML Configuration For TestNG

Previously we know basic configuration of TestNG classes and various annotation and attributes.Here we learn how to configure a xml file for TestNG classes and it's various tags and attributes. Obviously some question in our mind

Q. Can I run test without using testng.xml file?
 Ans:- Well answer is yes, we can run. But once our test become larger and requires configuration of multiple test at one place, the best option is testng.xml.

Ok this is the time to know more about TestNG xml file.....

The simple testng.xml file will have following hierarchy

•        The root tag of this file is <suite>.
• 
  

  o   <suite> tag can contain one or more <test> tags.

  §  <test> tag can contain one or more <classes>

  ·         <classes> tag can contain one or more <class> tags.

  o   <class> tag can contain one <methods> tags.

  §  <methods> tags contain one or more<include> or <exclude>

  §   <test> tag can contain one <groups> tag

  ·         <groups> tag contain <define>

  o   <define>tag contain one or more<include> tag

  ·         <groups> tag contain <run> tag

  o   <run>tag contain one or more<include> or <exclude> tag

  o   <suite> tag can contain one or more <groups> tags.

  §  <groups> tag contain <define>

  ·         <define>tag contain one or more<include> tag

  §  <groups> tag contain <run> tag

  ·         <run>tag contain one or more<include> or <exclude> tag 

  § <groups> tag contain <dependencies>

  ·   <dependencies> contains <group>tag

If <groups> tag under <test> tag then it is available for this test onlt but if it is on <suite> tag then it is applicable for all test.

Some attribute description:

Increaseing Verbose value generate more log.


Parametrized Test using @Parameters

    @Parameters({"paraTest"})
    @Test
    public void runMethodGP1(String a) {
            System.out.println("Invoked testString " + a);
        System.out.println("runMethodGP1()");
    }

    <test name="TestNGAnotationClass2" time-out="1" preserve-order="true" annotations="JDK">
        <parameter name="paraTest" value="Test Parameter">
        </parameter>
        <classes>
            <class name="excelfilereadapachepoi.TestBase">
            </class>
            <class name="excelfilereadapachepoi.TestNGAnotationClass1">
            </class>
        </classes>
    </test>

Some TestNG Annotations with Attributes

In TestNg have so many attribute with the annotation we will try to learn some of them, here almost everything with example.......

enabled =Boolean value

If we set it false then method will not execute means it is not prepared for test or method not to be tested.
     
Example Code:
 
@Test(enabled = false)
    public void testMethod() {
        System.out.println("TestMethod()");
    }
@Test
    public void anotherTestMethod() {
        System.out.println("AnotherTestMethod()");
    }
OutPut:

 [TestNG] Running:
  Command line suite

AnotherTestMethod()From Group1

===============================================
Command line suite
Total tests run: 1, Failures: 0, Skips: 0
===============================================

 

timeOut  = Integer value

It is required if we set the value for time to executed if it required more time than defined then test failed elase passed.
Example Code:
    
@Test(timeOut = 1)
    public void testMethod() throws InterruptedException {
        Thread.sleep(2000);
        System.out.println("TestMethod()");
    }
@Test
    public void anotherTestMethod() {
        System.out.println("AnotherTestMethod()");
    }

OutPut:

[TestNG] Running:
  Command line suite

AnotherTestMethod()From Group1 and Group2

===============================================
Command line suite
Total tests run: 2, Failures: 1, Skips: 0
===============================================

Java Result: 1

 

groups ={"GroupName1","GroupName2",.......}

Groups attribute is used to grouping the methods.

Example Code:

@Test(groups = {"group2"})
    public void testMethod() {
        System.out.println("TestMethod() From Group1");
    }
@Test(groups = {"group1","group2"})
    public void anotherTestMethod() {
        System.out.println("AnotherTestMethod()From Group1 and Group2");
    }

OutPut:

[TestNG] Running:
  Command line suite

AnotherTestMethod()From Group1 and Group2
TestMethod() From Group1

===============================================
Command line suite
Total tests run: 2, Failures: 0, Skips: 0
===============================================

 dependsOnGroups={"GroupName1","GroupName2" ,.....}

It's used to imply the dependency of a another group(groups). If all the methods of this group(groups) is not passed then it is skipped.

Example Code:

    /*
     * This @Test have timeout limit 1 Sec
     * but it required 2 Sec becausing using of this
     * Thread.sleep(2000);
     * So it will be failed due to timeOut
     */

    @Test(groups = {"group1"}, timeOut = 1)
    public void testMethod() throws InterruptedException {
        Thread.sleep(2000);
        System.out.println("TestMethod() From Group1");
    }
   
 /*
     * This @Test dependes on "group1"
     * and "group1" failed due to timed-out
     * So This test should be Skipped though
     * "group2" Passed.
     */

    @Test(dependsOnGroups = {"group1", "group2"})
    public void anotherTestMethod() {
        System.out.println("AnotherTestMethod()From Group1 and Group2");
    }

    @Test(groups = {"group2"})
    public void testMethodFromGP2() {
        System.out.println("TestMethod() From Group2");
    }

OutPut:

[TestNG] Running:
  Command line suite

TestMethod() From Group2

===============================================
Command line suite
Total tests run: 3, Failures: 1, Skips: 1
===============================================

Java Result: 3

 

dependsOnMethods = {"methodName1","methodName2",........}

It's used to imply the dependency of a another method(methods). If all the method/methods is not passed then it is skipped.

Example Code:
 
     /*
     * This @Test have timeOut limit 1 Sec
     * but it required 2 Sec becausing using of this
     * Thread.sleep(2000);
     * So it will be failed due to timeOut
     */
    @Test(timeOut = 1)
    public void testMethod() throws InterruptedException {
        Thread.sleep(2000);
        System.out.println("TestMethod() From Group1");
    }
    /*
     * This @Test dependes on "testMethod"
     * and "testMethod" failed due to timed-out
     * So This test should be Skipped though
     * "testMethodFromGP2" Passed.
     */

    @Test(dependsOnMethods = {"testMethod", "testMethodFromGP2"})
    public void anotherTestMethod() {
        System.out.println("AnotherTestMethod()From Group1 and Group2");
    }

    @Test
    public void testMethodFromGP2() {
        System.out.println("TestMethod() From Group2");
    }

OutPut:

[TestNG] Running:
  Command line suite

TestMethod() From Group2

===============================================
Command line suite
Total tests run: 3, Failures: 1, Skips: 1
===============================================

Java Result: 3 

priority =Integer value

It is used to maintain the sequence means which method we want to execute first we set low value (Set 0).
One thing keep in mind if a method depends on a lower priority method or depends on a group, that may hold lower priority methods then this method or methods should be run first then higher priority method.

Example Code:   

    /*
     * This Method should be executed
     * brfore anotherTestMethod()though
     * testMethod()have lower priority
     * than it but anotherTestMethod()
     * depends on testMethod()
     */

    @Test(priority = 2)
    public void testMethod() throws InterruptedException {
        Thread.sleep(2000);
        System.out.println("TestMethod() From Group1");
    }
    
    /*
     * Though it have the Higher Priority
     * than "testMethod()" but it dependes
     * on this lower priority method it
     * should be executed after "testMethod()"
     */

    @Test(dependsOnMethods = {"testMethod"},priority = 1)
    public void anotherTestMethod() {
        System.out.println("AnotherTestMethod()From Group1 and Group2");
    }

  /*
   * This TestMethod will execute First
   * Because it have the Highest Priority
   */
 
    @Test(priority = 0)
    public void testMethodFromGP2() {
        System.out.println("TestMethod() From Group2");
    }


OutPut:

[TestNG] Running:
  Command line suite

TestMethod() From Group2
TestMethod() From Group1
AnotherTestMethod()From Group1 and Group2

===============================================
Command line suite
Total tests run: 3, Failures: 0, Skips: 0
===============================================

alwaysRun = Boolean value

If it is set then it should be run in any condition it does not depends on any group or methods.

Example Code:
    /*
     * This @Test have timeout limit 1 Sec
     * but it required 2 Sec becausing using of this
     * Thread.sleep(2000);
     * So it will be failed due to timeOut
     */

    @Test(timeOut = 1)
    public void testMethod() throws InterruptedException {
        Thread.sleep(2000);
        System.out.println("TestMethod() From Group1");
    }
   
    /*
     * This @Test dependes on "testMethod"
     * and "testMethod" failed due to timed-out
     * So This test should be Skipped but we
     * set alwaysRun = true so it should be executed
     */

    @Test(dependsOnMethods = {"testMethod"},alwaysRun = true)
    public void anotherTestMethod() {
        System.out.println("AnotherTestMethod()From Group1 and Group2");
    }


OutPut:

[TestNG] Running:
  Command line suite

AnotherTestMethod()From Group1 and Group2

===============================================
Command line suite
Total tests run: 2, Failures: 1, Skips: 0
===============================================

Java Result: 1

Different TestNG Annotation and Functionality

Every Testing framework have some annotation to run each and every unit test.TestNG is inspired by JUnit so it's annotation have some similarity with JUnit and also have some extra feature by which TestNG is more robust and can overcome the difficulties of JUnit.

Here we can start the learn some annotation of TestNG and later we know some TestNG xml configuration by which we can run multiple test with the one configuration file.

Let's start to know some Annotation of TestNG

Annotation	Description
@BeforeSuite	The annotated method will be run only once before all tests in this suite have run.
@AfterSuite	The annotated method will be run only once after all tests in this suite have run.
@BeforeClass	The annotated method will be run only once before the first test method in the current class is invoked.
@AfterClass	The annotated method will be run only once after all the test methods in the current class have been run.
@BeforeTest	The annotated method will be run before any test method belonging to the classes inside the @Test tag is run.
@AfterTest	The annotated method will be run after all the test methods belonging to the classes inside the @Test tag have run.
@BeforeGroups	The list of groups that this configuration method will run before. This method is guaranteed to run shortly before the first test method that belongs to any of these groups is invoked.
@AfterGroups	The list of groups that this configuration method will run after. This method is guaranteed to run shortly after the last test method that belongs to any of these groups is invoked.
@BeforeMethod	The annotated method will be run before each test method.
@AfterMethod	The annotated method will be run after each test method.
@DataProvider	Marks a method as supplying data for a test method. The annotated method must return an Object[ ][ ] where each Object[ ] can be assigned the parameter list of the test method. The @Test method that wants to receive data from this DataProvider needs to use a dataProvider name equals to the name of this annotation.
@Factory	Marks a method as a factory that returns objects that will be used by TestNG as Test classes. The method must return Object[ ].
@Listeners	Defines listeners on a test class.
@Parameters	Describes how to pass parameters to a @Test method.
@Test	Marks a class or a method as part of the test.

In below a sample code example is given

/*
 * To change this license header, choose License Headers in Project Properties.
 * To change this template file, choose Tools | Templates
 * and open the template in the editor.
 */
package excelfilereadapachepoi;

import org.testng.annotations.AfterClass;
import org.testng.annotations.AfterGroups;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.AfterSuite;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.BeforeGroups;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

/**
 *
 * @author Administrator
 */
public class TestNGAnotationClass1 {

    @BeforeSuite
    public void beforeSuite() {
        System.out.println("!.......BeforeSuite From Class TestAnnotation.......!");
    }

    @BeforeClass
    public void beforeClass() {
        System.out.println("!.......BeforeClass From Class TestAnnotation.......!");
    }

    @BeforeTest
    public void beforeTest() {
        System.out.println("!.......BeforeTest From Class TestAnnotation.......!");
    }

    @BeforeMethod
    public void beforeMethod() {
        System.out.println("!.......BeforeMethod From Class TestAnnotation.......!");
    }

    @DataProvider(name = "dp")
    public static Object[][] dataprovider() {
        return new Object[][]{{1, "Test 1"}, {1, "Test 2"}};
    }

    @Test(dataProvider = "dp")
    public void dataConsumer(int i, String j) {
        System.out.println("This INTEGER data " + i + " Consumed and " + " This STRING data " + j + " Consumed");
    }

    @AfterSuite
    public void afterSuite() {
        System.out.println("!.......AfterSuite From Class TestAnnotation.......!");
    }

    @AfterClass
    public void afterClass() {
        System.out.println("!.......AfterClass From Class TestAnnotation.......!");
    }

    @AfterMethod
    public void afterMethod() {
        System.out.println("!.......AfterMethod From Class TestAnnotation.......!");
    }

    @AfterTest
    public void afterTest() {
        System.out.println("!.......AfterTest From Class TestAnnotation.......!");
    }

    @BeforeGroups(groups = {"Group-One", "Group-Two"})
    public void beforeGroup() {
        System.out.println("!.......BeforeGroups From Class TestAnnotation.......!");
    }

    @Test(groups = "Group-One")
    public void gpOneMethodOne() {
        System.out.println("This is gpOneMethodOne() from Group-Two");
    }

    @Test(groups = "Group-One")
    public void gpOneMethodSecond() {
        System.out.println("This is gpOneMethodSecond() from Group-One");
    }

    @Test(groups = "Group-Two")
    public void gpSecondMethodOne() {
        System.out.println("This is gpSecondMethodOne() from Group-Two");
    }

    @Test(groups = "Group-Two")
    public void gpSecondMethodSecond() {
        System.out.println("This is gpSecondMethodSecond() from Group-Two");
    }

    @AfterGroups
    public void afterGroups() {
        System.out.println("!.......AfterGroups From Class TestAnnotation.......!");
    }

}

OUTPUT

[TestNG] Running:
  Command line suite

!.......BeforeSuite From Class TestAnnotation.......!
!.......BeforeTest From Class TestAnnotation.......!
!.......BeforeClass From Class TestAnnotation.......!
!.......BeforeMethod From Class TestAnnotation.......!
This INTEGER data 1 Consumed and  This STRING data Test 1 Consumed
!.......AfterMethod From Class TestAnnotation.......!
!.......BeforeMethod From Class TestAnnotation.......!
This INTEGER data 1 Consumed and  This STRING data Test 2 Consumed
!.......AfterMethod From Class TestAnnotation.......!
!.......BeforeGroups From Class TestAnnotation.......!
!.......BeforeMethod From Class TestAnnotation.......!
This is gpOneMethodOne() from Group-Two
!.......AfterMethod From Class TestAnnotation.......!
!.......BeforeMethod From Class TestAnnotation.......!
This is gpOneMethodSecond() from Group-One
!.......AfterMethod From Class TestAnnotation.......!
!.......BeforeGroups From Class TestAnnotation.......!
!.......BeforeMethod From Class TestAnnotation.......!
This is gpSecondMethodOne() from Group-Two
!.......AfterMethod From Class TestAnnotation.......!
!.......BeforeMethod From Class TestAnnotation.......!
This is gpSecondMethodSecond() from Group-Two
!.......AfterMethod From Class TestAnnotation.......!
!.......AfterClass From Class TestAnnotation.......!
!.......AfterTest From Class TestAnnotation.......!
!.......AfterSuite From Class TestAnnotation.......!

===============================================
Command line suite
Total tests run: 6, Failures: 0, Skips: 0
===============================================
 

TestNG SetUp With Eclips

TestNG(Test Next Generation) is nothing but a unit testing framework but using it we can do functional testing, integration and many other of testing can be done.We later know how to use this in Selenium WebDriver .TestNG is come with NetBeans so here we know how to add this in Eclips.

There is two way

1. Add TestNG jar to the particular project.
2. Add TestNG as a plugin.

Keep in mind that there is only difference is that if we add TestNg as a plugin it is applicable for all project and if we add it as a jar then it is only available for the particular project.

Step For Installation:

Step1:

• Launch Eclipse (Here we use Kepler).
• On the menu bar, click Help.
• Choose the "Install New Software..." option.

Step 2:

•  In the Install dialog box, click the Add button.
• In "Name", type TestNG.
• In "Location", type http://beust.com/eclipse .
• Click OK
  

 Step3.

• Notice that "TestNG - http://beust.com/eclipse" was populated onto the "Work with:" textbox.
• Check the "TestNG" check box as shown below, then click Next.
• Select the check box and follow the instruction like accept license and some click on the next button.

                                

 Or we can escape the above steps and just go to the Help->Eclipse Marketplace
             

      and give the name TestNG as a product.

 

Step 4:

• After that need to restart the IDE.
• Go to the Run As option and TestNG should be there.
  
  
  

 Step 5:

• Create a project project from File ->New->Other->TestNG->TestNGclass.

           

• Create class for testing

  

• Click finish.

Step 6:

• Right click on file from project pane(Right hand side of the IDE).
• RunAs ->TestNG Test.

• Result should be shown like that.