Merging Extent Spark Reports

July 2, 2021 26 Comments

Introduction

ExtentReports library is used for creating detailed and interactive test reports. The browser based Spark report is the most common report format. This article discusses the technique of combining Spark reports. The merging is accomplished by using the JSON ExtentReport as an intermediary.

Two or more JSON reports can be combined to generate a consolidated JSON and Spark report. The merging logic can be fine tuned by using various options. Combining rerun reports is a specialized case where the merging logic is restricted to failed scenarios being replaced by available passed scenarios.

There are two methods of execution, from the command line and from inside java class.

Source

The source code can be found at this location. Sample implementations for combining reports can be found here.

The entry method for the command line execution is the default main method in the Combiner class which accepts an array containing the options from the ArrayOptionsInputType class. Similarly the entry method for the java class execution is the main method which accepts a PojoOptions object as a parameter.

Dependency

The following dependency needs to be added in the POM for use in a Maven project.

<dependency>
	<groupId>tech.grasshopper</groupId>
	<artifactId>extentreports-merge</artifactId>
	<version>1.1</version>
</dependency>

In the case of command line execution, use the jar from this location. The jar contains all the required dependent classes.

Extent JSON Report

The ExtentReport JSON report is the most important component, which is used as the basis for merging. This report can be generated from the cucumber 6 adapter and cucumber 5 adapter by setting the extent.reporter.json.start option to true and assigning an output folder value to the extent.reporter.json.out option in extent.properties. The cucumber extent plugin generates the report by default.

Usage

The artifact can be executed by using various options. The rerun reports merging which is a specific case is also mentioned here.

Mandatory Option

The simplest method of execution requires the list of JSON report paths and the final merged report path. The former is specified from the command line by using the -jsonReportPaths option and in java code the jsonReportPaths() builder method of the PojoOptions class is used. The later option uses the -mergedReportDirPath and mergedReportDirPath() builder method for command line and java code respectively.

Below is the code to execute from the command line, when the JSON reports are available in the combine folder and the merged JSON and Spark reports are to be generated in the combine/reports folder. (sample)

java -cp extentreports-merge-cli.jar tech.grasshopper.combiner.Combiner -jsonReportPaths "combine/first/ExtentJsonOne.json, combine/second/ExtentJsonTwo.json" -mergedReportDirPath combine/reports

Similarly this can be accomplished in a java class by using the below code. (sample)

List<String> jsonReportPaths = Arrays.asList(new String[] { "combine/first/ExtentJsonOne.json", combine/second/ExtentJsonTwo.json" };

PojoOptions pojoOption = PojoOptions.builder().jsonReportPaths(jsonReportPaths).mergedReportDirPath("combine/reports").build();

Combiner.main(pojoOption);
Media Option

When medias such as images are present in the original JSON reports the media location paths need to be provided in the options. The later option uses the -mediaPaths and mediaPaths() builder method for command line and java code respectively. If this option is not present then the value of the jsonReportPaths is used by default. This can also handle base64 strings for images.

The index of particular report location in the jsonReportPaths value and the mediaPaths value should match. If there are reports without any medias than a blank string needs to be provided in the correct location in the array.

Code for command line. (sample)

java -cp extentreports-merge-cli.jar tech.grasshopper.combiner.Combiner -jsonReportPaths "combine/first/ExtentJsonOne.json, combine/second/ExtentJsonTwo.json" -mediaPaths "combine/first/media, combine/second/media" -mergedReportDirPath combine/reports

Java code for pojoOption in addition to the mandatory options. (sample1 sample2)

List<String> mediaPaths = Arrays.asList(new String[] { "combine/first/media", combine/second/media" };

PojoOptions pojoOption = PojoOptions.builder().jsonReportPaths(jsonReportPaths).mergedReportDirPath("combine/reports").mediaPaths(mediaPaths).build();
Extent Configuration Option

The extent configuration settings can be provided in a XML file named extent-config.xml and needs to be placed inside the final report folder. If the format is in JSON then use a file named extent-config.json. The option value needs to be either xml or json. For the command line use the -configType option and for java code use the configType() builder method of the PojoOptions class. (sample1 sample2)

Rerun Report Options

The only extra configuration that needs to be set for merging a st of rerun reports is the reportType, to the value rerun. For the command line use the -reportType option and for java code use the reportType() builder method of the PojoOptions class. This is in addition to the options mentioned above.

The jsonReportPaths option values needs to be in the order of execution. This means that the first value should point to the JSON report initially generated. Then the second value should point to the JSON report generated from the rerun report of the first runner. For the third value, the report is generated from the second rerun report and so on.

Code for command line. (sample)

java -cp extentreports-merge-cli.jar tech.grasshopper.combiner.Combiner -reportType rerun -jsonReportPaths "combine/first/ExtentJsonOne.json, combine/second/ExtentJsonTwo.json" -mediaPaths "combine/first/media, combine/second/media" -mergedReportDirPath combine/reports -configType xml

Java code for pojoOption. (sample)

PojoOptions.builder().reportType("rerun").jsonReportPaths(jsonReportPaths).mergedReportDirPath("combine/reports").mediaPaths(mediaPaths).configType("xml").build();
Matching Scenario Option

When there are same named scenario, inside the same named feature or scenario outline structure in two reports, then by default the scenario with the later start time (name later) is selected. The other options are earlier start time (value earlier), lower status (name lowstatus) and high status (name highstatus) selection. This can be changed by using the -matchingStrategy option from the command line and for java use the matchingScenarioTestStrategy() builder method of the PojoOptions class. The various options are available in this class. The higher and lower statuses are calculated from the Status enum in ExtentReports code.

This option is not available for rerun report type for which the matching criterion is fixed, the failed scenario is replaced by the passed scenario if available.

Start And End Time Display

The start time displayed on the Spark report dashboard is the earliest start time of all the features in the merged report. Similarly, the end time is equal to the latest end times of the features. This is NOT a true representation of the duration of the test run in most cases and should not be relied upon.

Advanced Usage

There are four hooks in the merging logic which can be modified by using defined values or adding a custom class. One of these hooks, with values is defined in the section above – Matching Scenario Option. (sample)

  1. ExtraScenarioTestStrategy – This strategy decides how an extra feature, scenario outline and scenario from the secondary report which is not present in the merging JSON report is managed. By default these are added to the merged report (name add). In the case of rerun report type an exception is thrown when there is an extra feature, scenario outline and scenario (name error). A custom class, with the full package name, which implements ExtraScenarioTestStrategy can also be assigned.
  2. MatchingScenarioTestStrategy – In addition to the explanation in the section above a custom class, with the full package name, which implements MatchingScenarioTestStrategy can also be assigned. In the case of rerun report type only failed scenarios matched by same named passed scenarios are replaced (name passstatus).
  3. PrimaryJsonReportCheckStrategy – This is used to execute a check on the first JSON report provided in the -jsonReportPaths option or jsonReportPaths() builder method. By default there is an empty check. In the case of rerun report type an exception is thrown when no failed scenarios are present (name rerun). A custom class, with the full package name, which implements PrimaryJsonReportCheckStrategy can also be assigned.
  4. SecondaryJsonReportCheckStrategy – This is used to execute a check on the second and further JSON reports provided in the -jsonReportPaths option or jsonReportPaths() builder method. By default there is an empty check. In the case of rerun report type the report processing is skipped if no passed scenarios are present (name rerun). A custom class, with the full package name, which implements SecondaryJsonReportCheckStrategy can also be assigned.

Options Overview

OptionCmd OptionCmd ValueBuilder MethodBuilder ValueRemarks
Report Type-reportTypeText. Default is ‘combine’, ‘rerun’.reportType()String. Default is “combine”, “rerun”
JSON Report Paths-jsonReport
Paths		Comma delimited text.jsonReportPaths()List<String>Details of report folder path and name.
Media Paths-mediaPathsComma delimited text.mediaPaths()List<String>Details of media folders for report. Default is value of jsonReportPaths.
Merged Report Folder-mergedReport
DirPath		Text.mergedReportDirPath()StringPath to folder of the merged JSON and Spark report.
Configuration Report Type-configTypeText. ‘xml’ or ‘json’configType()String. “xml” or “json”Format of ‘extent-config’ located in jsonReportPath folder.
Extra Scenario Strategy-extraStrategyText. Default is ‘add’, ‘error’. Custom class.extraScenario-TestStrategy()String. Default is “add”, “error”. Custom class. Class needs to implement ExtraScenarioTestStrategy. Full class name with package.
Matching Scenario Strategy-matching
Strategy		Text. Default is ‘later’, ‘earlier’, ‘highstatus, ‘lowstatus’. Custom class.matchingScenario-TestStrategy()String. Default is ‘later’, ‘earlier’, ‘highstatus, ‘lowstatus’. Custom class. Class needs to implement MatchingScenarioTestStrategy. Full class name with package.
Primary JSON Report Check Strategy-primaryJson
Strategy		Text. Default is no check, ‘rerun’. Custom class.primaryJsonReport-
CheckStrategy()		Text. Default is no check, ‘rerun’. Custom class. Class needs to implement PrimaryJsonReportCheckStrategy. Full class name with package.
Secondary JSON Report Check Strategy -secondaryJson
Strategy		 Text. Default is no check, ‘rerun’. Custom class. secondaryJsonReport-CheckStrategy() Text. Default is no check, ‘rerun’. Custom class. Class needs to implement SecondaryJsonReportCheckStrategy. Full class name with package.

26 thoughts on “Merging Extent Spark Reports”

  1. Hi,Bro, I had some doubts please clarify.
    1.In merging is it possible make it in pdf format
    2.Is it possible customise merged spark.html?If possible and how to do it
    3.After create merging report .my cucumber report not generated.

    Reply

    1. 1.In merging is it possible make it in pdf format – This is currently not possible.
      2.Is it possible customise merged spark.html?If possible and how to do it – What kind of customization? Most probably not.
      3.After create merging report .my cucumber report not generated. – This will generate the merged extent json report and not the cucumber one

      Reply

  2. Exception in thread “main” tech.grasshopper.combiner.exception.CombinerException: Unable to copy media files.
    at tech.grasshopper.combiner.media.MediaHandler.lambda$copyReportMedias$3(MediaHandler.java:50)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
    at tech.grasshopper.combiner.media.MediaHandler.copyReportMedias(MediaHandler.java:44)
    at tech.grasshopper.combiner.report.JSONReportsProcessor.manageSecondaryMediaLogs(JSONReportsProcessor.java:106)
    at tech.grasshopper.combiner.report.JSONReportsProcessor.processMatchingScenarioTest(JSONReportsProcessor.java:79)
    at tech.grasshopper.combiner.report.JSONReportsProcessor.processScenario(JSONReportsProcessor.java:66)
    at tech.grasshopper.combiner.report.JSONReportsProcessor.processScenarioOutline(JSONReportsProcessor.java:51)
    at tech.grasshopper.combiner.report.JSONReportsProcessor.processFeature(JSONReportsProcessor.java:32)
    at tech.grasshopper.combiner.report.BaseReportType.mergeReportTests(BaseReportType.java:84)
    at tech.grasshopper.combiner.report.BaseReportType.combineTests(BaseReportType.java:70)
    at tech.grasshopper.combiner.report.BaseReportType.generateReport(BaseReportType.java:49)
    at tech.grasshopper.combiner.Combiner.process(Combiner.java:19)
    at tech.grasshopper.combiner.Combiner.main(Combiner.java:11)
    at mergeExtentReport.CombineReport.main(CombineReport.java:18)
    Caused by: java.nio.file.NoSuchFileException: C:\Users\987993\Merchant_Automation\reports\All Report 10-Jul-23 16-58-28 PM\test-output\JsonReport\embedded1.png
    at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
    at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
    at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
    at java.base/sun.nio.fs.WindowsFileCopy.copy(WindowsFileCopy.java:98)
    at java.base/sun.nio.fs.WindowsFileSystemProvider.copy(WindowsFileSystemProvider.java:283)
    at java.base/java.nio.file.Files.copy(Files.java:1295)
    at tech.grasshopper.combiner.media.MediaHandler.lambda$copyReportMedias$3(MediaHandler.java:47)
    … 13 more

    Process finished with exit code 1

    While merging json file it shows. could help me to get over this

    Reply

  3. Hi, I am trying to use this library with the Cucumber 7 JVM report generation, but I haven’t had any luck merging two different JSONs. I have tried following with multipleReportsCombine(), it doesn’t seem to merge no matter how many jsons I list in jsonReportPaths. Currently I am calling this in a Cucumber Test Runner class with a @AfterClass annotation, and it is running after all tests are finished but I am still only getting one test case instead of several. Is there something I am missing?

    Reply

    1. Hi, I will only be able to check this out tomorrow.
      Can you try to merge from outside the test execution process? From the command line or a separate java program?
      Do you get any exception stacktrace?

      Reply

      1. I do not get any exception stack traces which is the weird thing. I will try doing this through a separate java program and get back to you.

        Reply

          1. Yes. I think I know what my problem is, I’m going to give it a try and see if it works. But something else I was wondering, if I wanted to keep a log of old tests how would I do that with Cucumber 7 JVM Report generation? I noticed in your samples that there is only 1 log of the same test when merged.

            Reply

              1. Sorry, I worded that poorly. I understand the Merging Extent Reports library does not support that. So, I was wondering how could I use the Cucumber 7 JVM plugin to append older Extent Reports into a newer one.

              2. Simplest is to keep them separate. You will also need the cucumber-json files of the earlier executions.

                You can create a simple Java program which contains the merging logic and run it with the exec-maven-plugin.
                This can be added to any phase after the test is run. Or it could even work in the same phase if listed, in the POM, after the test execution (Not sure though)

  4. Hi. I am currently trying to use this library with Cucumber-JVM 7 Report generation using ExtentReports Adapter plugin, but for some reason I am unable to get it to work. I am currently trying to merge the jsons after all the tests have been done with @AfterClass in the Cucumber Test runner class, but it never merges with the other jsons that I specify. Is that not what I should be doing? Here is the method that is inside the test runner.

    @AfterClass
    public static void combineReport(){
    List paths = Arrays.asList(“test-output/JsonReport/Json.json”,
    “test-output/JsonReport/JsonTwo.json”,
    “test-output/SparkReport/json.json”,
    “test-output/JsonReport/JsonThree.json”);
    PojoOptions options = PojoOptions.builder()
    .jsonReportPaths(paths)
    .mergedReportDirPath(“test-output/SparkReport”).configType(“xml”).build();
    CombinerOptions generatedoptions = PojoOptionsInputType.builder().options(options).build()
    .generateOptions();

    generatedoptions.getReportType().generateReport();
    }

    Thank you for your time.

    Reply

  5. Hi,
    With this plugin the system environment variables I set in the extent.properties are not displayed in the combined report. Is there a way to toggle it on?

    Reply

      1. I tried to also use the extentreports-cucumberjson-plugin and that also seems not to display the system information. Can support be added for it?
        Thanks

        Reply

  6. Hi Mounish,
    I tried this way.
    1st:-
    This is my runner class where mentioned rerun.txt file :
    @CucumberOptions(plugin = { “pretty”, “com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter:”,
    “timeline:test-output-thread/”,”json:target/json-report/cucumber.json”,”rerun:target/rerun.txt”}, monochrome = true, glue = {
    “stepDefinition” }, tags = “@smoketest”, features = {
    “src/test/resources/featureFiles” })
    In second rerun file mentioned like:
    public class Rerunner {

    @CucumberOptions(plugin = { “pretty”, “com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter:”,
    “timeline:test-output-thread/”,”json:target1/json-report/cucumber.json”}, monochrome = true, glue = {
    “stepDefinition” }, features = {“@target/rerun.txt” })

    public class ParallelRunner extends AbstractTestNGCucumberTests {
    @Override
    @DataProvider(parallel = true)
    public Object[][] scenarios() {
    return super.scenarios();
    }
    }

    }
    And from testNG.xml I am running both runner file. But both result is not getting merged.
    Could you please suggest me what needs to do.

    Reply

    1. This is what I understand. After the testng.xml has finished execution, you have one report with the failed results and another with the passed rerun results. Is this correct?
      If so then you can simply follow the instructions in the ‘Rerun Report Options’ section of this article.

      Reply

      1. Hi Mounish,
        Two JSON report generated under target\json-report folder i.e. cucumber.json and cucumberrerun.json . Is this the report abt you are asking or any other report you are looking for.(Note- I am looking for cucumber Extent PDF report after merge of two JSON file) Next my query is I followed “Rerun Report Options”
        but not able to understand one thing for report Type should I create another runner file and will mention like
        below or how is it? Could you please guide me. It will be really helpful.

        @CucumberOptions(plugin = { “pretty”,”com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter:”,
        “timeline:test-output-thread/”,”-reportType rerun -jsonReportPaths “combine/first/ExtentJsonOne.json, combine/second/ExtentJsonTwo.json”}, monochrome = true, glue = {
        “stepDefinition” }, features = {“@target/rerun.txt” })

        Reply

        1. This merging utility jar only supports Spark html report and not the pdf report (I missed this in your initial comment). I will have a look at this over the weekend to extend this to merging pdf reports.
          Anyways you are using it wrong. Go over the article and just look up the samples.

          Reply

  7. Hi,
    In my framework I was trying to rerun the failed scenario. For that I was using IRetryAnalyzer and IAnnotationTransformer. It was retrying the failed cases. But failed cases which was retrying it was not updated in extent pdf report which is auto generated. Still it is showing failed in extent pdf report. But in real it is passed after retry. Could you please help me how it will update in auto generate extent pdf report,

    Thanks,
    Subha

    Reply

    1. Are you getting a failed child test and then the retried passed test under a parent Extent Test in the report?
      The report will not manage the case of failed retried test case as this will lead to data hiding. This will need to be handled in the code by removing the failed Extent test and then add the retried test details.

      Reply

  8. Hi, I am currently using tech.grasshopper:extentreports-cucumber6-adapter to generate the report, in which i’ve a problem with reporting.
    I’ve tried two ways,
    Way 1-
    My testng.xml is set up with two class, regular and rerun class.
    Both class point to one cucumber runner, 1st with plugin option to generate “rerun:target/rerun txt” and cucumber.json and second runner picks up the failed test and generates cucumberRerun.json, which is fine.
    Way 2-
    Using testng retry analyzer, able to run failed cucumber tests.

    But I’m getting single report with result from both runs(duplicates tests). I’m looking for merging the reports.
    (Take the latest run result, if there is a failure).
    TIA

    Reply

    1. Hi, If u run the two classes which configure the runner separately, then you should get 2 reports. These can be merged with the rerun strategy. The adapter in a single execution will only generate one report as there is only ExtentReports instance.

      With the current setup of single report, what u can try is to parse the final json report using gson etc and then removing the original test failure node or test. Then write the new json report to a file. This can then be used to create Spark or any other report.

      Let me know if u need further help. thx

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *