Performance

We evaluate performance using different approaches.

This comparison is very light and approximate. It’s just to give an idea of the cost to generate files and compare them.

We show below time to execute different types of tests. First, we execute a very simple test with JUnit, Approvals and using Git. With Git, we execute it with option no-assert or not. This option, is used when we launch tests with script to check failure only at the end.

We show what each test provide as output to compare a classical test to the approach based on the documentation. With Junit, we just display test source code.

We see a factor of 7 between JUnit and Git. In fact, there is a time taken to build JavaProjectBuilder to be able to parse code and retrieve comments. This initialization finished, tests are pretty fast with a factor of 3 with JUnit. With Approvals, preformance is less good (but may be optimized). This approach is clearly slower than using Junit, but difference is not really signifiant unless there is a lot a tests. We execute 1000 tests in less than 1 second.

Performances

We compare performance of tests using different approaches.

Using Git, we can verify documentation on each test or not. When we run all tests, this verification is done at the end by script that launch tests. To have total time using Git without verification on each test, we should add time of global checking. This step is fast, it’s just a shell script with some git commands.

Verify documentation with Approvals library.

Demo of a simple usage to generate documentation.

Should give person information

With an instance of Person:

First Name: John
Last Name: Doe
Year of birth: 1977

Results calling methods are:

age(): 45 (if we are in 2022)
toString(): John Doe

Check file modification with git after each test.

Demo of a simple usage to generate documentation.

Should give person information

With an instance of Person:

First Name: John
Last Name: Doe
Year of birth: 1977

Results calling methods are:

age(): 45 (if we are in 2022)
toString(): John Doe

Use Git to check file modification but after all tests. The time shown not include this verification.

Demo of a simple usage to generate documentation.

Should give person information

With an instance of Person:

First Name: John
Last Name: Doe
Year of birth: 1977

Results calling methods are:

age(): 45 (if we are in 2022)
toString(): John Doe

Using JUnit assertions to test.

@Test
public void should_give_person_information() {
    checkPerson();
}

public static void checkPerson() {
    final LocalDate now = LocalDate.now(Person.CLOCK);
    final int current_year = now.getYear();
    final int age = 45;
    final Person person = new Person("John", "Doe", LocalDate.of(current_year - age, 11, 23));

    assertEquals(age, person.age());
    assertEquals("John Doe", person.toString());
}