Living Documentation
This project show some techniques to extract documentation from your project like:
-
Extract code documentation (javadoc)
-
Use annotation
-
Extract xml information
-
Execute code to find values
-
Formating documentation
-
Generate graph
It’s also present use cases like:
-
Create a glossary
-
Retrieve good example from code
-
Visualize workflow
-
Document tests
-
Visualize dependencies
-
Create a release note
Getting started
Prerequisites
-
JDK 11
-
Maven
-
Docker
This project is auto-documented. To generate full documentation, just call
. ./generateDoc.sh
Documentation is generated in ./target/docs folder.
A generated documentation is already available here: https://sfauvel.github.io/livingdocumentation
Library dependencies
In these demos, we use libraries:
-
com.github.javaparser: Parse java code.
-
com.thoughtworks.qdox: Extract javadoc.
-
io.github.livingdocumentation.dotdiagram: Generate graphviz diagrams.
-
javax.xml.parsers: Read xml files.
-
org.eclipse.jgit: Execute git commands.
-
org.reflections: Find classes, annotated classes and methods.
The graph below shows which libraries is used in demos.

Available demos
List of demo classes available in this project.
Each demo is a simple program that extract some documentation from the code. It illustrates a use case or a technique. It contains a 'main' to make it a standalone application to be able to see what is generated and to execute it independently. We try to not use utilities classes to keep all generation code into a single class to have all information necessary to reproduce example
These demonstrations are minimalist. They just show what is possible to do but may not worked on more generic cases.
Annotation
Annotated method demo
From: org.dojo.livingdoc.demo.FunctionnalityDoc
Display method with annotation that contains attribute.
We need to create an annotation (here Functionnality) that be used on each method to document. This annotation contains attributs to specify additional information.
@Retention(RetentionPolicy.RUNTIME)
public @interface Functionnality {
String name();
}
@Functionnality(name = "Living Documentation")
public void functionnalityToDocument() {
// ...
}
public String generateFunctionnalities() {
Set<Method> annotatedMethod = getAnnotatedMethod();
return annotatedMethod.stream()
.map(m -> formatDoc(m))
.collect(joining("\n\n"));
}
/// Retrieve methods with a specific annotation.
private Set<Method> getAnnotatedMethod() {
String packageToScan = "org.dojo.livingdoc";
Reflections reflections = new Reflections(packageToScan, new MethodAnnotationsScanner());
return reflections.getMethodsAnnotatedWith(Functionnality.class);
}
/// Extract information from annotation parameters (here the attribute name).
private String formatDoc(Method method) {
return String.format("*%s* (%s): %s",
method.getName(),
method.getDeclaringClass().getSimpleName(),
method.getDeclaredAnnotation(Functionnality.class).name());
}
findAnnotatedMethod (ClassToDocument): Find all method with a specific annotation.
functionnalityToDocument (FunctionnalityDoc): Living Documentation
Glossary demo
From: org.dojo.livingdoc.demo.GlossaryDoc
Display annotated classes.
Retrieve all classes annotated (annotation Glossary) to be included into glossary.
public GlossaryDoc() {
builder = new JavaProjectBuilder();
builder.addSourceTree(new File("src/main/java"));
}
public String generateGlossary() {
return new Reflections("org.dojo.livingdoc")
.getTypesAnnotatedWith(Glossary.class, false)
.stream()
.map(this::formatGlossary)
.collect(joining("\n"));
}
/// Format class to generate glossary information.
private String formatGlossary(Class<?> classToDocument) {
return classToDocument.getSimpleName() + "::" + getDescription(classToDocument);
}
private String getDescription(Class<?> classToDocument) {
JavaClass javaClass = builder.getClassByName(classToDocument.getCanonicalName());
return " " + Optional.ofNullable(javaClass.getComment()).orElse("");
}
- Person
-
A physical person.
- City
-
Description of a city.
Change log
Extract changelog from git messages
From: org.dojo.livingdoc.demo.GitLogMessage
Extract commit messages from Git.
It can be used to generate release note.
public String generateGitMessages() throws IOException, GitAPIException {
Repository repository = new FileRepositoryBuilder()
.setGitDir(gitPath.toFile())
.setMustExist(true)
.build();
Iterable<RevCommit> logs = new Git(repository).log().call();
return StreamSupport.stream(logs.spliterator(), false)
.limit(10)
.map(rev -> formatMessage(rev, "dd/MM/yyyy"))
.collect(Collectors.joining("\n"));
}
public String formatMessage(RevCommit rev, String dateFormat) {
Date authorDate = rev.getAuthorIdent().getWhen();
String dateFormatted = new SimpleDateFormat(dateFormat).format(authorDate);
return String.format("* *%s* (%s): %s",
dateFormatted,
rev.getName().substring(0, 10),
rev.getShortMessage());
}
-
13/04/2020 (8e3c5574f6): Add information on library used in demos
-
13/04/2020 (e83e6796ec): Add a demo using dot-diagram
-
25/03/2020 (78e4d74f51): Fix asciidoctor version, use Rouge instead of pygments
-
25/03/2020 (3ef7e83c7b): Remove useless dependency
-
06/01/2020 (edd8fb0d0a): Add documentation of classes with a main method.
-
05/01/2020 (57326a7b2e): Add documentation of classes with a main method.
-
23/11/2019 (ce23a9df10): Publish only index.html in docs folder
-
23/11/2019 (98419f3890): Rename index.html to demo-full.html
-
23/11/2019 (9e7a2c8327): Rename demo-full.html to index.html
-
22/11/2019 (03a604aa18): Modify output folder for generation
Include a changelog file
From: org.dojo.livingdoc.demo.Changelog
A simple way to make a change log is to have a changelog file with an asciidoctor file.
It needs to be strict to update file on each changes. But, if merge request is used in development process, it could be verify it was updated before accepting request. It also easier to update a change log file than rewrite git history when there is something to correct.
To find some information on how to write a change log: https://keepachangelog.com
= Change log
## [1.0.1] - 2019-10-05 == Changed - Improve change log example. - Reorganized project.
## [1.0.0] - 2019-08-27 == Added - Add a change log example. - Add a workflow graph demo.
== Changed - Improve commit message demo.
Example of changelog file when included
Change log
[1.0.1] - 2019-10-05
Changed
-
Improve change log example.
-
Reorganized project.
[1.0.0] - 2019-08-27
Added
-
Add a change log example.
-
Add a workflow graph demo.
Changed
-
Improve commit message demo.
public String generatePomDescription() throws IOException {
Files.copy(Paths.get("CHANGELOG.adoc"),
Paths.get("./target/doc/CHANGELOG.adoc"),
StandardCopyOption.REPLACE_EXISTING);
return "include::CHANGELOG.adoc[]";
}
Execute to get information
Execute maven command
From: org.dojo.livingdoc.demo.MavenDoc
Execute Maven command to find dependencies informations.
Here, we execute mvn dependency:list to retrieve dependencies of each module in project. Then, we draw a graph with these dependencies.
public String generateDendencies()
throws IOException, SAXException {
Path PROJECT_PATH = Path.of("src", "main", "resources", "project");
List<String> modules = getModules(PROJECT_PATH);
String dependenciesGraph = modules.stream()
.flatMap(module -> findDependencies(PROJECT_PATH.resolve(module)).stream()
.map(dependency -> String.format("\"%s\" -> \"%s\"", module, dependency))
)
.collect(Collectors.joining("\n"));
return String.join("\n",
"",
"[graphviz]",
"----",
"digraph g {",
dependenciesGraph,
"}",
"----");
}
private List<String> findDependencies(Path path) {
List<String> recorder = executeCommand(path, List.of("mvn", "dependency:list"));
return recorder.stream()
.map(s -> s.replaceFirst("^\\[INFO\\]", ""))
.map(String::trim)
.filter(isADependencyLine())
.map(this::extractArtifact)
.collect(Collectors.toList());
}
private List<String> getModules(Path path) {
return Arrays.stream(path.toFile().listFiles(java.io.File::isDirectory))
.filter(f -> Paths.get(f.getPath(), "pom.xml").toFile().exists())
.map(java.io.File::getName)
.collect(Collectors.toList());
}

Get information executing code.
From: org.dojo.livingdoc.demo.ExecuteDoc
Execute some code to retrieve information.
Sometimes, it’s not possible or too difficult to find information directly from the code. It could be easier to execute the code to get information.
In this demonstration, we are creating a configuration object to get default values.
An object instance is created and all getters are called using reflexion. Values returned are defaults values returned by the object.
public String generateDoc(Object instance) {
return String.format("Default values of %s class\n\n", instance.getClass().getSimpleName())
+ String.format("[options=\"header\"]\n|===\n|Field|Default value\n%s\n|===\n",
Arrays.stream(Configuration.class.getDeclaredMethods())
.filter(this::isGetter)
.map(m -> formatRow(instance, m))
.collect(Collectors.joining("\n")));
}
private String formatRow(Object instance, Method method) {
try {
return String.format("|%s|%s", method.getName(), method.invoke(instance));
} catch (IllegalAccessException | InvocationTargetException e) {
return "Value could not be retrieve";
}
}
private boolean isGetter(Method m) {
return (m.getName().startsWith("get") || m.getName().startsWith("is"))
&& Modifier.isPublic(m.getModifiers());
}
Default values of Configuration class
Field | Default value |
---|---|
getVersion |
5.2 |
isVerbose |
false |
Show methods called
From: org.dojo.livingdoc.demo.CallFlowDoc
Display contributors calls.
We execute a method and trace every calls to injected services.
public String generateCallFlow() throws Error {
final TraceAnswer recordCalls = new TraceAnswer();
final Notifier notifier = spyWithTracer(new NotifierImpl(), recordCalls);
final Dao dao = spyWithTracer(new DaoImpl(notifier), recordCalls);
// Call method to trace
final Service service = new Service(dao, notifier);
service.findHomonyms(5);
return String.join("\n",
"",
".Calls from Service.findHomonyms method",
"[plantuml]",
"----",
recordCalls.linksPlantUml.stream()
.collect(Collectors.joining("\n")),
"----");
}
/**
* Create a spy over a object to trace every methods called.
* @param instance
* @param recordCalls
* @param <T>
* @return
*/
private <T> T spyWithTracer(T instance, TraceAnswer recordCalls) {
return Mockito.mock((Class<T>) instance.getClass(),
Mockito.withSettings()
.spiedInstance(instance)
.defaultAnswer(recordCalls));
}

Show workflow from code
From: org.dojo.livingdoc.demo.WorkflowDoc
Show algorithm workflow.
Wokflow configuration is defined in code. We extract information to display a graph.
We use graphviz to draw the graph.
public String generateWorkflowGraph() throws Error {
final Workflow workflow = new Workflow();
return String.join("\n",
"",
"[graphviz]",
"----",
"digraph g {",
Arrays.stream(Workflow.State.values())
.flatMap(state -> formatStateLinks(workflow, state))
.collect(Collectors.joining("\n")),
"",
"}",
"----");
}
private Stream<String> formatStateLinks(Workflow workflow, Workflow.State currentState) {
return workflow.availableTransition(currentState).stream()
.map(availableState -> currentState + " -> " + availableState);
}
/**
* Class in application that defined workflow.
*/
class Workflow {
public enum State {
OPEN, RESOLVED, IN_PROGRESS, CLOSED, REOPENED;
}
public List<State> availableTransition(State state) {
switch (state) {
case OPEN:
return Arrays.asList(RESOLVED, IN_PROGRESS, CLOSED);
case RESOLVED:
return Arrays.asList(REOPENED, CLOSED);
case CLOSED:
return Arrays.asList();
case IN_PROGRESS:
return Arrays.asList(OPEN, RESOLVED);
case REOPENED:
return Arrays.asList(CLOSED, IN_PROGRESS);
default:
return Arrays.asList();
}
}
}

Show workflow using dot-diagram
From: org.dojo.livingdoc.demo.WorkflowDocWithDotDiagram
Show algorithm workflow.
Wokflow configuration is defined in code. We extract information to display a graph.
We use graphviz to draw the graph and dot-diagram to generate dot text.
public String generateWorkflowGraph() throws Error {
final Workflow workflow = new Workflow();
final DotGraph graph = new DotGraph("");
final DotGraph.Digraph digraph = graph.getDigraph();
Arrays.asList(Workflow.State.values()).forEach(state -> addStateTransitions(digraph, workflow, state));
return String.join("\n",
"",
"[graphviz]",
"----",
graph.render(),
"----");
}
private void addStateTransitions(DotGraph.Digraph digraph, Workflow workflow, Workflow.State currentState) {
digraph.addNode(currentState.name()).setLabel(currentState.name());
workflow.availableTransition(currentState)
.forEach(availableState -> digraph.addAssociation(currentState.name(), availableState.name()));
}

Extract javadoc
JavaDoc with JavaParser
From: org.dojo.livingdoc.demo.DescriptionWithJavaParserDoc
Get description from javadoc comment using JavaParser.
It’s a simple example retrieve javadoc from class and methods.
public String generateDoc() {
Class<?> classToDocument = ClassToDocument.class;
// Parse class source code.
SourceRoot sourceRoot = new SourceRoot(Paths.get("src/main/java"));
CompilationUnit cu = sourceRoot.parse(
classToDocument.getPackage().getName(),
classToDocument.getSimpleName() + ".java");
// Visit code tree to retrieve javadoc.
JavadocVisitorAdapter javadocVisitor = new JavadocVisitorAdapter();
cu.accept(javadocVisitor, null);
// Format result to create documentation.
return String.join("\n",
formatClass(javadocVisitor.javaDocOfClasses),
"",
javadocVisitor.javaDocOfMethods.stream()
.map(this::formatMethod)
.collect(Collectors.joining()));
}
/// Visitor to store class and methods javadoc.
public static class JavadocVisitorAdapter extends GenericVisitorAdapter<Object, Void> {
JavaDocOfElement javaDocOfClasses;
List<JavaDocOfElement> javaDocOfMethods = new ArrayList<>();
@Override
public Object visit(ClassOrInterfaceDeclaration declaration, Void arg) {
String className = declaration.getFullyQualifiedName().orElse(null);
javaDocOfClasses =
new JavaDocOfElement(className, declaration.getComment());
return super.visit(declaration, arg);
}
@Override
public Object visit(MethodDeclaration declaration, Void arg) {
javaDocOfMethods.add(
new JavaDocOfElement(declaration.getNameAsString(), declaration.getComment())
);
return super.visit(declaration, arg);
}
}
org.dojo.livingdoc.application.ClassToDocument: Class to show a javadoc extraction.
-
main: Starting point of the application.
-
simpleMethod: Simple method documented.
-
findAnnotatedMethod: No description.
JavaDoc with QDox
From: org.dojo.livingdoc.demo.DescriptionWithQDoxDoc
Get description from javadoc comment with QDox.
public String generateDoc() {
JavaProjectBuilder builder = new JavaProjectBuilder();
builder.addSourceTree(new File("src/main/java"));
JavaClass javaClass = builder.getClassByName(classToDocument.getCanonicalName());
return String.format("%s: \n%s\n\n%s",
javaClass.getName(),
javaClass.getComment(),
methodList(javaClass));
}
private static String methodList(JavaClass javaClass) {
return javaClass.getMethods().stream()
.map(javaMethod -> String.format("- %s: %s",
javaMethod.getName(),
javaMethod.getComment()))
.collect(Collectors.joining("\n"));
}
ClassToDocument: Class to show a javadoc extraction.
-
main: Starting point of the application.
-
simpleMethod: Simple method documented.
-
findAnnotatedMethod: null
Reflexion
Classes with main method
From: org.dojo.livingdoc.demo.FindMainDoc
Display classes with a main method.
Retrieve all main methods in project using Reflections library. We search all classes in given package and retain only those who have a main method. Result is display in a list.
public FindMainDoc() {
builder = new JavaProjectBuilder();
builder.addSourceTree(new File("src/main/java"));
}
public String generate() {
return new Reflections("org.dojo.livingdoc", new SubTypesScanner(false))
.getSubTypesOf(Object.class).stream()
.filter(this::isContainMainMethod)
.map(o -> o.getSimpleName())
.collect(joining("\n* ", "* ", ""));
}
private boolean isContainMainMethod(Class<?> aClass) {
return Arrays.stream(aClass.getDeclaredMethods())
.anyMatch(m -> Modifier.isStatic(m.getModifiers())
&& m.getName().equals("main")
);
}
-
PomDoc
-
FunctionnalityDoc
-
DemoDocumentation
-
FindMainDoc
-
ReferenceToCodeDoc
-
FormulaDoc
-
GitLogMessage
-
WorkflowDoc
-
MavenDoc
-
DescriptionWithQDoxDoc
-
CallFlowDoc
-
ClassToDocument
-
DescriptionWithJavaParserDoc
-
GlossaryDoc
-
AsciidoctorGeneration
-
WorkflowDocWithDotDiagram
-
ParseDoc
-
ExecuteDoc
Static analysis
Document formula with stem
From: org.dojo.livingdoc.demo.FormulaDoc
Display formula like \$sum_(i=1)^n i^3\$ using default formula syntax asciimath.
It needs to add :stem:
option in document (see https://asciidoctor.org/docs/user-manual/#activating-stem-support)
/**
* stem:[(1.55^"level") + sqrt(12*"level") + 50]
*/
public double xpNeedsToNextLevel(int level) {
return Math.pow(1.55, level) + Math.sqrt(12*level) + 50;
}
We can extract formula from Javadoc. It’s easy (see generateFormulaFromJavaDoc method) but it’s not the real formula used in code.
Another way of doing is to parse code and format it using asciimath syntax. Below, we show a naive implementation used to parse example.
public static String fromJava(String javaFormula) {
return new FormulaAsciiMath().parse(javaFormula);
}
private String parse(String javaFormula) {
final Provider codeProvider = Providers.provider("class X { String formula = " + javaFormula + " }");
ParseResult<CompilationUnit> result = (new JavaParser()).parse(ParseStart.COMPILATION_UNIT, codeProvider);
if (!result.isSuccessful()) {
throw new RuntimeException(result.getProblems().stream()
.map(problem -> problem.getVerboseMessage())
.collect(Collectors.joining("\n")));
}
final CompilationUnit compilationUnit = result.getResult().get();
final FormulaVisitor formulaVisitor = new FormulaVisitor();
Stack<String> stack = new Stack<>();
compilationUnit.accept(formulaVisitor, stack);
return stack.peek();
}
public static class FormulaVisitor extends VoidVisitorAdapter<Stack<String>> {
@Override
public void visit(IntegerLiteralExpr n, Stack<String> arg) {
super.visit(n, arg);
arg.push(n.getValue());
}
@Override
public void visit(DoubleLiteralExpr n, Stack<String> arg) {
super.visit(n, arg);
arg.push(n.getValue());
}
@Override
public void visit(BinaryExpr n, Stack<String> arg) {
n.getLeft().accept(this, arg);
String left = arg.pop();
n.getRight().accept(this, arg);
String right = arg.pop();
arg.push(left + n.getOperator().asString() + right);
}
@Override
public void visit(NameExpr n, Stack<String> arg) {
super.visit(n, arg);
arg.push("\"" + n.getNameAsString() + "\"");
}
@Override
public void visit(MethodCallExpr n, Stack<String> arg) {
for (Expression expression : n.getArguments()) {
expression.accept(this, arg);
}
final String mathFunction = n.getName().asString();
if ("pow".equals(mathFunction)) {
String exponent = arg.pop();
String base = arg.pop();
arg.push("(" + base + "^" + exponent + ")");
} else if ("sqrt".equals(mathFunction)) {
String value = arg.pop();
arg.push(mathFunction + "(" + value + ")");
} else {
throw new RuntimeException("Function '"+mathFunction+"' not supported");
}
}
}
public FormulaDoc() {
builder = new JavaProjectBuilder();
builder.addSourceTree(new File("src/main/java"));
}
public String generateFormulaFromJavaDoc() {
JavaProjectBuilder builder = new JavaProjectBuilder();
builder.addSourceTree(new File("src/main/java"));
JavaClass javaClass = builder.getClassByName(SpecificRule.class.getName());
final List<JavaMethod> methods = javaClass.getMethods();
return methods.stream()
.map(method -> method.getName() + ": " + method.getComment())
.collect(Collectors.joining("\n"));
}
public String generateFormulaParsingCode() {
final Class classWithFormula = SpecificRule.class;
final String methodWithFormula = "xpNeedsToNextLevel";
String javaCode = extractMethodBody(classWithFormula, methodWithFormula);
String formulaCode = javaCode.replaceAll("^\\{\\s*return (.*)\\s*\\}$", "$1");
return methodWithFormula + ": stem:[" + FormulaAsciiMath.fromJava(formulaCode) + "]";
}
private String extractMethodBody(Class classWithFormula, String methodWithFormula) {
SourceRoot sourceRoot = new SourceRoot(Paths.get("src/main/java"));
CompilationUnit cu = sourceRoot.parse(
classWithFormula.getPackage().getName(),
classWithFormula.getSimpleName() + ".java");
StringBuffer javaCode = new StringBuffer();
cu.accept(new VoidVisitorAdapter<StringBuffer>() {
@Override
public void visit(MethodDeclaration n, StringBuffer arg) {
if (methodWithFormula.equals(n.getNameAsString())) {
final String str = n.getBody()
.map(body -> body.toString())
.orElse("");
System.out.println("BODY:" + str);
javaCode.append(str);
}
}
}, null);
return javaCode.toString();
}
xpNeedsToNextLevel: \$(1.55^"level") + sqrt(12*"level") + 50\$
xpNeedsToNextLevel: \$(1.55^"level")+sqrt(12*"level")+50\$
Extract a code fragment
From: org.dojo.livingdoc.demo.ReferenceToCodeDoc
Extract a code fragment to include in documentation.
To identify code to include into documentation, it have to be surrounded by tag::[TAG] and end::[TAG].
// tag::InterestingCode[]
public void doNothing() {
// Really interesting code.
}
// end::InterestingCode[]
public String includeCodeToDoc() {
return String.join("\n",
"[source,java,indent=0]",
".Best practice to follow",
"----",
"include::{sourcedir}/org/dojo/livingdoc/application/TechnicalStuff.java[tags=InterestingCode]",
"----");
}
public void doNothing() {
// Really interesting code.
}
Extract imports parsing code
From: org.dojo.livingdoc.demo.ParseDoc
Parse code to extract informations.
We can retrieve import, conditions, attributes, …
JavaParser: https://github.com/javaparser/javaparser
public String execute() throws Error {
final Reflections reflections = new Reflections("org.dojo.livingdoc");
Set<Class<?>> typesAnnotatedWith =
reflections.getTypesAnnotatedWith(ClassDemo.class, false);
SourceRoot sourceRoot = new SourceRoot(Paths.get("src/main/java"));
return getImports(typesAnnotatedWith, sourceRoot)
.limit(3)
.collect(Collectors.joining("\n", "", "\n* ..."));
}
private Stream<String> getImports(Set<Class<?>> typesAnnotatedWith, SourceRoot sourceRoot) {
return typesAnnotatedWith.stream().map(aClass -> {
CompilationUnit cu = sourceRoot.parse(
aClass.getPackage().getName(),
aClass.getSimpleName() + ".java");
List<String> imports = new ArrayList();
cu.accept(new RecordImportsVisitor(), imports);
return (String.format("* %s\n%s",
aClass.getSimpleName(),
imports.stream()
.distinct()
.filter(importName -> !importName.startsWith("java"))
.map(s -> "** " + s)
.collect(Collectors.joining("\n"))));
});
}
/// Visitor that record imports
class RecordImportsVisitor extends GenericVisitorAdapter<Object, List<String>> {
@Override
public Object visit(ImportDeclaration declaration, List<String> imports) {
imports.add(extractPackageFromImport(declaration, imports));
return super.visit(declaration, imports);
}
}
-
GlossaryDoc
-
com.thoughtworks.qdox
-
com.thoughtworks.qdox.model
-
org.dojo.livingdoc.annotation
-
org.reflections
-
-
WorkflowDoc
-
org.dojo.livingdoc.annotation
-
org.dojo.livingdoc.demo.Workflow.State
-
-
MavenDoc
-
org.dojo.livingdoc.annotation
-
org.xml.sax
-
-
…
Extract information from pom.xml
From: org.dojo.livingdoc.demo.PomDoc
Extract information from pom.xml (or a xml file).
You may have some information stored in a XML file like the project description into the pom.xml.
In this demo, we parse the file and display the content of the 'description' tag.
public String generatePomDescription()
throws ParserConfigurationException, IOException, SAXException {
Element root = parsePom().getDocumentElement();
return root.getElementsByTagName("description").item(0).getTextContent();
}
private static Document parsePom()
throws ParserConfigurationException, SAXException, IOException {
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
return builder.parse(new File("pom.xml"));
}
Demo of living documentation