When I became a researcher, my day-to-day programming moved from Java to scientific computing with Python. On this transition, a notable change was the character of the documentation. Java APIs were typically presented as structured systems, organized around packages and classes that could be browsed with relative ease. In contrast, much of the Python documentation I encountered, including the standard library and popular projects such as NumPy and SciPy, appeared primarily as a sequence of authored pages like a manual.
Python’s documentation is extensive and of excellent quality. However, the prevailing interaction model favors search and sequential reading over systematic exploration. I repeatedly observed Java developers describing Python as “poorly documented,” and I believe that what they actually miss is structural visibility. Looking outside of the Java ecosystem made this issue evident and raised a broader question:
What kind of artifact is API reference documentation meant to be?
Motivation
API reference documentation and manuals serve different purposes, yet modern web publishing increasingly blurs the distinction. This is a category mistake.
Manual-style documentation is excellent for tutorials, onboarding and conceptual explanations. It assumes a reading order and guides users along a predefined path. API reference documentation, by contrast, describes a system. It answers questions such as:
- What does this API expose?
- How is it structured?
- Which elements belong together?
- Where does a given type fit into the overall design?
These questions are not well served by linear text. They require a representation that mirrors the structure of the codebase itself: packages, modules, namespaces, classes, and their relationships. Historically, Java supported this distinction well. Classic Javadoc kept structure continuously visible. Comparable approaches existed and still exist elsewhere: the MSDN Library, Unix manual hierarchies and Android’s customized Javadocs. These systems present API documentation as a space to explore.
Over time, API documentation was adapted to page-centric web publishing. In Java, removing frames did not eliminate structural elements such as package pages, overview pages, breadcrumbs, and indices, but it made the structure less present. Navigation gradually shifted from exploration towards lookup. In Python, linear documentation is the norm. Even core libraries are difficult to approach without knowing what to search for. Learners in particular struggle to discover what exists.
The javadoc.io platform improves Javadoc accessibility by making it directly web-browsable. However, as a mirror of Maven Central, it republishes whatever each project provides. Docs generated decades apart appear side by side – sometimes with frames, sometimes not, sometimes with very old CSS.
APIdia’s landing page.
APIdia (apidia.net) is based on the premise that structural API documentation should be treated as a first-class artifact that mirrors the organization of the codebase. Modern web technology should reinforce and refine this concept. Visually, APIdia follows established Javadoc conventions, while addressing long-standing painpoints and enabling consistent navigation across project boundaries. The aim is to restore API reference documentation to its proper role: a structured, browsable representation of a software system that complements tutorials and manuals for understanding, learning and teaching.
Assessing the Broader Picture
When I started working on APIdia, the first step was a technology survey to evaluate possible foundations. Doxygen, the popular multi-language API doc generator, was the most obvious candidate. It supports numerous languages including Java, Python and C++, can cross-link multiple libraries, and produces HTML, PDF and XML output. With Doxygen often being considered a fallback alternative to Javadoc, the findings that follow may be surprising for some.
Such a survey can hardly be complete without looking at Sphinx, the standard documentation tool in the Python ecosystem. Beyond Python, Sphinx is used by numerous established projects like e.g. LLVM. While Java is the starting point for APIdia, Python is an important future target language, for which Sphinx is foundational.
Other tools such as Asciidoctor, Dokka, or Scaladoc are certainly relevant but beyond the scope of this particular article.
Doxygen
Doxygen is a mature tool for structured API documentation. In my experiments, I tested Doxygen on an extensive amount of Java code, e.g. from the standard library. All problems were reported to the issue tracker and often discussed with the developers. Many smaller issues were addressed quickly, which speaks for the project’s responsiveness. However, three crucial limitations remain unsolved:
- Annotations, central to modern Java, are often ignored: #7335, #9118.
- Nested generics confuse the parser and result in incomplete documentation: #8495.
- Malformed HTML inside doc comments can cause the parser to overrun doc comment boundaries and leak subsequent source code into the output: #8618.
To some extent, these are symptoms of the C++-oriented core design but effectively render Doxygen insufficient for modern Java. Nonetheless, Doxygen remains a promising foundation for future C++ support in APIdia via its structured XML output.
Doxygen sample HTML with prematurely terminated generic declarations and missing @NotNull method annotations.
In Doxygen HTML output, the navigation uses a collapsible tree view, which is in principle a good choice for presenting the code hierarchy. However, expanding or collapsing nodes shifts the visible area and often requires scrolling to regain orientation. Also horizontal scrolling becomes common. Expanded and collapsed nodes make the GUI highly stateful, which introduces complexity and raises practical questions: Should the state be stored in browser memory, or reset on each visit?
Sphinx
In Sphinx, the main organizing concept is the document rather than the code structure. Authors compose reStructuredText or Markdown files and define a table of contents to link them together.
The autodoc extension can integrate API elements from source code seamlessly into documents like tutorials and manuals. In more recent Sphinx releases, the apidoc extension can automatically compose a table of contents for all API elements in the codebase. This automation has not been widely adopted yet and most projects continue to integrate API documentation directly into their manuals via autodoc. A notable disadvantage is that Sphinx does not generate summary sections like Javadoc. This can make orientation in large modules difficult.
Layouts and visual identity vary widely across projects due to Sphinx’s templating flexibility. In all built-in themes, the navigation scrolls together with the main content, which is not always ideal for usability. The only way to uncouple the scrolling is manual CSS adjustment or using third-party themes like PyData or ReadTheDocs.
Sample documentation using the ReadTheDocs theme, generated via Sphinx extensions apidoc and autodoc.
Using Sphinx for Java was once possible through the now unmaintained javasphinx extension. However, it required employing a markup language supported by Sphinx in the doc comments. Sphinx’s mature text-processing ecosystem, particularly the libraries docutils and Napoleon, will prove useful for future Python support in APIdia.
Implications for APIdia
The evaluation of Doxygen and Sphinx led to concrete design decisions:
- avoid collapsible tree navigation
- independent scrolling of navigation and content, never horizontal
- minimize interaction state so that it can be represented in URLs
- use Java’s own parsing infrastructure
(modulesjava.compiler,jdk.compiler,jdk.javadoc)
Architectural Principles of APIdia
APIdia’s visual resemblance of classic Javadoc is the result of systematic evaluation rather than mere nostalgia (although that does play its part).
Illustration of APIdia’s interface (constructor summary of java.lang.Integer)
A central challenge is defining navigational units that remain valid across languages. Java provides modules, packages and classes; Python inserts modules between packages and classes; C++ is organized around headers and namespaces. Instead of privileging any specific abstraction, APIdia adopts the physical structure of the source code as guiding organization principle.
Files and folders offer a stable, language-agnostic model. Developers naturally maintain balanced directory hierarchies and manageable file sizes. The conclusion is not to build a file browser, but to gain a balanced organization structure by distinguishing between units of folder character and file character. This yields three conceptual levels: folders, files and content.
In Java, modules and packages have folder character, while top-level classes function as file-level entities and inner classes appear within their enclosing types. In Python, packages map to folders and modules to files. Although mappings differ, the principle remains consistent.
Stability Is a Priority
The three levels – folders, files, content – naturally correspond to the three-frame structure familiar from classic Javadoc. APIdia’s interface reflects this through three persistent regions. The upper left lists folders, including a project-level “Overview.” The lower left shows the content of the selected folder in a stable list rather than a collapsible tree. Folder-like entries are marked with arrows: the arrow enters the folder, while the name opens the content. This distinction resolves the dual nature of constructs such as Java packages (package-info.java), Java modules (module-info.java), or Python packages (__init__.py).
APIdia’s layout in three regions.
The main region displays the content of exactly one file-level unit at a time. A lightweight in-page navigation on the right provides orientation without altering global state.
Avoiding accumulated interaction state is a recurring design constraint. Just like collapsible trees, expandable detail views would contribute state tedious to handle. APIdia therefore retains Javadoc’s summary/detail pattern but introduces one powerful refinement: Each detail section links back to its summary entry. This simple addition provides the practical convenience of collapsible views without hidden state.
Visibility and Filtering
Documentation is always generated with complete structural coverage: All types and members are included. Visibility is controlled at the presentation level. Private or internal elements remain in the document, hidden by default via CSS. A configurable filter allows adjustment along three dimensions:
- access level (
publictoprivate) - module exposure (
exported,opened, internal) - module selection
Filtering by access level, exposure and module selection.
This introduces limited state, but the state reflects an explicit perspective on the API surface rather than incidental navigation history. It remains managable precisely because state was carefully avoided elsewhere. Conventional Javadoc typically generates separate artifacts for public and internal views using flags such as private. APIdia models these as configurable views on a single structural representation, now useful for library maintainers and users alike.
Maven Artifacts and Modules
Java’s module system provides an organization structure parallel to Maven artifacts. To resolve this, APIdia consistently represents Maven artifacts as modules. If a declared Java module exists inside an artifact, it is used; otherwise, the artifact becomes an automatic module and is labeled accordingly. This keeps navigation uniform without introducing Maven as a separate abstraction.
In practice, libraries are often composed of multiple artifacts and presenting them separately would lead to fragmentation. Since Maven provides no universally reliable mechanism for grouping artifacts into coherent project-level units, heuristics are applied: common versioning within a group identifier, BOM files, or parent artifacts. There are some projects, such as OSGi, in which versioning and grouping conventions are not exposed on Maven level. These cases require manual curation.
Example jackson-databind: All Jackson artifacts are combined to a coherent project profile.
As a result, artifact aggregation remains one of the most demanding aspects of APIdia, often further complicated by the presence of incomplete or shadowed dependencies. It also marks another distinction from javadoc.io, which mirrors artifacts individually. Where javadoc.io presents e.g. jackson-databind as a standalone unit, APIdia integrates it into the broader Jackson project context, maintaining a consistent look and navigational model across all related modules.
Cross-Library Linking
The ambition that all references to external types resolve into links requires the presence of a project’s dependency tree. This is challenging because these trees can be very large for certain projects. Mirroring the complete Maven Central repository is thereby neither feasible nor desirable. In addition to the sheer volume, it contains obsolete versions, experimental projects and abandoned forks.
APIdia therefore operates selectively and incrementally. Starting from foundational libraries and widely used frameworks like Jakarta EE, Spring, Apache Commons and many more, dependency trees are expanded gradually to cover a canonical subset of the ecosystem. Users can request additional projects to be included as new roots for this system.
Conclusion
API documentation is frequently approached as a form of technical writing. More precisely, it represents the exposed structure of a software system. When this structural role is obscured, navigation becomes dependent on search and prior knowledge, making systematic exploration difficult.
The comparison of Javadoc, Doxygen and Sphinx shows how tooling shapes documentation culture. Some ecosystems emphasize authored narratives; others structural extraction. Javadoc historically supported structural browsing, but web conventions have shifted the focus towards individual pages and search.
APIdia is a platform made for hierarchical API documentation. It combines established structural principles and applies them consistently across dependency trees. Cross-library linking enables a coherent representation of an entire technical ecosystem.
This article is part of the JAVAPRO magazine issue:
From Coder To System Designer
Understand what it means to move from coding to designing systems in the age of AI.
Take a closer look at modern Java platforms, architectural thinking, and the responsibilities that come with shaping complex software systems.
Discover the edition →
