DOM API: Modifying the HTML Document Before Conversion

With the v42 layout engine, PD4ML gives Java code access to the HTML document it has parsed: the DOM API. Between readHTML() and the output (writePDF(), writeRTF(), writeDOCX(), renderAsImages()), the document can be read, queried with CSS selectors and changed, and the output is laid out as if the HTML had been written that way. A second part of the API, layout(), lays the document out into pages without writing it and tells where each element ended up, so code can check a layout and adjust the document before the real output.

No external library is involved: the API is a thin layer over PD4ML's own document tree, in the package com.pd4ml.dom.

Typical uses

• Filling a template. Put values into an HTML template by id or selector instead of building the HTML as a string.
• Cleaning up third-party HTML. Remove navigation, cookie banners or tracking pixels; add missing alt texts; fix links.
• Adding content. Insert a cover note, a generated table, or a style sheet with @page rules from code.
• Checking the layout. Find out how many pages a document takes, whether a block is split across pages, or where an element is on its page, and react before writing.

Getting the document

import com.pd4ml.Constants;
import com.pd4ml.PD4ML;
import com.pd4ml.dom.Document;
import com.pd4ml.dom.Element;

PD4ML pd4ml = new PD4ML();
pd4ml.useHtmlRenderer(Constants.LayoutEngine.v42);
pd4ml.readHTML(new URL("https://example.com/invoice.html"));

Document doc = pd4ml.getDocument();

doc.getElementById("total").setTextContent("1 250,00 EUR");
for (Element img : doc.querySelectorAll("img:not([alt])")) {
    img.setAttribute("alt", "");
}

pd4ml.writePDF(new FileOutputStream("invoice.pdf"));

getDocument() returns the same tree on every call: changes made through one Document object are seen through any other, and by every later output.

The objects of the API are views of the tree's nodes. Two objects for the same node are equals() and have the same hashCode(), but are not necessarily the same object: compare nodes with equals(), not ==. They can be used as keys of a HashMap or members of a HashSet.

The node types

All classes are in com.pd4ml.dom and extend Node:

Document also describes the document as a whole:

Navigating the tree

Every node has:

An element also has element-only navigation, which skips text and comments:

Lists returned by the API are snapshots: they do not change when the document changes afterwards. It is therefore safe to remove or move the nodes of a list while going through it.

Querying

The searches work on the Document and on an Element; on an element, they look only inside it.

The selectors are those of PD4ML's CSS engine, so anything that works in a style sheet works here: type, class, id and attribute selectors ([href^="http"], [data-x~="y"]), the combinators (descendant, >, +, ~), structural pseudo-classes (:root, :first-child, :nth-child(2n+1), :nth-last-child(), :nth-of-type(), :last-of-type, :only-child, :empty), :not(), :is(), :where(), :has(), :lang(), :dir(), :checked, :any-link, and selector lists (h1, h2, h3). An invalid selector throws an IllegalArgumentException.

// every price cell in the last column
List<Element> prices = doc.querySelectorAll("table.prices td:last-child");

// the table row a cell is in
Element row = cell.closest("tr");

// sections without a heading
for (Element s : doc.querySelectorAll("section:not(:has(h1, h2, h3))")) {
    s.addClass("untitled");
}

Reading elements

Attribute names are case-insensitive, as in HTML. Values are returned as they are in the document, with entities decoded; relative URLs are not resolved (doc.getBaseURI() is the base to resolve them against).

Changing the document

Creating and inserting nodes

Element note = doc.createElement("p");
note.setClassName("note");
note.setTextContent("Prices include VAT.");
doc.getBody().insertBefore(note, doc.querySelector("table.prices"));

Element footer = doc.createElement("div");
footer.appendChild(doc.createTextNode("Generated " + LocalDate.now()));
footer.appendChild(doc.createComment("generated by the invoice service"));
doc.getBody().appendChild(footer);

The rules are those of the browser DOM:

• A node that is already in the tree is moved when it is inserted somewhere else. A node from another PD4ML document is moved into this one.
• A node cannot be inserted into itself or into one of its descendants, a document cannot be inserted at all, and text, comments and the doctype cannot have children. These throw an IllegalArgumentException, as does a ref or old that is not a child of the node.
• A detached node can be kept and inserted again later.

Text

setTextContent(text) replaces everything inside an element with one text node. The text is taken as text, not HTML: < stays a < character. On a Text or Comment node it sets the node's data.

Attributes and classes

The document is read again when it is laid out, so a changed style attribute, class, colspan, width or src takes effect in the next output exactly as if the source had had it:

Element totals = doc.getElementById("totals");
totals.setAttribute("style", "page-break-inside: avoid; border-top: 2px solid black");
doc.querySelector("img.logo").setAttribute("src", "https://example.com/logo-print.png");

Images added or changed are loaded when the document is laid out.

Inserting HTML

doc.querySelector("tbody#items").setInnerHTML(
        "<tr><td>Consulting</td><td>8 h</td></tr>"
      + "<tr><td>Travel</td><td>1</td></tr>");

doc.getElementById("legacy-banner").setOuterHTML("<p class='notice'>See the attached terms.</p>");

• setInnerHTML(html) replaces the element's content with the HTML, parsed the way a browser parses innerHTML: as the content of an element of that tag. Table rows can be set into a <tbody> or <tr>, options into a <select>, list items into a <ul>.
• For <script>, <style>, <textarea> and <title>, the string is set as text, as is. This is the way to replace a style sheet.
• setOuterHTML(html) replaces the element itself with the nodes the HTML parses into, and detaches the element. It throws an IllegalStateException if the element has no parent element.

Adding style sheets

A <style> element added from code applies like one in the source, @page rules included (see CSS @page Rules):

Element style = doc.createElement("style");
style.setTextContent(
        "@page { size: A4 landscape; margin: 15mm;"
      + "  @bottom-center { content: 'Page ' counter(page) ' of ' counter(pages) } }"
      + ".draft { display: none }");
doc.getHead().appendChild(style);

A <link rel="stylesheet" href="..."> added to <head> is loaded when the document is laid out.

Writing HTML back out

Void elements (<br>, <img>, ...) are written without an end tag, script and style content as is, text and attribute values escaped. This is useful to log or archive the document as it was converted:

Files.write(Paths.get("invoice-as-converted.html"), doc.toHtml().getBytes(StandardCharsets.UTF_8));

Inline SVG and custom tags

An inline <svg> and a custom tag (MathML <math>, <qrcode>, and tags registered with addCustomTagHandler()) are drawn from their markup, not from child nodes. In the DOM they have no child nodes: their content is read and changed as HTML with getInnerHTML()/setInnerHTML(), or the whole element is replaced with setOuterHTML(). Their attributes are ordinary attributes:

Element logo = doc.querySelector("svg#logo");
logo.setInnerHTML("<circle cx='50' cy='50' r='40' fill='teal'/>");
logo.setAttribute("width", "120");

Inserting a node into such an element throws an IllegalArgumentException.

Reading the layout

pd4ml.layout() lays the document out into pages as writePDF() would, without writing it, and returns a Layout: where each element is, on which page, and its computed style.

import com.pd4ml.dom.Layout;
import com.pd4ml.dom.LayoutBox;

Layout layout = pd4ml.layout();

System.out.println(layout.getPageCount() + " pages");
for (LayoutBox box : layout.getBoxes(doc.getElementById("signature"))) {
    System.out.println(box);   // page 2: 56.69, 640.10, 481.89 x 85.00 pt
}

A LayoutBox has getPage(), getX(), getY(), getWidth() and getHeight(). Positions are in PDF points (1/72 inch) from the top left corner of the page, not from the bottom left as PDF coordinates usually are. A box is the element's border box (content, padding and border). When the element continues on the next page, it has a box on each page for the part there; an inline element broken over several lines has a box per line.

The layout uses the page size, margins, @page rules, setHtmlWidth() and the other settings in effect, so a writePDF() with the same settings puts the elements at the same places.

Example: keep a block on one page

Layout layout = pd4ml.layout();
Element totals = doc.getElementById("totals");
if (layout.getPages(totals).size() > 1) {
    // split over two pages: start it on a new one
    totals.setAttribute("style", "page-break-before: always");
}
pd4ml.writePDF(out);

Example: fit a document on one page

int htmlWidth = 800;
pd4ml.setHtmlWidth(htmlWidth);
while (pd4ml.layout().getPageCount() > 1 && htmlWidth < 1600) {
    htmlWidth += 100;              // a wider layout is scaled down more
    pd4ml.setHtmlWidth(htmlWidth);
}
pd4ml.writePDF(out);

Example: where to put a signature field

LayoutBox box = pd4ml.layout().getBoxes(doc.getElementById("sign-here")).get(0);
// box.getPage(), box.getX(), box.getY(), box.getWidth(), box.getHeight():
// where a PDF tool should place the signature field after writing

Example: read computed styles

Layout layout = pd4ml.layout();
for (Element h : doc.querySelectorAll("h1, h2")) {
    System.out.println(h.getTextContent() + ": " + layout.getComputedStyle(h, "font-size")
            + " on page " + layout.getPages(h));
}

Timing

Make the changes after readHTML() and before the output:

• $[name] placeholders of setDynamicData() are already filled in and pd4ml:include is expanded when readHTML() returns, so the DOM shows the document with them.
• The document title (getDocumentTitle(), $[title]) is taken from the document by readHTML(), so changing <title> afterwards does not change it. Use setDocumentTitle().
• Settings such as setPageSize(), setHtmlWidth() and setPageHeader() can still be changed between readHTML() and the output; layout() and the outputs use the settings in effect when they are called.
• Writing a document, and layout(), lays it out and moves footnotes and endnotes into place in the tree. Later changes still apply to later outputs, but pd4ml:footnote/pd4ml:endnote elements added after the first layout are not numbered.
• One PD4ML object can write several outputs from one readHTML(), with changes in between: for example a PDF, then the same document with some elements removed as DOCX.

Complete example: an invoice from a template

PD4ML pd4ml = new PD4ML();
pd4ml.useHtmlRenderer(Constants.LayoutEngine.v42);
pd4ml.readHTML(new File("templates/invoice.html").toURI().toURL());
Document doc = pd4ml.getDocument();

doc.getElementById("number").setTextContent(invoice.getNumber());
doc.getElementById("customer").setTextContent(invoice.getCustomerName());

Element rows = doc.querySelector("table.items tbody");
rows.setInnerHTML("");                                  // drop the sample rows
for (Item item : invoice.getItems()) {
    Element tr = doc.createElement("tr");
    for (String value : new String[] { item.getName(), item.getQuantity(), item.getAmount() }) {
        Element td = doc.createElement("td");
        td.setTextContent(value);
        tr.appendChild(td);
    }
    rows.appendChild(tr);
}
doc.getElementById("total").setTextContent(invoice.getTotal());

if (!invoice.isDraft()) {
    for (Element e : doc.querySelectorAll(".draft-only")) {
        e.remove();
    }
}

// the payment details must not be split over two pages
Element payment = doc.getElementById("payment");
if (pd4ml.layout().getPages(payment).size() > 1) {
    payment.setAttribute("style", "page-break-before: always");
}

pd4ml.writePDF(new FileOutputStream("invoice-" + invoice.getNumber() + ".pdf"));

See also

• CSS @page Rules: Page Size, Margins, Named Pages and Margin Boxes
• PD4ML API Reference (Javadoc) -- the com.pd4ml.dom package, PD4ML.getDocument() and PD4ML.layout().
• PD4ML v42 -- HTML5 Element Support Matrix -- what the v42 layout engine renders.