Linux service targets¶

The Agent can deploy a renewed certificate to HAProxy, Caddy, Traefik, LiteSpeed, Postfix, Dovecot, PostgreSQL, MySQL and Tomcat on Linux, in addition to the NGINX, Apache httpd and Tomcat types of the terminal interface.

How it works¶

The Agent service runs unprivileged: it cannot write into /etc/haproxy or reload a service itself. For these targets it uses the root hook runner:

1. The Agent saves the certificate as PEM files with an unencrypted key into its own folder (one the service may write; the terminal interface prepares it).
2. A hook.run step asks the root hook runner to run the starter hook /etc/certintel/hooks.d/certintel-<kind>.sh (owned by root:root, mode 0700).
3. The hook installs the files where the service reads them, with the service's owner, group and mode, runs the configuration test and reloads the service.

The hook is an ordinary shell script; the paths, owner, group, mode and commands are written into it and you can edit them. The configuration test runs before the reload, so a mistake stops the script and the running service keeps its current certificate. An existing hook is never replaced: to start again, remove it first. Its output is in journalctl -t certintel-agent-hooks.

A reload returns before the service serves the new certificate, so the starter hooks of services whose workflow then checks the endpoint wait briefly after the reload: 2 seconds for NGINX, Apache httpd, HAProxy and Caddy, 5 seconds for LiteSpeed (a full graceful restart), Traefik (it reads the files again once the hook touches its configuration) and Tomcat (after its restart).

Targets¶

$ID stands for the certificate ID ($CERTINTEL_CERTIFICATE_ID). The paths are the defaults the terminal interface fills in; change them to what your service reads.

The files must not be writable by other users; the hook builder refuses such a mode.

Point the service at the files¶

The hook only installs the files. Configure the service once to read them, for example:

Server detection can read the current paths from an existing HAProxy or LiteSpeed configuration and pre-fill them.

Add the TLS site without restarting the service. HAProxy, Caddy and MariaDB do not start while the certificate files they name are missing, and the starter hook reloads a running service (a reload does not start a stopped one): restart such a service only once the first certificate's files exist.

Tomcat¶

The Tomcat (starter hook) workflow (template tomcat-standard, Agent 1.9.0+) gets a certificate, saves it as PEM files in the Agent's folder, and runs the root starter hook certintel-tomcat.sh, which installs the files for Tomcat and restarts Tomcat. The Agent then checks that Tomcat's HTTPS port serves the new certificate. It works with Tomcat 9, 10 and 11, which read PEM files directly: no keystore, keystore password or Java tool is involved.

What you need:

• the Agent on the Linux server, connected and approved;
• Tomcat 9, 10 or 11 running as a systemd service;
• the name resolves to the server, and Tomcat's HTTPS port is reachable from the server itself.

1. Write the hook, once, on the server:

sudo certintel hooks starter tomcat --write

It writes /etc/certintel/hooks.d/certintel-tomcat.sh (owner root, mode 0700) and prints the connector lines of step 2. The dashboard refuses a Tomcat proposal until the Agent reports this script installed.

2. Add the connector to server.xml, once. Add these lines inside <Service name="Catalina"> in conf/server.xml (/etc/tomcat10/server.xml on Debian and Ubuntu, /etc/tomcat/server.xml on the Red Hat family), with your HTTPS port and the certificate's id as the dashboard and certintel status show it:

<Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol" SSLEnabled="true">
  <SSLHostConfig>
    <Certificate certificateFile="conf/certintel/CERTIFICATE-ID-cert.pem"
                 certificateKeyFile="conf/certintel/CERTIFICATE-ID-key.pem"
                 certificateChainFile="conf/certintel/CERTIFICATE-ID-chain.pem" />
  </SSLHostConfig>
</Connector>

The paths are relative to Tomcat's base folder, so the lines are the same on every system. The script never edits server.xml. Leave Tomcat running after adding the lines: the script restarts it.

Where the files go. conf/certintel/ below Tomcat's base folder (on Debian /etc/tomcat10/certintel/): the owner is root and the group is Tomcat's group, the files have mode 0640 and the folder 0750, so Tomcat can read the key and cannot replace it.

What the script finds by itself.

When it finds none, or the wrong one, set TOMCAT_BASE, TOMCAT_USER or TOMCAT_UNIT at the top of the script. It stops and says which one to set, and it stops when it finds two Tomcats.

What restarts. Tomcat does not load a new certificate while it runs, so the script runs systemctl restart on Tomcat's unit; connections are cut for the seconds Tomcat takes to start. It restarts Tomcat only when server.xml names the files, and only after it installed new files or server.xml changed. Until server.xml names the files it installs them, prints the connector lines to the journal (journalctl -t certintel-agent-hooks) and fails the step without a restart.

The first certificate. If the connector is already in server.xml, the first run installs the files, restarts Tomcat and passes. If it is not, the first run fails and the dashboard waits for the site: add the lines, and the Agent tries again every 30 seconds for about 15 minutes, or at once with sudo certintel deployments retry CERTIFICATE-ID.

Renewal is automatic: new files, one restart, and the Agent confirms that the port serves the new certificate.

The port. Get a certificate for a server uses port 8443. For another port, create the workflow with New workflow and set its HTTPS port; use the same port in the connector.

The keystore templates tomcat-jks and tomcat-p12 are for Windows only. The Agent does not detect Tomcat on a server; you choose Tomcat (starter hook) as the server type.

In the terminal interface¶

Run sudo certintel tui, press a on the Workflows screen and choose the service in Type on the deployment page. The page pre-fills the target paths, owner, group, mode and the test and reload commands from the table above; you can change each one. A service type always saves an unencrypted, owner-only key, because the service cannot read a password-protected one. When you save, the terminal interface prepares the Agent's folder and writes the starter hook as root. S (post-renewal scripts) also offers each starter as Install for <Service> (certintel-<kind>.sh).

From the command line¶

certintel hooks starter haproxy           # print the script
sudo certintel hooks starter haproxy --write

certintel hooks starter KIND [--write] prints the starter hook certintel-KIND.sh for one of haproxy, caddy, traefik, litespeed, postfix, dovecot, postgresql, mysql or tomcat (and nginx and apache, which test the configuration and reload the web server; the server reads the saved files as root), or with --write (as root) writes it to /etc/certintel/hooks.d and prints Wrote … (root:root, mode 0700). Check the paths, owner and commands in it before the first renewal. Exit codes: 0 success, 1 the script could not be written (for example script … already exists; edit it, or remove it first to start again), 2 invalid usage or not root.

Dashboard-assigned workflows¶

The catalogue templates haproxy-standard, caddy-standard, traefik-standard, litespeed-standard, postfix-standard, dovecot-standard, postgresql-standard, mysql-standard and tomcat-standard, shown as <Service> (starter hook), can be assigned from the dashboard. Their fields are Agent folder (the absolute folder the Agent saves the PEM files in, for example /etc/certintel/certs/haproxy), Filename prefix (default certificate) and, for HAProxy, Caddy, Traefik and LiteSpeed, Verification hostname and Verification port (443) for an endpoint check after the reload. Tomcat has Verification hostname and HTTPS port (8443).

The starter hook is installed on approval (Agent 1.14.0+)¶

An assignment carries no script. From Agent 1.14.0 the Agent installs its own starter hook when it is missing and a workflow that needs it is approved on the device or applied by the auto-apply policy:

• The service asks the root hook runner for the script by kind. The runner writes the same certintel-KIND.sh that sudo certintel hooks starter KIND --write writes, with its default paths and commands, as root:root with mode 0700. It never replaces an existing script, so one you edited stays.
• certintel assignments show and the terminal interface's approval question list it before you approve: Installs: certintel-nginx.sh: the agent's own starter hook, written to /etc/certintel/hooks.d (root, mode 0700) when this is approved; it runs as root after each renewal.
• The event starter-hook-installed records the script, the assignment and who approved it (a local administrator or the auto-apply policy).
• If the script cannot be installed, nothing is changed on the device, the dashboard shows the reason, and Send again proposes it once more.

The default scripts suit a standard installation of each service. Where the service's unit or paths differ (Apache on the Red Hat family is httpd, not apache2), write the script first with hooks starter KIND --write and edit it; the Agent then leaves it alone.

Before you assign a template¶

With an earlier Agent, or with install-starter-hooks off, an assignment cannot create a script and step 1 is needed. Step 2 always applies.

1. Install the starter hook: sudo certintel hooks starter KIND --write, or create the workflow once in sudo certintel tui. Until the hook exists the Agent refuses the assignment with starter hook certintel-KIND.sh is not installed; run certintel hooks starter KIND --write as root. The Agent reports whether each template's hook is installed, so the dashboard shows Needs setup on the device on the form and refuses to send the proposal (HTTP 422) until the Agent has reported the hook. hooks starter KIND --write reports it at once; a hook installed another way is reported with the Agent's next check of dashboard assignments (about every 5 minutes), or at once with sudo certintel assignments refresh.
2. Choose an Agent folder the Agent may write. A folder directly below /etc/certintel/certs (for example /etc/certintel/certs/haproxy) needs no preparation: the installer gives /etc/certintel/certs to the Agent's account and the service unit allows it, so the Agent creates the folder itself. Any other folder must be prepared once with sudo certintel tui. The Agent refuses the assignment when it arrives, before any certificate is ordered, if the folder is one the service cannot write, or if a file the save step would replace there belongs to root or is a link (the agent's service cannot replace a file it would save in … (… belongs to root); remove that file (the agent then creates its own) or choose another file name prefix).

HAProxy from the dashboard¶

Get a certificate for a server on the dashboard's Overview proposes haproxy-standard with the Agent folder /etc/certintel/certs/<certificate-id> (the certificate id is the first name with dots as dashes). For Agent 1.14.0+ with install-starter-hooks on, the page says Nothing to install first and names the script the Agent installs when the proposal is approved or applied. Otherwise it gives the commands to run first:

sudo certintel hooks starter haproxy --write
sudo install -d -o root -g haproxy -m 0750 /etc/haproxy/certs

The HAProxy starter hook creates a missing /etc/haproxy/certs itself (as root:haproxy, mode 0750, when its parent exists; Caddy's and Traefik's hooks do the same for their folders). The page shows the hook as the server last reported it (Installed or Not installed yet): hooks starter haproxy --write reports it to CertIntel at once, sudo certintel assignments refresh does too, and the page reads the report again by itself (Check again does so at once); Propose the certificate appears once the hook is installed. The hook installs the combined file /etc/haproxy/certs/<certificate-id>.pem (full chain and key): exactly the file the bind line below names, so the line needs no change. The certificate ID is the name the page shows (the hook calls it $CERTINTEL_CERTIFICATE_ID); the starter hooks list the files they write in their header comments, with <certificate id> where the path holds it. The page shows the bind line, with a Copy button, before you propose. As soon as the Agent has saved the files (within about a minute of the proposal on a server that applies proposals by itself, otherwise of your approval), add it to the frontend that serves the site:

bind :443 ssl crt /etc/haproxy/certs/www-example-com.pem

and run sudo haproxy -c -f /etc/haproxy/haproxy.cfg && sudo systemctl reload haproxy.

On Linux, nginx-standard and apache-standard are built the same way: they save <prefix>-chain.pem and <prefix>-key.pem (an unencrypted key) in the Agent folder, run certintel-nginx.sh or certintel-apache.sh as root (nginx -t -q or apachectl configtest, then systemctl reload), and check that the endpoint serves the new certificate. Point ssl_certificate / ssl_certificate_key (NGINX) or SSLCertificateFile / SSLCertificateKeyFile (Apache) at those two files; the Agent does not edit the server's configuration. Agent 1.14.0+ installs the hook when the workflow is approved; with an earlier Agent install it first with sudo certintel hooks starter nginx --write (or apache). Tomcat on Linux uses tomcat-standard (see Tomcat); the Tomcat JKS and PKCS#12 templates are for Windows only.

Replacing the files the server already reads¶

(Agent 1.14.0+) nginx-standard and apache-standard have an optional setting, Certificate file NGINX reads (or Certificate file Apache httpd reads). Set it to the certificate file the server uses now: the path its configuration names with ssl_certificate (NGINX) or SSLCertificateFile (Apache). The Agent then replaces that file with the new certificate and its chain, and the key file the configuration names with it with the new key, each time it issues or renews the certificate. You do not change the server's configuration, and you do not prepare a folder.

When you start a workflow from a server the Agent detected (Create workflow under Detected on this server), the dashboard fills the setting in for you and says so above the form.

What to know:

• Any folder that only root can change works, for example /etc/letsencrypt/live/www.example.com/fullchain.pem or a file below /root/.acme.sh. The files are replaced by the Agent's root helper (the one that runs the starter hook), not by the Agent's service, which still writes only below /etc/certintel/certs.
• The folder must belong to root. The folder that holds each file, and every folder above it, must be owned by root and not writable by its group or by others. Otherwise the step fails with folder … can be changed by an account other than root, so the runner does not write to … as root; nothing was replaced. Correct the folder (chown root, chmod go-w), or leave the setting empty.
• A server that already reads the Agent's files needs no setting. If the configuration names files below /etc/certintel/certs, the server gets every renewal already. Leave the setting empty; the dashboard does not fill it in for such a server.
• Only files the server's configuration names are replaced. The Agent reads the NGINX or Apache configuration on the machine each time. If no configuration names the file as a certificate, the workflow step fails with … is not a certificate file the NGINX or Apache configuration on this host names …; nothing was replaced, and no file is written. You never enter the key file: the Agent takes it from the same configuration.
• Only with a certificate for that site. The new certificate must hold at least one of the names the configuration gives the site (server_name, ServerName, ServerAlias). A certificate for other names is refused with the new certificate (…) holds none of the names the server that reads … answers for (…), so a workflow for one site cannot replace the certificate of another. A site without a server name takes any certificate.
• A certificate that several sites share keeps all their names. When more than one site reads the certificate file, the new certificate must hold every one of their names that the present certificate holds. If it does not, the step fails with the new certificate (…) does not hold …, which the certificate in … holds now for the servers that read it. Add the missing names to the workflow's certificate.
• A key or chain file shared with another certificate is not replaced. If another site's configuration names the same key file or chain file together with a different certificate, the step fails with … is also named by …, together with other certificate or key files: replacing it would break that site.
• The job result says so. The script step of a successful job reads replaced 2 of the server's certificate files (3 with a separate chain file). The files are named in journalctl -t certintel-agent-hooks.
• Each new file keeps the owner, group and mode of the file it replaces. A key file is never left readable by other users. A link, such as the ones in certbot's live folder, is replaced by a regular file; the file it pointed to is left alone.
• Apache with SSLCertificateChainFile: the certificate file gets the certificate alone and the chain file gets the chain. Otherwise the certificate file gets the certificate followed by its chain. A certificate that comes without an issuing chain is refused while the configuration names a chain file.
• Stop the tool that renewed the certificate before (certbot, acme.sh), or it will put its own certificate back at its next renewal.
• The Agent keeps its own copy in the Agent folder, as before.
• Leave the setting empty to keep the earlier behaviour: the Agent saves the files in the Agent folder and you point the server at them.

The setting is for NGINX and Apache httpd. The other starter hooks (HAProxy, LiteSpeed and the rest) install the files at the paths written in their own script.

Related pages¶

• Linux Agent
• Post-renewal scripts (hooks)
• Server detection
• Modular workflows