Overview

The Enterprise Integration Service is a Spring Service based on the Apache Camel framework. Apache Camel is a rule-based flow control engine that implements the Enterprise Integration Pattern. It provides an API for creating rule-based flows and for process communication.

Using simple scripts (Groovy), processing routes can be created (also called pipelines) that allow document processing processes to be started in a time-, rule- or event-driven manner.

Route elements can be: Upload to an ECM, OCR Service, Full Text Service, Integration ESB, etc.

The Integration Service can import files into any repository. Next to the over 300 Camel endpoints we provide an import processor for arveo.

Configuration

Configuration parameters

The following are the configuration parameters of the Integration Service:

Table 1. Configuration Parameters
parameter meaning default value

groovy-engine.script-root-path

Root directory for all Groovy scripts. Scripts can be organised in subdirectories (like Java packages).

"./scripts"

groovy-engine.source-encoding

Encoding of script files

UTF-8

groovy-engine.debug

Enable debug mode of the Groovy engine

false

groovy-engine.recompile-source

Recompile scripts regularly

true

groovy-engine.min-recompilation-interval

Interval (in milliseconds) in which scripts are recompiled

10000

file-scanning-interval

Interval (in milliseconds) in which scripts are checked for new or changed script files

60000

Managing routes

Routes are stored in Groovy scripts in the <groovy-engine.script-root-path>/routes directory and are started automatically by the Integration Service. If a script file is changed, the route defined in it is stopped and restarted. If a script file is removed, the route is stopped. For this purpose, it is important that each route has a unique ID that can be defined in the script. The following example shows a simple route that copies files from the "input" directory to "output":

Example of a route
package scripts.routes

import de.eitco.commons.integration.core.ExceptionProcessor
import org.apache.camel.builder.RouteBuilder

return new RouteBuilder() {

    @Override
    void configure() throws Exception {

        onException(Exception.class)
                .process(new ExceptionProcessor())
                .handled(true)
                .maximumRedeliveries(0)

        from("file://input?noop=true")
                .routeId("file-out-test")
                .to("file://output")
    }
}

Camel provides a large number of standard components that can be used in a route. An overview can be found here: https://camel.apache.org/components.html. In addition, further systems can be connected via self-implemented components.

Supplied components

The following components are supplied as standard with the Integration Service:

Other components that are frequently required can be included in the standard delivery on request.

Additional components must be available in the classpath when starting. To do this, you can define your own lib directory, for example, by giving the JVM the parameter -Dloader.path=lib/ when starting. This parameter must be entered in the XML file for the service as a separate <start argument> tag.

hawtio

The Integration Service can be monitored with hawtio. To do this, the Actuator endpoints must be enabled and released for Jolokia. This requires the following adjustments to the configuration:

management:
  endpoints:
    web:
      exposure:
        include: health,info,jolokia

security:
  general:
    secured-ant-matchers: "/api/**"
    open-ant-matchers: "/actuator/**"
    role-for-secured-access: "INTEGRATION_SERVICE_USER"
    cors-configuration:
      allowed-origins: "*"
      allowed-headers: "*"
      allowed-methods: "GET,POST,PUT,PATCH,DELETE,OPTIONS"
      max-age: 3600

In a hawtio instance started as a standalone process, a remote connection to the Integration Service can then be established. The Jolokia endpoint is accessible under the path /actuator/jolokia.

Installation

Windows Service Wrapper

Like all Common Services, the Integration Service is delivered with the WinSW Service Wrapper, with which you can easily create and configure a Windows service. The documentation can be found on GitHub.

Helm Chart

For deployments to a kubernetes cluster, we provide a helm chart containing the integration service.

The integration service helm chart expects the eitco service registry and the eitco config service running. It is advised to use their helm charts for that.

The helm chart is hosted in the repository https://nexus.eitco.de/repository/helm-default/. The name of the helm chart is eitco-integration-service. Assume that the repository https://nexus.eitco.de/repository/helm-default/ is registered as eitco, then you can deploy the integration service with the following command:

helm install eitco/eitco-integration-service --version 10.0.0 --name-template eitco-integration-service

Routes

Of course, since there is no route configured this will not do much. To add groovy routes, add the groovy files to a configmap and give the name of the configmap as the parameter routes.configMap. For example if your routes are in the configmap routes the command will look like that:

helm install eitco/eitco-integration-service --version 10.0.0 --name-template eitco-integration-service --set routes.configMap=routes

Includes

Should you want to add groovy scripts that are only included, but are no routes by itself, you can add them to another configmap, and give the name of the config map as the parameter includes.configMap. These files will be mounted in the include-subdirectory of the groovy script root path. So if your include-scripts are located in the configMap include, install the helm chart as follows:

helm install eitco/eitco-integration-service --version 10.0.0 --name-template eitco-integration-service --set routes.configMap=routes --set includes.configMap=include

Integration Service Classpath

There are several occasions where you need to add jars to the integration services classpath. This could be the case if you want to add additional endpoints (for example the arveo endpoints). Or maybe you do not want to write your routes in groovy but implement a spring component that provides your routes in java or kotlin. In this case you - again - you need to provide your files in a configMap. The parameter name for this config map is extension.configMap. Any jar located there will be part of the classpath of the integraiton service. So, assume that you added a spring boot autostarter that configures your routes in the configmap extensions, you would install the helm chart like this:

helm install eitco/eitco-integration-service --version 10.0.0 --name-template eitco-integration-service --set extensions.configMap=extensions

Custom Deployment

A project using the integration service will make its kubernetes deployment easiest if it creates its own helm chart. This helm chart should have the integration service helm chart (and the helm charts for the service registry, configuration service and other services it might need) as dependencies. That way it can define the configMaps used by the integration service by itself, eliminating the need to do this manually.

Development

In this chapter you will learn about developing custom Camel components.

Architecture overview for the development of own Camel components:

integration service development camel components

Source: https://stackoverflow.com/a/19706238)

The Camel documentation also contains a chapter on developing your own components: https://camel.apache.org/writing-components.html

The components for the {product-name} and for Saperion can be used as examples. They are located in https://git.eitco.de/projects/CMN/repos/integration-service/browse/components.

There is a good overview of how Camel components work on Stackoverflow: https://stackoverflow.com/a/19706238.

Web Authentication

Here we describe the process of authentication for Spring web endpoints. By default, the Integration Service does not provide any Spring web endpoints. When custom Spring web endpoints are added or when the HTTP Actuator interface is required, one of the additional web authentication modules must be added to the classpath. The following web authentication modules are available. Each module contains the actuator endpoints and a configuration class for the respective authentication mechanism.

There is a test-module for each authentication module with a sample configuration.

Overview

The module cmn-integration-web-auth-basic provides a simple HTTP basic authentication configuration that supports one configurable user. The username and password can be configured using the following parameters:

  • integration-service.basic-auth.username: The name of the user (default: integration-service-user)

  • integration-service.basic-auth.password: The user’s password (default: changeme)

This module is recommended in environments where mainly the Actuator web interface is used.

OAuth2

The module cmn-integration-web-auth-oauth2 provides OAuth2 authentication based on the OAuth2 module from commons. This is the recommended module in environments where multiple different users and/or applications will access the web endpoints.

x.509

The module cmn-integration-web-auth-x509 provides an authentication mechanism based on the client’s certificate using mutual SSL authentication (x.509).

This module is recommended in environments where only applications will access the web endpoints.