Configuration

This section will describe the basic configuration for the most important components of the application.

Configuration files

Most of the configuration properties of the web application are specified in properties files which can be found inside the src/main/resources folder of the espd-web Maven module. There are different files for each profile under which the application can be started, for example the application-prod.properties file will be used for the prod profile which must be specified as a startup parameter as -Dspring.profiles.active=prod. If no profile is specified, then a default one will be used by Spring Boot and the corresponding configuration file would be just application.properties. Here you can find more information about properties and configuration with Spring Boot.

Spring

The ESPD application is built from the ground-up with Spring Boot, allowing us to simplify the project configuration and the management of third party libraries.

It does so by specifying in the main pom.xml of the project a parent POM pointing to the Spring Boot starter.

pom.xml

<parent>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-parent</artifactId>
  <version>1.3.2.RELEASE</version>
</parent>

Since the application is using Spring Boot extensively by inheriting from spring-boot-starter-parent, this provides it with the following features:

Default Java compiler level
UTF-8 source encoding
A Dependency Management section, allowing you to omit <version> tags for common dependencies, inherited from the spring-boot-dependencies POM.
Sensible resource filtering, including application.properties and application.yml type of files
Sensible plugin configuration

For more information about the Spring Boot project, you can check the official documentation.

The eu.europa.ec.grow.espd.config package contains the Spring configuration classes. The main class of the application is EspdApplication which makes use of the @SpringBootApplication annotation in order to enable the Spring Boot auto configuration. The package also contains the web, JAXB and web resources configuration files.

Spring MVC

The web configuration part is found in the WebConfiguration class in conjunction with the following Spring Boot properties:

application-dev.properties

spring.mvc.view.prefix=/WEB-INF/views/
spring.mvc.view.suffix=.jsp

# Default locale to use
spring.mvc.locale=en

The WebConfiguration contains the definition of various Spring beans: view resolvers, tiles configuration, locale interceptors, monitoring filter.

WebConfiguration.java

@Bean
UrlBasedViewResolver viewResolver() {
  UrlBasedViewResolver viewResolver = new UrlBasedViewResolver();
  viewResolver.setViewClass(TilesView.class);
  return viewResolver;
}

@Bean
TilesConfigurer tilesConfigurer() {
  TilesConfigurer tilesConfig = new TilesConfigurer();
  tilesConfig.setDefinitions("/WEB-INF/tiles.xml");
  return tilesConfig;
}

Internationalization (i18n)

Concerning internationalization (i18n), the ESPD application takes advantage of the auto-configuration provided by Spring Boot in the MessageSourceAutoConfiguration class.

application.properties

# The location to the resource bundles needed by i18n
spring.messages.basename=i18n/messages

# Loaded resource bundle files cache expiration, in seconds. When set to -1, bundles are cached forever.
spring.messages.cache-seconds=3

The message files are located in the src/main/resources/i18n folder and follow a very simple naming convention of messages_${locale language}.

The application.properties file which is used only for local development is able to refresh potential changes in the message files without requiring a server restart by specifying the spring.messages.cache-seconds property.

The corresponding Java configuration is summarized below.

WebConfiguration.java

import org.springframework.context.annotation.Bean;
import org.springframework.web.servlet.LocaleResolver;
import org.springframework.web.servlet.i18n.CookieLocaleResolver;
import org.springframework.web.servlet.i18n.LocaleChangeInterceptor;
import org.springframework.web.servlet.config.annotation.InterceptorRegistry;

@Bean
LocaleChangeInterceptor localeChangeInterceptor() {
  LocaleChangeInterceptor lci = new LocaleChangeInterceptor();
  lci.setParamName("lang");
  return lci;
}

@Bean
LocaleResolver localeResolver() {
  CookieLocaleResolver resolver = new CookieLocaleResolver();
  resolver.setCookieName("ESPD_LOCALE");
  resolver.setDefaultLocale(Locale.ENGLISH);
  return resolver;
}

@Override
public void addInterceptors(InterceptorRegistry registry) {
  registry.addInterceptor(localeChangeInterceptor());
}

The LocaleResolver enables the application to automatically resolve messages files using the client’s locale which is stored in a ESPD_LOCALE cookie.

The LocaleChangeInterceptor uses an HTTP request parameter named lang to detect the language changes on the server side.

Web Resources Optimization

Static resources

ESPD uses advanced resource handling features provided by Spring MVC and Spring Boot. We have chosen a path that relies on optimizing resources at build-time using WRO4J and leveraging Spring MVC Resolvers and Transformers and WRO4J filter at run-time.

The static resources of the application (Javascript and CSS files) are versioned using a content-based hashing strategy and handled with the idea of cache busting where resources are served with aggressive HTTP cache directives (e.g. 1 year into the future) and relying on version-related changes in the URL to "bust" the cache when necessary. The content-based hash version changes whenever the content of the file changes and this happens at build time.

application-dev.properties

# Cache period for the resources served by the resource handler, in seconds (1 year).
spring.resources.cache-period=31622400

# Enable the Spring Resource Handling chain.
spring.resources.chain.enabled=true

# Enable the content Version Strategy.
spring.resources.chain.strategy.content.enabled=true

# Comma-separated list of patterns to apply to the Version Strategy.
spring.resources.chain.strategy.content.paths=/static/bundle/**

Links to resources are rewritten at run-time using a ResourceUrlEncodingFilter.

WebConfiguration.java

import org.springframework.web.servlet.resource.ResourceUrlEncodingFilter;
import org.springframework.context.annotation.Bean;

/**
* If the template engine you are using calls the response encodeURL() method, the version information
* will be automatically added to the URL of the static resources that will be cached.
* This will work in JSPs in conjunction with spring:url tag.
* <p>It needs to be mapped on '/*'.</p>
*
* @return
*/
@Bean
ResourceUrlEncodingFilter resourceUrlEncodingFilter() {
  return new ResourceUrlEncodingFilter();
}

And this is how the static resources are referenced in the view part:

espdTemplate.jsp

<link rel="stylesheet" type="text/css" href="<s:url value="/static/bundle/all.css"/>">
<script src="<s:url value="/static/bundle/all.js"/>"></script>

For example, a request made to the all.js file would be translated into a request made to a Javascript file with a hash: https://ec.europa.eu/espdstatic/bundle/all-60d9cd4aee2d53a2a4bd69a5546a9d18.js.

WRO4J

Another set of static resources optimizations are handled with WRO4J using a simple Java filter at run-time and the Maven plugin at build-time. WRO4J concatenates and minifies the static resources like Javascript or CSS files into a single file per each type of resource so that the number of HTTP requests made by the clients that load the application is reduced drastically.

Run-time solution with a Servlet filter

The Java filter configuration makes use of Spring Boot auto-configuration provided by the wro4j-spring-boot-starter library.

This is only used by the default Spring Boot profile which should be active only at development time.

Wroj4Config.java

import ac.simons.spring.boot.wro4j.Wro4jAutoConfiguration;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;

@Profile("default")
@Configuration
class Wro4jConfig extends Wro4jAutoConfiguration {

  // only used for development ('default' profile) when we need the Wro4J Filter

}

The run-time properties are defined in the application.properties file.

application.properties

# Integer value for specifying how often (in seconds) the resource changes should be checked. When this value is 0,
# the cache is never refreshed. When a resource change is detected, the cached group containing changed resource
# will be invalidated. This is useful during development, when resources are changed often.
wro4j.resourceWatcherUpdatePeriod=3

# Integer value for specifying how often (in seconds) the cache should be refreshed.
# When this value is 0, the cache is never refreshed.
wro4j.cacheUpdatePeriod=3

wro4j.disableCache=true

wro4j.debug=true

wro4j.filterUrl=/static/bundle

# A comma separated values describing pre processor aliases to be used during processing.
wro4j.managerFactory.preProcessors=fallbackCssDataUri, cssUrlRewriting, cssImport, semicolonAppender, cssMinJawr, jsMin

Build-time solution with Maven

The build-time solution uses a Maven plugin and needs two WRO4J configuration files placed under the src/main/resources folder.

wro.xml

<groups xmlns="http://www.isdc.ro/wro">
  <group name="all">
    <css minimize="false">/static/bootstrap-3.2.0/css/bootstrap.min.css</css>
    <css minimize="true">/static/css/espd.css</css>

    <js minimize="false">/static/jquery/jquery.min.js</js>
    <js minimize="false">/static/bootstrap-3.2.0/js/bootstrap.min.js</js>
    <js minimize="true">/static/js/init.js</js>
  </group>
</groups>

wro.properties

###############################################################################
#####     THIS FILE IS USED AT BUILD TIME BY THE WRO4J MAVEN PLUGIN      ######
###############################################################################

#If true, it is DEVELOPMENT mode, by default this value is true.
debug=false

# A comma separated values describing pre processor aliases to be used during processing.
preProcessors=fallbackCssDataUri,cssUrlRewriting,cssImport,semicolonAppender

postProcessors=cssVariables,cssMinJawr,jsMin

# The alias of the HashStrategy used to compute ETags & cache keys.
hashStrategy=MD5

# The alias of the NamingStrategy used to rename bundles.
namingStrategy=noOp

The Maven plugin bundles all the Javascript and CSS files into the src/main/webapp/static/bundle folder, applying minimization where necessary and creating a all.js and a all.css file.

espd-web.pom.xml

<configuration>
  <wroFile>${basedir}/src/main/resources/wro.xml</wroFile>
  <extraConfigFile>${basedir}/src/main/resources/wro.properties</extraConfigFile>
  <cssDestinationFolder>${basedir}/src/main/webapp/static/bundle/</cssDestinationFolder>
  <jsDestinationFolder>${basedir}/src/main/webapp/static/bundle/</jsDestinationFolder>
  <wroManagerFactory>ro.isdc...factory.ConfigurableWroManagerFactory</wroManagerFactory>
  <ignoreMissingResources>false</ignoreMissingResources>
  <incrementalBuildEnabled>true</incrementalBuildEnabled>
</configuration>

Lombok

To reduce some of the boilerplate code inherent to the Java language, the project uses the Lombok library which leverages Java annotations.

The library can be used in Eclipse by double clicking the lombok.jar and in IntelliJ by installing the Lombok plugin.

If you do not like the basic idea behind Lombok you can delombok the source code and go back to standard Java source code.

Here is a very simple example of how Lombok might be used inside the ESPD project.

TedRequest.java

import lombok.Builder;
import lombok.Getter;

@Builder
@Getter
public class TedRequest {

  private String receptionId;

}

LombokExample.java

public TedRequest prepare() {
  TedRequest request = TedRequest.builder()
                    .receptionId("16-000136-001")
                    .build();
  log.debug("This is the reception id: '{}'.", request.getReceptionId());
  return request;
}

Logging

Logging in the application is handled using the SLF4J API and the chosen implementation is provided by the Logback library.

Since the espd-web module depends on spring-boot-starter-web and this one transitively depends on spring-boot-starter-logging, the default logging implementation configured by Spring Boot is Logback.

The application-${profile}.properties files declare the path to the Logback configuration. The logging configuration files are stored in the espd-web/src/main/resources/logback folder.

application-dev.properties

# The path to the logback configuration file depending on the profile
logging.config=classpath:logback/logback-dev.xml

To use logging in the code, you can take advantage of the facilities provided by Lombok.

Logging.java

import lombok.extern.slf4j.Slf4j;

@Slf4j
class Logging {

  void logSomething(String parameter) {
    log.info("Logging the following value '{}'.", parameter);
  }
}

Monitoring and analytics

Java Melody

Basic monitoring of the ESPD application is handled using the Java Melody library by registering a Java filter inside the Spring application context.

WebConfiguration.java

import org.springframework.context.annotation.Bean;
import net.bull.javamelody.MonitoringFilter;
import net.bull.javamelody.Parameter;

@Bean
MonitoringFilter melodyMonitoringFilter() {
  return new MonitoringFilter();
}

@Bean
FilterRegistrationBean melodyFilterRegistration(MonitoringFilter melodyFilter) {
  FilterRegistrationBean frb = new FilterRegistrationBean(melodyFilter);
  frb.addInitParameter(Parameter.NO_DATABASE.getCode(), "true");
  frb.addInitParameter(Parameter.ALLOWED_ADDR_PATTERN.getCode(),
        "(158\\.16[6-8]\\..*)|(127\\.0\\.0\\.1)|(localhost)");
  frb.addInitParameter(Parameter.URL_EXCLUDE_PATTERN.getCode(), "(/img/.*)|(/js/.*)|(/css/.*)|(.*/.woff)");
  return frb;
}

The filter configuration makes the monitoring accessible only to a certain range of IP addresses, excludes requests pointing to static resources and specifies that no database monitoring should be active.

Additional monitoring capabilities could be added by activating the Spring Boot actuator features.

Piwik

Analytics capabilities for the application are provided via the Piwik server of DG Growth.

application-dev.properties

# Enable or disable the Piwik integration
piwik.enabled=false

# Piwik id for ESPD project
piwik.id=2

# Piwik server for ESPD project
piwik.server=https://webgate.ec.europa.eu/pwar/piwik.php

[NOTE] You might want to disable the integration with the Piwik server of DG Growth by setting the piwik.enabled parameter to false in the corresponding application.properties file.

Criteria definitions

With the 2016.06 version, the criteria requirement groups have been restructured with regards to interoperability with the VCD application and three JSON files have been added under src/main/resources/criteria. These files contain the definitions for the exclusion, selection and other criteria used by the static version of ESPD.

exclusionCriterion.json

{
  "name": "Conflict of interest due to its participation in the procurement procedure",
  "uuid": "b1b5ac18-f393-4280-9659-1367943c1a2e",
  "shortName": "Conflict of interest due to its participation in the procurement procedure",
  "description": "Is the economic operator aware of any conflict of interest, as indicated in national law, the relevant notice or the procurement documents due to its participation in the procurement procedure?",
  "criterionType": {
    "description": "Grounds for exclusion relating to possible conflicts of interests",
    "espdType": "CONFLICT_OF_INTEREST",
    "code": "CRITERION.EXCLUSION.CONFLICT_OF_INTEREST.PROCEDURE_PARTICIPATION"
  },
  "legislationReference": {
     "title": "DIRECTIVE 2014/24/EU OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 26 February 2014 on public procurement and repealing Directive 2004/18/EC",
     "description": "Directive 2014/24/EU",
     "url": "http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex:32014L0024",
     "article": "57(4)"
  },
  "groups": [
    {
    "name": "G1",
    "id": "30450436-f559-4dfa-98ba-f0842ed9d2a0",
    "requirements": [
      {
      "id": "974c8196-9d1c-419c-9ca9-45bb9f5fd59a",
      "description": "Your answer?",
      "responseType": "INDICATOR",
      "espdCriterionFields": [
          "answer"
      ]
      }
    ]
    }
  ],
  "espdDocumentField": "conflictInterest"
}

ESPD Exchange Data Model

The ESPD-EDM is a separate project containing the XML schemas used to generate the JAXB annotated Java classes.

pom.xml

<dependency>
  <groupId>eu.europa.ec.grow.espd</groupId>
  <artifactId>exchange-model</artifactId>
  <version>${espd-exchange-model.version}</version>
</dependency>

The marshalling and unmarshalling of the XML files produced by the application is handled by a Spring Jaxb2Marshaller.

The JAXB configuration declares the packages to be scanned for potential JAXB annotated classes.

JaxbConfiguration.java

import grow.names.specification.ubl.schema.xsd.espdrequest_1.ESPDRequestType;
import grow.names.specification.ubl.schema.xsd.espdresponse_1.ESPDResponseType;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.oxm.jaxb.Jaxb2Marshaller;
import javax.xml.bind.Marshaller;
import java.util.HashMap;
import java.util.Map;

@Configuration
public class JaxbConfiguration {

@Bean
public Jaxb2Marshaller jaxb2Marshaller() {
    Jaxb2Marshaller jaxb2Marshaller = new Jaxb2Marshaller();
    jaxb2Marshaller.setPackagesToScan(ESPDRequestType.class.getPackage().getName(),
            ESPDResponseType.class.getPackage().getName());
    Map<String, Object> map = new HashMap<>(2);
    map.put(Marshaller.JAXB_FORMATTED_OUTPUT, true);
    jaxb2Marshaller.setMarshallerProperties(map);
    return jaxb2Marshaller;
  }
}

Date and time

Date and time objects are handled with the Joda-Time library. There are two adapters that are used to populate the Date objects inside the JAXB POJOs. These adapters convert and parse String objects into LocalDate or LocalTime Joda-Time objects.

LocalDateAdapter.java

import org.joda.time.LocalDate;
import org.joda.time.format.DateTimeFormat;
import org.joda.time.format.DateTimeFormatter;

public final class LocalDateAdapter {

  private static final DateTimeFormatter DATE_FORMAT = DateTimeFormat.forPattern("YYYY-MM-dd");

  private LocalDateAdapter() {

  }

  public static LocalDate unmarshal(String v) {
      return LocalDate.parse(v, DATE_FORMAT);
  }

  public static String marshal(LocalDate v) {
      return v.toString(DATE_FORMAT);
  }
}

REST template

The interaction with external RESTful APIs (e.g. TED) is done with the Spring RestTemplate.

There is one global Spring bean of type RestTemplate defined in the application.

EspdApplication.java

import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.context.web.SpringBootServletInitializer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.http.client.SimpleClientHttpRequestFactory;
import org.springframework.web.WebApplicationInitializer;
import org.springframework.web.client.RestTemplate;

@SpringBootApplication
@ComponentScan("eu.europa.ec.grow.espd")
public class EspdApplication extends SpringBootServletInitializer implements WebApplicationInitializer {

  @Value("${http.client.connect.timeout.millis:30000}")
  private int connectTimeout;

  @Bean
  ObjectMapper objectMapper() {
    return new ObjectMapper();
  }

  @Bean
  RestTemplate restTemplate() {
    RestTemplate restTemplate = new RestTemplate();

    SimpleClientHttpRequestFactory rf = (SimpleClientHttpRequestFactory) restTemplate.getRequestFactory();
    rf.setReadTimeout(connectTimeout);
    rf.setConnectTimeout(connectTimeout);

    return restTemplate;
  }
}

You can find an example of how to use the RestTemplate in the eu.europa.ec.grow.espd.ted.TedService class.

TED REST service

Information about the procurement procedure can be provided by the publication office via the TED REST service. In order to be able to retrieve the information from their remote service we need to provide four parameters.

application-dev.properties

# The base URL of the TED contract notice REST service
ted.api.base.url=https://esentool.ted.europa.eu/api/espd/v1.0/notice

# Timeout in milliseconds for the Spring RestTemplate client
rest.template.connect.timeout.millis=30000

# user for TED API
ted.api.user=passed as server startup parameter

# Password for TED API
ted.api.password=passed as server startup parameter

The part of the code that handles the TED service can be found in the eu.europa.ec.grow.espd.ted package.

PDF printing

Printing the ESPD Request and ESPD Response to PDF files is achieved via Apache FOP. To produce a PDF file, we start from the HTML content of the ESPD entity which we want to print and use an XSLT stylesheet that converts the HTML to XSL-FO. This is the first step in the processing chain. The second step will be done by Apache FOP when it reads the generated XSL-FO document and formats it to a PDF document.

XSL-FO is an XML vocabulary that is used to specify a pagination and other styling for page layout output. The acronym FO stands for Formatting Objects. XSL-FO can be used in conjunction with XSLT to convert from any XML format into a paginated layout ready for printing or displaying. The XSLT files taking care of the HTML transformation can be found in src/main/resources/tenderned/pdfrendering/xslt.

The printing implementation resides inside the eu.europa.ec.grow.espd.tenderned.HtmlToPdfTransformer class while the infrastructure setup of Apache FOP is defined in the eu.europa.ec.grow.espd.config.ApacheFopConfig class.

Apache FOP requires a configuration file whose location can be configured via the apache.fop.xml.configuration.location application parameter. The default locations point to files belonging to the src/main/resources/grow/fop/ folder.

In order to display the PDF correctly across all European languages we need to use a font which contains all the glyphs for these languages. Otherwise, if no glyph can be found for a given character, Apache FOP will issue a warning and use the glyph for "#" (if available) instead. The font also needs to be embedded in the PDF so that the document is correctly displayed on all clients which are viewing the generated files. For these reasons we are using a custom font called DejaVu.

Inside the fop-config.xml file we need to make sure that the font files are correctly configured and then properly loaded by Apache FOP across multiple Servlet containers. Please notice the ember-url attribute.

fop-config.xml

<font metrics-url="fonts/DejaVuSans/ttf/DejaVuSans.xml"
      embed-url="fonts/DejaVuSans/ttf/DejaVuSans.ttf">
  <font-triplet name="DejaVuSans" style="normal" weight="normal"/>
</font>

The application parameter apache.fop.defaultBaseUri can be specified to load the font files via different strategies. Embedded fonts can be loaded via absolute (Weblogic in Production mode) or relative paths or via classpath depending on the chosen strategy. When using an embedded server it is recommended to use the classpath approach.

The fonts are loaded by Apache FOP by using a custom org.apache.xmlgraphics.io.ResourceResolver which looks for them via the Spring ResourceLoader mechanism in a portable and consistent way across different Serlvet containers.

EspdResourceResolver.java

@Override
public Resource getResource(URI uri) throws IOException {
  log.debug("--- Fop resource resolver get resource: '{}'.", uri);
  InputStream is = resourceLoader.getResource(uri.toASCIIString()).getInputStream();
  return new Resource(is);
}

To be able to load the fonts when running the application with an embedded server we need to copy them in a location relative to the application context root.

pom.xml

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-war-plugin</artifactId>
  <configuration>
    <warName>${project.artifactId}</warName>
    <webResources>
      <resource>
        <directory>src/main/resources/fonts</directory>
        <targetPath>fonts</targetPath>
        <filtering>false</filtering>
      </resource>
    </webResources>
  </configuration>
</plugin>

Web Page Initialization Parameters

Certain HTTP parameters can be passed to the /initialization URL of the ESPD application to initialize specific fields. When issuing such a request, the application redirects the client to the /filter page.

lang is used to specify the language to be used by the application. It consists of a two letter code from the supported languages of the application. Example:
agent is used to choose between a contracting authority, contracting entity and economic operator. The only accepted values are:
- ca for contracting authority
- ce for contracting entity
- eo for economic operator
action defines what the user would like to do. The accepted values are:
- ca_create_espd_request for selecting the Create a new ESPD option as a contracting authority or contracting entity
- ca_reuse_espd_request for selecting the Reuse an existing ESPD option as a contracting authority or contracting entity
- ca_review_espd_response for selecting the Review ESPD option as a contracting authority or contracting entity
- eo_import_espd for selecting the Import ESPD option as an economic operator
- eo_merge_espds for selecting the Merge two ESPDs option as an economic operator
- eo_create_espd_response for selecting the Create response option as an economic operator
country for selecting the desired country of the authority or economic operator. It must be the two letter code of the country in uppercase.

When the agent is a contracting authority or contracting entity, the following parameters are supported for filling in the fields belonging to Part I - Information about the procurement procedure section.

officialName
procurerCountry - must be a two letter country code in uppercase
title
description
fileRefByCA
tedReceptionId specifies the received notice number

When the agent is economic operator, the parameters below can additionally be configured to initialize Part II Information concerning the economic operator - Section A:

name
vatNumber
anotherNationalId
website
street
postalCode
city
country - must be a two letter country code in uppercase
contactName
contactEmail
contactPhone

Initialization page request example.sh

HTTP POST @ espd/initialization?country=RO&city=Drobeta&agent=eo&action=eo_create_espd_response&procurerCountry=FR&lang=ro

All the parameters described above are defined in the EspdInitializationParameters class. It is recommended to use HTTP POST requests but HTTP GET requests are also supported.