3rd level RESTful API with Spring

By Tanja Hackenberg 2 Jahren agoNo Comments
Home  /  Tech Corner  /  3rd level RESTful API with Spring
In a modern world of distributed systems usually it is just a matter of time when your application has to communicate with another one. This communication can be one- or bidirectional and in the latter case you have to both consume other application’s API and roll out one for your application. That’s exactly what happened to us in Visual Meta and in this post I will tell from our experience how to build an API from scratch with Spring technology stack. Nowadays the most popular standard for API is REST, which states for REpresentational State Transfer. It is an architectural style for building communication layer in distributed systems. REST defines general concepts rather that strict rules, hence every API that enables such communication is RESTful. However the level of RESTfulness may be different, not every API is a mature API. Richardson Maturity Model model defines 4 levels of maturity:  
Level 0 API uses HTTP as a tunnel for remote interactions, but does not leverage any of the mechanisms of the web. Such API is usually based on Remote Procedure Invocation through a single URI, e.g SOAP, XML-RPC.
Level 1 API uses separate URIs for its resources, but the method to be executed is either still part of request body, e.g. {“action”:”delete”}, or becomes part of the URI, e.g. /photo/13/delete.
Level 2 API uses HTTP verbs (GET, POST, PUT, DELETE, etc.) to distinguish the operations performed against the resource.
Level 3 API uses Hypermedia to link the resources, client does not need to have the prior knowledge about URI structure, resources are revealed by following links from API entry point.
  We were aiming for level 3 from the very beginning once we started to build our API and if you are planning to build an API for your application you should do the same. You can find more information on the maturity model in “Richardson Maturity Model“ article by Martin Fowler or in “REST in Practice: Hypermedia and Systems Architecture” book by Jim Webber, Savas Parastatidis and Ian Robinson. Before we dive into implementation details I recommend you to check out and get familiar with our example REST API application as I will be referencing it a lot in the next sections. The source code is available at our Git repository. We use maven to build our applications at Visual Meta, hence it is a maven project, though personally I prefer and would recommend gradle.

Spring Boot

To expose an API one obviously needs a web server that will respond to HTTP requests: there are different Servlet containers such as Tomcat or Jetty, but we decided in favor of Spring Boot which makes it very easy to build standalone Spring applications and has an embedded web server so there is no need to deploy war files, after building an application we get a jar that we can “just run” to start serving API calls. The embedded web server is configurable with Spring application.properties and that is very convenient for the development: since it is part of the source code every developer has it from version controll system and there is no need to install and configure a standalone web server on every developer machine. Spring Boot also takes care of dependency management: it ensures that Spring libraries used in the application have compatible versions and also has predefined dependency packs, for example a spring-boot-starter-web pack contains Spring libraries required to build a web application. In order to leverage Spring Boot in our example application we specify spring-boot-starter-parent as a parent in pom.xml, add dependency on spring-boot-starter-web and add the Spring Boot Maven plugin to build plugins:
<parent>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-parent</artifactId>
	<version>1.2.5.RELEASE</version>
</parent>
<dependencies>
	<dependency>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-web</artifactId>
	</dependency>
</dependencies>
<build>
	<plugins>
		<plugin>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-maven-plugin</artifactId>
		</plugin>
	</plugins>
</build>
Once we have maven dependencies in place, we can already create an entry point for our application:
@SpringBootApplication
public class Application {
    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }
}
The @SpringBootApplication annotation not only marks the class as an entry point of spring boot application but also enables auto configuration, web mvc and component scan, so there is no need to add specific annotations for that. Now if one runs this class as either from IDE or console it will fire up the Spring Boot application which will serve requests at 8080 port (it is possible to change the port by specifying server.port property in application.properties file).

Domain model

Development of almost every program starts with domain model and API isn’t an exception. There may be a temptation to reuse the existing model of the application you build API for (in our case it would be Visual Meta back-end), but I’d rather refrain from doing this. Usually the domain model of underlying application is far more complex than one you want to expose to outer world, plus you may want to restructure the domain model so it fits better for the API client needs. API Gateway in this case acts as a facade of your application, in fact you may even have multiple API Gateways to the same application, each with its own domain model. The concept of API Gateways will be featured in more details in a separate post. Domain model of our example application is very basic, but derived from actual Visual Meta domain. There we have shops and products, both with very simple set of properties:
@Entity
public class Shop {

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    @JsonIgnore
    public long id;

    public String name;

    public String siteUrl;



   …
}

public class Product {

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    @JsonIgnore
    public long id;

    public String name;

    public Double price;

    @JsonIgnore
    public long shopId;

   …
}
Notice that we exclude primary and foreign keys from response JSON with @JsonIgnore annotation: the primary keys will be part of URIs, so there is no reason to repeat ourselves in payload, and foreign key relation will be featured by hypermedia as a link between two entities.

Repositories

A repository is a place where entities of a domain model class are stored. It can be an in-memory repository (a HashMap in the simplest case) purged every time when application shuts down, or, which is more realistic scenario, a persistent repository (e.g. SQL or NoSQL database). In our example application we use in-memory repositories as we want the example to be self-contained so there is no need to install and configure SQL server in order to run it. Java Persistence API provides an abstraction layer for ORM frameworks, so if one builds repositories on top of JPA it becomes relatively easy to switch between different repository strategies. This can be really handy in the development process, when in-memory repositories are used in dev or testing environment and persistent repositories are used in production. As well as in the case with web Spring Boot has a starter dependency pack for working with Java Persistence API:
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
Spring Boot can also auto-configure in-memory H2, HSQL and Derby databases. You don’t need to provide any connection URLs, simply include a build dependency to the embedded database that you want to use:
<dependency>
    <groupId>com.h2database</groupId>
    <artifactId>h2</artifactId>
    <scope>runtime</scope>
</dependency>
That is exactly what we do in the example application, but whenever we find appropriate we can easily switch, e.g. to MySQL repository, by specifying a DataSource in the application.properties: spring.datasource.url=jdbc:mysql://localhost/rest_example_db spring.datasource.username=dbuser spring.datasource.password=dbpass spring.datasource.driver-class-name=com.mysql.jdbc.Driver That’s all it takes to switch the repository strategy, no code changes required! In fact with Spring JPA we don’t even need to implement the repositories themselves, we just define their interface and let Spring do the rest. The ShopRepository of our example application is nothing more than this:
public interface ShopRepository extends CrudRepository<Shop, Long> {

}

Controllers

Once we have model and repositories ready we can start implementing controllers. All the required libraries are already included by the spring-boot-starter-web, so all we need is to create a class, mark it with @RestController annotation and map methods to specific URIs, it is also possible to bind the generic path to the controller itself, so there is no need to repeat it for every method. In its simplest form the controller will just translate the URL calls to correspondent methods on repositories. For instance, if we stripped the 3rd level hypermedia stuff from ShopController it would look like this:
@RestController
@RequestMapping(value = "/shops")
public class ShopController extends ApiController {

    private final ShopRepository shopRepository;

    @Autowired
    public ShopController(ShopRepository shopRepository) {
	this.shopRepository = shopRepository;
    }

    @RequestMapping(method = RequestMethod.GET, value = "")
    @ResponseStatus(HttpStatus.OK)
    public Iterable<Shop> listShops() {
	return shopRepository.findAll();
    }

    @RequestMapping(method = RequestMethod.POST, value = "")
    @ResponseStatus(HttpStatus.CREATED)
    public void createShop(@RequestBody Shop shop) {
	shopRepository.save(shop);
    }

    // other URI callbacks here
}

As you may have noticed both createShop and listShops methods a mapped to the same URI, but because the request method is different Spring knows which method to call. This is what makes API the second level API, generally request methods are mapped as following:
GET retrieving the resource, never changes the state of the resource
PUT full update of the resource, all the fields missing in the request payload are deleted from the resource
PATCH> partial update of the resource, only fields present in the request  payload are updated
DELETE deleting the existing resource
POST creating a new resource, or any other action that cannot be translated to a more specific method
Some clients may not support all the variety of HTTP methods, for example AJAX based applications can issue only POST and GET requests. Does this mean you should restrict your API only to these two methods if you expect such an application to be among your clients? Definitely not. You just need to provide an adapter instead. We did it with _method query parameter. By specifying these parameter client can override the HTTP method value. For example, PUT method can be done via POST if clients appends _method=PUT to the request URL. With Spring it takes a tiny effort to build such an adapter, all you need is to register a HiddenHttpMethodFilter, in the example application you can find it in our WebConfig configuration component:
@Bean
public FilterRegistrationBean filterRegistrationBean()
{
    FilterRegistrationBean registration = new FilterRegistrationBean();
    registration.setFilter(new HiddenHttpMethodFilter());
    registration.setDispatcherTypes(EnumSet.allOf(DispatcherType.class));
    registration.addUrlPatterns("/*");
    return registration;
}
Not only the request methods should match the API actions, but response codes too. For example, one should get OK (200) when retrieving the resource, CREATED (201) when creating, NO CONTENT (204) after resource has been successfully deleted or updated or ACCEPTED (202) when resource has been marked for deletion or update, but the operation may be not completed yet, in this case the response body often contains an URI where client can check if the operation is completed already. Check out section 6 of the RFC 7231 for more information on HTTP response codes, it has a very good description for every code and helps a lot when implementing an API for the first time.

Dealing with exceptional situations

It is not always possible for the API to return a valid response, specially if request is incorrect. The most obvious case is if client requests nonexistent resource and API responds with NOT FOUND (404) code. The question is how to override the response code that is already bound to controller method in such an exceptional case. The answer is with throwing an exception, adding exception handler and binding an error response code to it. For example, following code in ShopController deals with retrieving nonexistent shops:
Shop shop = shopRepository.findOne(shopId);
if (shop == null) {
    throw new NoSuchElementException("Shop with id " + shopId + " does not exist");
}
Since ProductController also throws NoSuchElementException when the requested product cannot be found, we’ve put the exception handler to an abstract ApiController class and made both controllers extend it:
public abstract class ApiController {
    @ExceptionHandler(value = NoSuchElementException.class)
    @ResponseStatus(HttpStatus.NOT_FOUND)
    final String handleNotFound(HttpServletRequest req, NoSuchElementException e) {
	return e.getMessage();
    }
}
Another example is if, let’s say, API does not allow to delete shops that have products. In this case it makes sense to throw IllegalStateException whenever DELETE request is issued and register an exception handler that will map IllegalStateException to Conflict (409) response code.

Links

The proper use of HTTP verbs brings API to the second level in the maturity model and in order to make the last step and get to the third level we need to add hypermedia to the entities. Spring Hateoas is the library that helps us in this case:
<dependency>
    <groupId>org.springframework.hateoas</groupId>
    <artifactId>spring-hateoas</artifactId>
</dependency>
Spring Hateoas contains a generic Resource class, that can wrap any entity (e.g. Shop) and provides the “add links” functionality, incl. the link to the entity itself, i.e. self ref, which is useful when you have a multiple URLs that load the same entity – with self ref you can provide the canonical URL for that entity. It is a good idea to place the code converting an entity to the Resource object in a separate class that implements ResourceAssembler interface from Spring Hateoas, though you may still prepare the resource directly in the controller if you’re not going to have a lot of links or reuse the conversion code in other methods. Let’s look at the toResource method of ShopResourceAssempler that converts Shop to Resource in our example application:
@Override
public Resource<Shop> toResource(Shop shop) {
    Resource<Shop> resource = new Resource<Shop>(shop);
    resource.add(linkTo(methodOn(ProductController.class).listProducts(shop.identifier))
       .withRel("products"));
    resource.add(linkTo(methodOn(ShopController.class).getShop(shop.identifier)).withSelfRel());
    return resource;
}
In this method two links are added to the shop resource: canonical shop URL and a link to shop’s products. The client does not need, neither it should, know the URI structure to retrieve the products for the given shop (/products?shop_id={shop_id}), instead it should follow the link specified at that shop resource. This allows API provider to modify the URI structure without breaking client’s logic. For example, in the future we can decide that we want products to be always encapsulated by the shop and as a result the URI for the shop’s products will become /shops/{shop_id}/products. Since we use hypermedia to link shops to their products there will be no need to support old URIs, the link within Shop resource will be updated automatically and clients that have followed our instructions regarding hypermedia won’t be affected by the change. After the ResourceAssembler is ready, we update all the methods in ShopController that returned Shop to return Resource instead:
@RequestMapping(method = RequestMethod.GET, value = "/{shopId}")
@ResponseStatus(HttpStatus.OK)
public Resource<Shop> getShop(@PathVariable long shopId) {
    Shop shop = shopRepository.findOne(shopId);
    if (shop == null) {
        throw new NoSuchElementException("Shop with id " + shopId + " does not exist");
    }
    return shopResourceAssembler.toResource(shop);
}

@RequestMapping(method = RequestMethod.GET, value = "")
@ResponseStatus(HttpStatus.OK)
public Resources<Resource<Shop>> listShops() {
    Resources<Resource<Shop>> resources = shopResourceAssembler.toResources(shopRepository.findAll());
    resources.add(linkTo(methodOn(getClass()).listShops()).withSelfRel());
    return resources;
}
We also update createShop method to include link to the created resource in the HTTP response headers. That’s it! Now we’ve got to the 3rd level and we can finally run the application and play around with it. There are different tools you may use to emulate API client, I use Chrome extension, called Postman: it’s free, it has convenient UI, it allows you to save requests and organize them into collections and also synchronize those collections across multiple machines.

Monitoring

That’s great that we can now start our API and serve client requests, but how do we confirm that API application is running smoothly, that it does not break when given the input we did not expect? Clearly we have to monitor API health at least its most crucial characteristics, which are response time and response code. If either response time gets too high or there are too many 500 response codes API is not as stable as we expected and requires fixing. In our real API we save those performance stats to Elasticsearch cluster and then use Kibana to visualize the summary, e.g.we have max/min/average response time histograms, status code distribution pie chart, etc. This topic deserves its own post, so I won’t cover it here – in the example application we simply output those characteristics to the log. How do we calculate the response time? Clearly response time is the time the application has spent on executing request handler method and the usual way to compute method execution time is to take time snapshot in the beginning and in the end of the method body and then compute the difference:
void handler {
    Instant start = Instant.now();
    // method body
    Duration duration = Duration.between(start, Instant.now());
}
But that means we need to modify every handler method which is both tedious and error prone, as we may forget to add this code to some of the handlers. Fortunately Spring provides a way to add an interceptor, which is called before and after any handler of the request is called. First we implement the interceptor:
public class ApiMonitor implements HandlerInterceptor {

    private final Logger log = Logger.getLogger(getClass());

    private final ConcurrentMap<HttpServletRequest, Instant> requestTime = new ConcurrentHashMap<>();

    @Override
    public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
	requestTime.put(request, Instant.now());
	return true;
    }

    @Override
    public void postHandle(HttpServletRequest request, HttpServletResponse response, Object handler, ModelAndView modelAndView) throws Exception {
    }

    @Override
    public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex) throws Exception {
	Duration duration = Duration.between(requestTime.remove(request), Instant.now());
	log.info("Response took " + duration.toMillis() + "ms, response code: " + response.getStatus());
    }
}

Then we register it in WebConfig:
@Override
public void addInterceptors(InterceptorRegistry registry) {
    registry.addInterceptor(new ApiMonitor());
}
Now each request is accompanied by a log message, which you may’ve seen already if you executed requests to example application. Reacting on running application failures is good, but preventing those is even better. That’s where testing comes in place, in fact we use tests not only to assure quality but also to generate API documentation with Spring REST Docs. This is also quite big and isolated topic hence deserves to be in a separate post. If you would like to get more information on API design topic I would recommend subscribing to Spring Developer channel on Youtube, there you can find several good lectures by Josh Long and Ben Hale, these are definitely must-see, in fact our API design is pretty much inspired by those videos. If you prefer reading to watching videos a good place to start is tutorials and documentation at spring.io. Also, as already mentioned, I will cover related topics like “Monitoring API health with ElasticSearch and Kibana”, “Securing an API” and “API Gateway: facade of distributed systems” in the upcoming posts, stay tuned.
Category:
  Tech Corner
this post was shared 0 times
 000
About

 Tanja Hackenberg

  (18 articles)

Leave a Reply

Your email address will not be published.