APIs First

Designing web service APIs can be a tricky art form. APIs serve as a virtual playbook necessary for interacting with your business domain, and more importantly, they are the contracts that bind service owners with their consumers. Therefore, getting design right early on is an important part of the service engineering we do at Squarespace.

At the same time, the code that backs API interactions is littered with boilerplate. In our Java service stack there are typically three components that make up the plumbing of our services:

• Jersey or Spring MVC-mapped RESTful resources
• JSON-mapped transport models
• Asynchronous-capable network clients/SDKs

As our service count grew, it became increasingly clear that we needed to focus our efforts on the specification of our APIs and their underlying business logic without being bothered by the boilerplate plumbing, input validation, fault tolerance and client isolation. That’s why we introduced and implemented our own code generation framework based on a well known specification, the Open API Specification.

Open API (formerly known as Swagger)

Swagger has been around for some time, and is well known for the interactive documentation it produces. While that documentation is a great feature, the real power is the platform-independent specification which is used as a simple and consistent way to describe your APIs. By leveraging the flexible specification and features such as vendor extensions, we were able to create our own code-generation framework that captures our preferred patterns and code style. It also provides us with a simple way for our engineers to immediately and transparently leverage the abstractions that define the pillars of our services.

The specification also provides the added benefit of being able to generate client SDKs in various languages (Go, Python, Ruby, etc).

Doesn’t Swagger already have Java codegen?

Yes, it does. However, the code that Swagger Codegen generates didn’t fully meet our needs. In our organization, we have standardized on a service client for our Java stacks based on Reactive Java. The service client encapsulates what we feel is requisite when communicating with services over a network. These requisites include fault tolerance via circuit breakers and bulkheads, service discovery with our Consul cluster, standardized metrics, standardized logging, and distributed tracing.

We also incorporate many of the open API specification validation attributes and automatically apply input validation in our codebase. Our code generation is integrated seamlessly in our CI/CD pipeline through Gradle. The generated code doesn’t need to be checked into our Git repository.

Here is an example piece of Swagger specification and the corresponding generated Java:

/subscriptions/domains/{domain}:
    get:
      tags:
        - subscription
      summary: "GET 0 or more subscriptions for the provided domain"
      description: "This method will retrieve a list of subscriptions for the provided domain"
      operationId: getSubscriptions
      parameters:
        - name: domains
          in: path
          type: string
      responses:
        200:
          schema:
            $ref: '#/definitions/CertificateSubscriptions'

 /**
   * This method will retrieve a list of subscriptions for the provided domain
   */
  @GET
  @Path("/subscriptions/domains/{domain}")
  @Produces("application/json")
  public CertificateSubscriptions getSubscriptions(@PathParam("domain") String domains) {
    return delegate.getSubscriptions(domain);
  }

 /**
   * This method will retrieve a list of subscriptions for the provided domain
   */
  public Observable<CertificateSubscriptions> getSubscriptions(String domain) {
    HttpRequestBuilder<ByteBuf> builder = getSubscriptions.requestBuilder()
      .withRequestProperty("domain", UrlEscapingUtils.escapePath(domain));

    return client
        .buildWithJsonDeserialization(builder, new TypeLiteral<CertificateSubscriptions>(){})
        .observe();
  }

@Generated("com.squarespace.service.codegen.Generator")
public class CertificateSubscriptions {
  @JsonProperty
  private int offset;

  @JsonProperty
  private int limit;

  @JsonProperty
  private long total;

  @JsonProperty
  private List<CertificateSubscription> results;

  public int getOffset() {
    return offset;
  }

  public int getLimit() {
    return limit;
  }

  public long getTotal() {
    return total;
  }

  public List<CertificateSubscription> getResults() {
    return results;
  }

Is it working out?

We have been generating our service clients, transport models, and resources for close to three years now. While the road has been a bit rocky, overall the Core Services team feels that codegen has proven it is a viable and effective way to accelerate the development process and treat APIs as first-class design artifacts.

Some challenges that we are still working with include:

• Fully supporting the entire Swagger spec and agreeing on our interpretations of what Java code should be generated for each facet of the spec.
• Providing valuable feedback for errors and warnings during the code generation phase of the build.

What’s next?

One of the main advantages of having our own custom code generation framework is the amount of control we have over the future roadmap of the framework itself. We plan on adding some neat features in the future including:

• Allowing users of the framework to choose between various Gang of Four patterns like proxy or command.
• Integrating and providing support for gRPC both as a first class generation artifact and as a reverse HTTP proxy.
• Adding support for reactive endpoints coming in the Spring Web Flux project.

We are hiring!

The Core Services team is always on the lookout for talent that can help us grow and improve our service framework. Click here to apply!