Sign in Subscribe
event-driven

Documentation for your Event-Driven API

thepylot

• 5 min read
Documentation for your Event-Driven API
Documentation for your Event-Driven API

In this article, we will dive into the topic of event-driven APIs and how they can be documented using the tool called AsyncAPI. As companies increasingly rely on event-driven services for better performance, it is essential to understand how this mechanism works and how documentation using AsyncAPI can improve communication between services.

Event-driven services use a publisher-consumer model, which enables services to communicate through message brokers such as RabbitMQ or Redis. This model allows for asynchronous communication between services, meaning that a service can send a message to another service and continue with its own processing without waiting for a response. This is a critical feature in modern applications, where scalability and performance are crucial.

AsyncAPI offers a comprehensive and creative way of documenting event-driven APIs. It allows developers to describe the structure of messages exchanged between services, add metadata to messages, and define message channels. AsyncAPI provides a way for developers to understand the communication between services, which can be helpful in debugging applications and in onboarding new team members.

Real Case Scenario

We had to create a service that handles slug changes on specific entities, such as products or categories. The standard REST API communication cannot handle millions of requests and process data to update or create a new record in the database. This would overload the server with tons of requests, causing the system to crash at a certain point.

To solve this problem, we publish a new message (data) in RabbitMQ each time a product or category changes its slug. Related consumers will then receive these messages from queues and process them further.

We also needed a tool to document technical details of the internal structure, allowing other teams to understand the communication between services, such as queue names, routing keys, message bodies, and so on.

Implementation

As we have a lot of microservices, each service needs to document its own APIs, which should include available servers based on environment and channel details. Additionally, since versions change frequently, the documentation needs to be deployed continuously to Gitlab pages. Gitlab pipelines can be used to deploy changes automatically when a branch is merged to the master.

To render the YAML configuration, you can use the HTML generator of AsyncAPI. It will generate all documentation files, which should be located inside the docs/ directory. Then, create another folder inside named source/, which will hold the YAML files of AsyncAPI.

Feel free to create another directory inside source/ to separate entity-level topics. For example, we created two additional directories for product/ and category/.

Finally, create a new YAML file 0.0.1.yml inside the directories created earlier to separate versions of the API.

The file structure should look like the following:

docs/
  source/
    product/
      0.0.1.yml
    category/
      0.0.1.yml

Unfortunately, I am not allowed to expose the real docs that we used but the example from AsyncAPI Studio will be pretty enough to understand the use cases. Copy the example AsyncAPI configuration below and paste it inside the YAML files:

asyncapi: '2.4.0'
info:
  title: Streetlights Kafka API
  version: '1.0.0'
  description: |
    The Smartylighting Streetlights API allows you to remotely manage the city lights.

    ### Check out its awesome features:

    * Turn a specific streetlight on/off 🌃
    * Dim a specific streetlight 😎
    * Receive real-time information about environmental lighting conditions 📈
  license:
    name: Apache 2.0
    url: https://www.apache.org/licenses/LICENSE-2.0

servers:
  test:
    url: test.mykafkacluster.org:8092
    protocol: kafka-secure
    description: Test broker
    security:
      - saslScram: []

defaultContentType: application/json

channels:
  smartylighting.streetlights.1.0.event.{streetlightId}.lighting.measured:
    description: The topic on which measured values may be produced and consumed.
    parameters:
      streetlightId:
        $ref: '#/components/parameters/streetlightId'
    publish:
      summary: Inform about environmental lighting conditions of a particular streetlight.
      operationId: receiveLightMeasurement
      traits:
        - $ref: '#/components/operationTraits/kafka'
      message:
        $ref: '#/components/messages/lightMeasured'

  smartylighting.streetlights.1.0.action.{streetlightId}.turn.on:
    parameters:
      streetlightId:
        $ref: '#/components/parameters/streetlightId'
    subscribe:
      operationId: turnOn
      traits:
        - $ref: '#/components/operationTraits/kafka'
      message:
        $ref: '#/components/messages/turnOnOff'

  smartylighting.streetlights.1.0.action.{streetlightId}.turn.off:
    parameters:
      streetlightId:
        $ref: '#/components/parameters/streetlightId'
    subscribe:
      operationId: turnOff
      traits:
        - $ref: '#/components/operationTraits/kafka'
      message:
        $ref: '#/components/messages/turnOnOff'

  smartylighting.streetlights.1.0.action.{streetlightId}.dim:
    parameters:
      streetlightId:
        $ref: '#/components/parameters/streetlightId'
    subscribe:
      operationId: dimLight
      traits:
        - $ref: '#/components/operationTraits/kafka'
      message:
        $ref: '#/components/messages/dimLight'

components:
  messages:
    lightMeasured:
      name: lightMeasured
      title: Light measured
      summary: Inform about environmental lighting conditions of a particular streetlight.
      contentType: application/json
      traits:
        - $ref: '#/components/messageTraits/commonHeaders'
      payload:
        $ref: "#/components/schemas/lightMeasuredPayload"
    turnOnOff:
      name: turnOnOff
      title: Turn on/off
      summary: Command a particular streetlight to turn the lights on or off.
      traits:
        - $ref: '#/components/messageTraits/commonHeaders'
      payload:
        $ref: "#/components/schemas/turnOnOffPayload"
    dimLight:
      name: dimLight
      title: Dim light
      summary: Command a particular streetlight to dim the lights.
      traits:
        - $ref: '#/components/messageTraits/commonHeaders'
      payload:
        $ref: "#/components/schemas/dimLightPayload"

  schemas:
    lightMeasuredPayload:
      type: object
      properties:
        lumens:
          type: integer
          minimum: 0
          description: Light intensity measured in lumens.
        sentAt:
          $ref: "#/components/schemas/sentAt"
    turnOnOffPayload:
      type: object
      properties:
        command:
          type: string
          enum:
            - on
            - off
          description: Whether to turn on or off the light.
        sentAt:
          $ref: "#/components/schemas/sentAt"
    dimLightPayload:
      type: object
      properties:
        percentage:
          type: integer
          description: Percentage to which the light should be dimmed to.
          minimum: 0
          maximum: 100
        sentAt:
          $ref: "#/components/schemas/sentAt"
    sentAt:
      type: string
      format: date-time
      description: Date and time when the message was sent.

  securitySchemes:
    saslScram:
      type: scramSha256
      description: Provide your username and password for SASL/SCRAM authentication

  parameters:
    streetlightId:
      description: The ID of the streetlight.
      schema:
        type: string

  messageTraits:
    commonHeaders:
      headers:
        type: object
        properties:
          my-app-header:
            type: integer
            minimum: 0
            maximum: 100

  operationTraits:
    kafka:
      bindings:
        kafka:
          clientId: my-app-id

Then, install the HTML generator module for AsyncAPI:

npm install -g @asyncapi/generator

Once installation is completed, create a new script file at the root level of your project. It will automatically detect YAML files and render them using the AsyncAPI HTML generator to create all required files.

generate_doc.sh

for f in $(find ./docs/source/ -name '*.yml');
    do file=$(echo $f | sed 's/.yml//;s/docs//;s/source//'); ag --force-write $f @asyncapi/html-template -o ./docs/generated/$file/;
done

Run the script ./generate_doc.sh and it will create another directory named generated/ with all static files of your rendered documentation.

Deployment

Let's assume that there are three environments: dev, pre-prod, and prod. At the very last stage, we need to deploy the documentation alongside the release.

To deploy the AsyncAPI docs via GitLab CI/CD, you can follow these steps:

  1. Create a folder named .gitlab/ci inside your project.
  2. Add a file named pages.yml to this folder. This file will include CI/CD of deployment.
  3. Insert the following code into the pages.yml file:
pages:
  stage: docs
  image: asyncapi/generator:1.9.5
  script:
    - cp -a ./docs/generated/. ./public/
  artifacts:
    paths:
      - public
  only:
    refs:
      - master
  needs:
    - pre-prod-deploy

This code will copy the generated docs (static files of generated docs, as mentioned in previous steps) to a folder named public. The only section specifies that the deployment should only happen when the master branch is updated. Finally, the needs section specifies that this deployment should happen after the pre-production deployment is completed.

Conclusion

Deploying the documentation is an essential step in the process of using AsyncAPI. By following the steps outlined above, you can easily deploy your documentation alongside your release. By continuously deploying your documentation, you ensure that it always reflects the latest version of your API, making it easier for other teams to understand the communication between services.

Overall, event-driven APIs and AsyncAPI offer exciting opportunities for developers to build modern applications that are scalable, performant, and well-documented.