Seamless API Management for Services in Service Catalog

8 min readJun 28, 2021

Most of the digital businesses tends to expose their services as APIs to enable the interactions between different applications. From a consumer point of view, when implementing services, it is important to have a way to be able to adapt for evolution. It can be either updates or retirements or bug fixes or new version release. A solution for managing the changes inherited with services is crucial when exposing it to consumers.

WSO2 API Manager 4.0.0 caters this requirement by providing a sophisticated way to manage services via Service Catalog. Service Developers can deploy their services seamlessly to API Manager which leads the API Developer to discover the service and expose it as managed APIs.

In this blog, we will see about

An introduction to the Service Catalog in API Manager
How services can be pushed to Service Catalog
Available options when pushing the services to Service Catalog
How APIs can be created out of the services
Managing the service versioning and updates

Deploy the services to Service Catalog

REST APIs are introduced in API Manager 4.0.0 to push the services from a different environment to API Manager. The back-end developers can seamlessly push their services to Service Catalog. Currently API Manager supports OAS 2.0 and 3.0 based REST Services and event driven services of types Server Send Events, Web Sockets, WebHooks and WebSub in Service Catalog.

Do you have an OAS definition for your REST service?

Do you have an Async API Definition for your event driven service?

If yes, you can simply push the services to Service Catalog. Let’s see how this can be done:

Deploy single service to Service Catalog

The following REST API can be used to push single service to Service Catalog in API Manager:

POST https://wso2.apim.com/api/am/service-catalog/v0/services

Let’s assume we have a OAS 3.0 based REST Service. We need to provide the following attributes in addition to the service definition (in YAML format) when pushing the service to Service Catalog:

Sample curl command for this operation is given below:

curl POST 'https://wso2.apim.com/api/am/service-catalog/v0/services'
--header 'Authorization: Bearer YWRtaW46YWRtaW4=' \
--form 'serviceMetadata="    {
            \"name\":\"REST Service\",
            \"description\": \"This is a sample REST Service\",
            \"version\": \"1.0.0\",
            \"serviceKey\": \"REST-Service-1.0.0\",
            \"serviceUrl\": \"http://am.back-end/com/oas\",
            \"definitionType\": \"OAS3\",
            \"securityType\": \"BASIC\",
            \"mutualSSLEnabled\": false
        }";type=application/json' \
--form 'definitionFile=@"/user/~/definition.yaml"'

You will get a successful response in the following format:

{
  "id": "7b3d976a-620a-4435-9649-e806f30dc7cf",
  "name": "REST Service",
  "description": "This is a sample REST Service",
  "version": "1.0.0",
  "serviceKey": "REST-Service-1.0.0",
  "serviceUrl": "http://am.back-end/com/oas",
  "definitionType": "OAS3",
  "securityType": "BASIC",
  "mutualSSLEnabled": false,
  "usage": 0,
  "createdTime": "2021-06-27T13:57:16.229",
  "lastUpdatedTime": "2021--06-27T13:57:16.229",
  "md5": "36583a6a249b410e7fc4f892029709cac09763ddb230e1a829d5f9134d1abd07",
  "definitionUrl": ""
}

Bulk import of services to Service Catalog

You can also execute bulk import of services to Service Catalog using the POST /import-services REST API. A CI/CD process can be configured for the service development to push the services periodically to Service Catalog. For example, minor bug fixes and improvements can be cumulated and pushed to Service Catalog.

Following REST API can be used to bulk import the services into Service Catalog:

POST https://wso2.apim.com/api/am/service-catalog/v0/services/import

When bulk importing services, each service should have separate directory which should contain:

metadata.yaml → The file that contains the meta data related to the service. Following meta data should be present in the file in order to deploy the service in Service Catalog (similar to the information we provided when deploying a single service).

key: ${optional-unique-key}
name: ${name-of-the-service}
description: ${description-of-the-service}
version: ${service-version}
serviceUrl: ${service-back-end-url}
definitionType: ${definition-type}
securityType: ${security-type}
mutualSSLEnabled: ${true-or-false}

definition.yaml → Definition of the service. It can be OAS 3 or OAS 2 definition if it’s a REST Service and it can be an Async API definition if it’s a event driven service.

These directories should be zipped and sent as payload. The directory structure of the payload zip should look like:

<zip_file>. 
├── service1 
│   ├── definition.yaml 
│   └── metadata.yaml 
└── service2     
    ├── definition.yaml     
    └── metadata.yaml

What if a service already exists in Service Catalog?

There can be situations, where

we need to update an existing service while deploying some other new services
we might need to prevent further changes to the services that were already pushed to Service Catalog
there could be integration process configured to push services to Service Catalog from different development servers. How to prevent concurrent requests to import or modify a service?

All this complex scenarios are smoothly handled in Service Catalog side. We will see one by one.

Force update a service

When bulk importing services, you can enable force update for the services that are already pushed in the Service Catalog. You can simply do it by setting a query parameter when performing bulk import as shown in the following cURL command:

curl -k 
   -H "Authorization: Bearer ${access-token}"             
   -F "file=@services.zip" 
   "https://wso2.apim.com/api/am/service-catalog/v0/services/import?overwrite=true"

By default, overwrite is set to false. If you try to re-import an existing service, you will get an error message saying:

Cannot update existing services when overwrite is false

Handle concurrent requests to update service

Multiple developers might be involved in developing single service. When pushing the changes to Service Catalog, there can be concurrent requests. In order to handle such cases, we have been provided with another parameter named as verifier.

When sending import request to Service Catalog, developer can send an additional parameter called verifier in payload which consists of service key and md5 of the service. md5 value for a service is caluculated by appending the md5 hash value of the metadata file and the definition file.

If this parameter is present in the request, it will be validated against the md5 value of the service in the system and the requested party will get notified if the request failed due to unmatched verifier. ^_^

An example of verifier:

[
  {
    "key": "rest-service-1.0.0",
    "md5": "6d6ac8a03c2cd5b95fbdda7c66ef285ebcbe4da2198807c5c9aa3be7e401f6a1"
  },
  {
    "key": "sse-service-1.0.0",
    "md5": "7f5850d98866b5ce34fa1812d0e6d080ad8b7082cd9387eb80311ba75bfe776"
  }
]

Example cURL command shows the format to send verifier in the request:

curl --request POST 'https://am.wso2.com/api/am/service-catalog/v0/services/import?overwrite=true' \
--header 'Authorization: Bearer ${access-token}' \
--form 'verifier="[
  {
    \"key\": \"rest-service-1.0.0\",
    \"md5\": \"6d6ac8a03c2cd5b95fbdda7c66ef285ebcbe4da2198807c5c9aa3be7e401f6a1\"
  },
  {
    \"key\": \"sse-service-1.0.0\",
    \"md5\": \"7f5850d98866b5ce34fa1812d0e6d080ad8b7082cd9387eb80311ba75bfe776\"
  }
]"' \
--form 'file=@"/~/User/Services.zip"'

Push Integration Services from Micro Integrator

As you may already know, Micro Integrator and Streaming Integrator have been added as new components of API Manager from 4.0.0 onward. Service Catalog module has been internally integrated with Micro Integrator and Streaming Integrator. Publishing your enterprise integrations services from Micro Integrator to Service Catalog can be done simply by adding the following configuration in deployment.toml file resides in <MI-Home>/confdirectory.

[[service_catalog]] 
apim_host = "https://am.wso2.com"
enable = true 
username = "${admin-username}" 
password = "${admin-password}"

Push Async Services from Streaming Integrator

Add the following configuration to deployment.yamlfile resides in <SI-Home>/conf/server directory to publish the Async services developed in Streaming Integrator to Service Catalog.

service.catalog.configs:
  enabled: true
  hostname: am.wso2.com
  port: 443
  username: ${admin-username}
  password: ${admin-password}

It will push all your integration and async services to Service Catalog during the server start up ^_^.

Create API from Services in Service Catalog

API Manager provides intuitive user interface to browse the services available in Service Catalog. Once the services are pushed to Service Catalog, you can view it in the Services section in the Publisher Portal. (Login to Publisher Portal by visiting https://wso2.am.com/publisher)

Following screenshot shows the listing view of services in Service Catalog, in Publisher portal.

You can select the relevant service and create it as an API by providing the name, version and context (and Async Protocol if it’s an Async API). Simple.. ^_^

We just created an API using the services in the Service Catalog 😎. You can apply whatever the API Management capablity to the API and secure your service. 🙌

Managing the versions and updates of Service

Let’s say we have created an API out of a service deployed in Service Catalog. What if the service developer updates the service thereafter. They can even push changes as new version if there are drastic. 🆒 This is also smoothly handled in API Manager.

Re-import the service used in the API

If there are updates pushed to the services in Service Catalog and you have already created an API out of it, you will be able to see that the current service used by the API is outdated as shown in the screenshot below:

Fig: Re-import service definition of an API

For Async APIs, navigate to Async API Definition section and you can see a warning ⚠️ Outdated Definition if the underlying service is updated. You can re-import the service if the updates are required 👌

Create new version of API using different service version

In a traditional API Management, it will not give a good user experience if we introduce drastic or breaking service changes via an existing API. In such case, API versioning will help to deliver the changes safely without breaking the requirement of existing consumers.

Some condition applied to the services as well. If there are new versions of service added to the Service Catalog, you can easily create new version of API out of the service. The option of selecting different service version is provided when creating new API version 👇.