Integrating AWS API Gateway with SQS

Integrating AWS API Gateway with SQS

Amazon Simple Queue Service

It is a fully managed message queueing service that allows you to decouple your applications. It helps you improve your applications’ performance and user experience by allowing you to communicate asynchronously between client and server.

A queue is like a buffer. You can put messages into a queue, and you can retrieve messages from a queue. Message queues operate so that any given message is only consumed by one receiver, although multiple receivers can be connected to the queue.

API Gateway

An API Gateway acts as a mediator between client applications and backend services in microservices architecture. The API gateway controls requests and responses by managing the traffic of APIs while enforcing security policies.

In some cases, you might want to process the requests that are coming into API Gateway in an asynchronous way. That could be happening for many reasons but usually you might want to do this if you want to avoid overwhelming some downstream integration.

Implementation Steps:

Step 1: Create an SQS queue

Go to the AWS Management Console and find SQS (Simple Queue Service).

Click on “Create queue”

For Type choose “Standard”

Click Create Queue button

Step2: Create an IAM policy

We need to create an IAM policy and a Role, this will define the permissions, we are going to need this for our API Gateway to send messages to the Queue.

In the AWS Management Console find the IAM service

Go to Policies and then click on “Create Policy “

Switch to the JSON tab

Add the following policy and make sure you replace SQS_QUEUE_ARN with the ARN of your SQS queue.

Step3: Create an IAM role

In the IAM console click on Roles and then on “Create role”. For Trusted Entity Type choose API Gateway from the dropdown called “Use cases for other AWS services”

Click on Next button

Click Next again

The role will be saved.

Click on the role created from the list displaying the Roles. When the page is loaded with the role details click on “Add permissions” and choose “Attach policies” from the dropdown.

Find the policy you created previously and select it then click on “Attach policies” and then copy the “ARN” of the role, you will need this later.

Step4: Create API Gateway and configure it

Go to the API Gateway service

Click on “Create API” and then on the “REST API” choose “Build”.

On the next page select “REST” and “New API”, give your API a name then click on “Create API"

On the API page click the “Actions” dropdown and select “Create Resource” and give a name to your resource

Click on the resource that you just created then on the “Actions” menu again and select “Create Method “.

On the created method select “POST” and then click on the checkmark.

You will see the integration page.

1. For integration type select “AWS Service”

2. For the AWS region select your preferred region

3. For AWS Service select “Simple Queue Service (SQS)”

4. For HTTP method select “POST “

5. For Action Type select “Use path override “

6. For Path override fill in the queue URL .It should be in the following format: ACCOUNT_ID/queuename

Example: 012345678912/myorderQueue

7. For the Execution role paste in the ARN of the Role that you created at step 3 and then click “Save"

On the page select “Integration Request” as shown in the image

Scroll down to HTTP Headers and expand it.

Click on “Add header”

for Name enter Content-Type

and for the Mapped from field enter ‘application/x-www-form-urlencoded’

and then click on the checkmark to save it

Expand Mapping Templates and for the Request body passthrough, select the option that fits your needs

Choose Add mapping template and then for Content-Type, enter application/json and choose “Create “.

For the template, enter Action=SendMessage&MessageBody=$input.body then click on “Save”.

Step5: Deploy and test the API

On the API page click on the “Actions” dropdown and select “Deploy API” then in the Deployment stage dropdown select [New Stage] , give it a name, example: dev and then click “Deploy“

Test the integration with curl from the command line, make sure to replace the following:

API_ID with the ID of your API (you can find this in the API Gateway console, go to the list of your API Gateways and in the list, there is also the ID besides the name)

REGION with your preferred region id, for example us-east-1

STAGE with the stage you created earlier (I used prod above)

RESOURCE with the name of your resource (route), I used sqs-caller above.

For the message you can write some message that you want

You will get response something like this.

You can see here under message available.

Go to SQS queue and select the queue you have created and click on Send and receive messages.

Click on the message you will see the message you send from the terminal

Now you have a working integration between API Gateway and SQS and your incoming requests is sent to SQS from where you will be able to process them in an asynchronous manner.