A neat feature of Amazon Web Service's API Gateway service is that it can integrate directly with other AWS services. The most common use of API Gateway is to integrate directly with a Lambda function, typically to perform an action like update a DynamoDB table or send a message to an SQS queue. But, sometimes it is not actually necessary to use a Lambda function at all - by taking advantage of API Gateway's AWS service integration, we can avoid this intermediary step altogether and build a much more efficient and resilient architecture.
I recently had a need to provision a simple API endpoint that would accept a JSON payload and store the data to be processed later. Typically I would look to use API Gateway backed by a basic Lambda function to accept the JSON data and store it in an SQS queue. Instead, for this project I opted to use the API Gateway AWS service integration with the SQS queue.
By using the AWS service integration, we benefit in a number of ways:
There are a number of blog posts that do an excellent job of describing how to set up this API Gateway to SQS architecture, but none that explain how to do the same using CloudFormation, Amazon's "Infrastructure as Code" solution.
After a bit of trial and error, I managed to come up with the below Cloudformation template which does the following:
Please feel free to use the below template or modify it to suit your own needs.
Description: API Gateway integration with SQS Outputs: ApiEndpoint: Description: Endpoint for this stage of the api Value: !Join - '' - - https:// - !Ref 'APIGateway' - .execute-api. - !Ref 'AWS::Region' - .amazonaws.com/ - prod QueueArnSQS: Description: ARN of SQS Queue Value: !GetAtt 'DestQueue.Arn' Parameters: queueName: Description: The name of the SQS queue to create. Type: String Resources: APIGateway: Properties: Description: API Endpoint to receive JSON payloads and queue in SQS Name: APIGateway Type: AWS::ApiGateway::RestApi APIGatewayRole: Properties: AssumeRolePolicyDocument: Statement: - Action: - sts:AssumeRole Effect: Allow Principal: Service: - apigateway.amazonaws.com Version: '2012-10-17' Path: / Policies: - PolicyDocument: Statement: - Action: sqs:SendMessage Effect: Allow Resource: !GetAtt 'DestQueue.Arn' - Action: - logs:CreateLogGroup - logs:CreateLogStream - logs:PutLogEvents Effect: Allow Resource: '*' Version: '2012-10-17' PolicyName: apig-sqs-send-msg-policy RoleName: apig-sqs-send-msg-role Type: AWS::IAM::Role DestQueue: Properties: DelaySeconds: 0 MaximumMessageSize: 262144 MessageRetentionPeriod: 1209600 QueueName: !Ref 'queueName' ReceiveMessageWaitTimeSeconds: 0 VisibilityTimeout: 30 Type: AWS::SQS::Queue PolicySQS: Properties: PolicyDocument: Statement: - Action: SQS:* Effect: Allow Principal: '*' Resource: !GetAtt 'DestQueue.Arn' Sid: Sid1517269801413 Version: '2012-10-17' Queues: - !Ref 'DestQueue' Type: AWS::SQS::QueuePolicy PostMethod: Properties: AuthorizationType: NONE HttpMethod: POST Integration: Credentials: !GetAtt 'APIGatewayRole.Arn' IntegrationHttpMethod: POST IntegrationResponses: - StatusCode: '200' PassthroughBehavior: NEVER RequestParameters: integration.request.header.Content-Type: '''application/x-www-form-urlencoded''' RequestTemplates: application/json: Action=SendMessage&MessageBody=$input.body Type: AWS Uri: !Join - '' - - 'arn:aws:apigateway:' - !Ref 'AWS::Region' - :sqs:path/ - !Ref 'AWS::AccountId' - / - !Ref 'queueName' MethodResponses: - ResponseModels: application/json: Empty StatusCode: '200' ResourceId: !Ref 'enqueueResource' RestApiId: !Ref 'APIGateway' Type: AWS::ApiGateway::Method enqueueResource: Properties: ParentId: !Ref 'v1Resource' PathPart: enqueue RestApiId: !Ref 'APIGateway' Type: AWS::ApiGateway::Resource prodDeployment: DependsOn: PostMethod Properties: RestApiId: !Ref 'APIGateway' Type: AWS::ApiGateway::Deployment prodStage: Properties: DeploymentId: !Ref 'prodDeployment' RestApiId: !Ref 'APIGateway' StageName: prod Type: AWS::ApiGateway::Stage v1Resource: Properties: ParentId: !GetAtt 'APIGateway.RootResourceId' PathPart: v1 RestApiId: !Ref 'APIGateway' Type: AWS::ApiGateway::Resource