"Open-Sourcing Our API Gateway Custom Auth Module: Plugin-Based Authorization for AWS"

Every API needs authentication. But wiring up a custom Lambda authorizer for AWS API Gateway typically means juggling IAM roles, Lambda packaging, deployment configurations, and authorizer settings across dozens of Terraform resources. We built a module that reduces all of that to a few lines of HCL — and we're open-sourcing it.

The Problem

Setting up a custom authorizer on API Gateway involves:

• Creating the REST API and configuring endpoint types
• Writing the Lambda function and packaging the deployment ZIP
• Setting up IAM roles for both the Lambda execution and the API Gateway invocation
• Configuring the authorizer resource with identity sources and TTL caching
• Managing deployments and stage configurations
• Wiring CloudWatch log groups for debugging

That's a lot of boilerplate for something every secured API needs. And if you want to swap out your auth logic — say, moving from API key validation to JWT — you're often rewriting the Lambda handler itself.

We wanted something better: a single module where auth logic is just Python files you drop into a folder.

---

How It Works

The module deploys a REST API Gateway with a Lambda authorizer that uses a plugin chain architecture. Instead of hardcoding validation logic into the handler, the authorizer dynamically discovers and executes Python plugin files at runtime.

API Request
    │
    ▼
┌─────────────────────────────┐
│     API Gateway (REST v1)   │
│      EDGE or REGIONAL       │
└─────────────┬───────────────┘
              │
              ▼
┌─────────────────────────────┐
│     Lambda Authorizer       │
│                             │
│  ┌───────────────────────┐  │
│  │   Plugin Chain        │  │
│  │                       │  │
│  │  01_extract_token.py  │  │
│  │  02_validate_jwt.py   │  │
│  │  03_check_scope.py    │  │
│  └───────────────────────┘  │
│                             │
│  ┌───────────────────────┐  │
│  │  IAM Policy Generator │  │
│  │  Allow / Deny         │  │
│  └───────────────────────┘  │
└─────────────────────────────┘
              │
              ▼
        Your Backend
     (Lambda, EC2, etc.)

Request flow:

1. API Gateway receives a request and forwards it to the Lambda authorizer
2. The handler builds an auth context from the event (token or full request details)
3. Plugins execute sequentially in alphabetical order
4. Each plugin can pass, enrich the context, or deny the request
5. The handler generates an IAM policy (Allow/Deny) and returns it to API Gateway
6. API Gateway caches the result based on your configured TTL

---

The Plugin System

This is the core idea. Each plugin is a standalone Python file with a single validate() function:

# plugins/01_validate_jwt.py

import jwt
import os

def validate(auth_context):
    """Validate JWT token and extract claims."""
    token = auth_context.get("token", "")

    if token.startswith("Bearer "):
        token = token[7:]

    secret = os.environ.get("JWT_SECRET")
    issuer = os.environ.get("JWT_ISSUER")

    decoded = jwt.decode(token, secret, algorithms=["HS256"],
                         issuer=issuer)

    return {
        "principal_id": decoded["sub"],
        "context": {
            "jwt_sub": decoded["sub"],
            "jwt_iss": decoded.get("iss", ""),
        }
    }

Plugin rules are simple:

• Files are loaded alphabetically — prefix with 01_, 02_, etc. to control order
• Files starting with _ are skipped (use them for shared utilities)
• Return True to pass without adding context
• Return a dict with principal_id and context to enrich the response
• Raise an exception to deny the request immediately
• Later plugins can override principal_id or add additional context variables

This chain-of-responsibility pattern means you can compose auth logic from reusable pieces. Need JWT validation plus rate limiting plus audit logging? That's three plugin files, each doing one thing well.

---

Two Authorizer Types

The module supports both API Gateway authorizer types:

TOKEN Authorizer

Extracts a single token from a header. Fast and cacheable.

module "api_auth" {
  source = "github.com/AIOpsCrew/api-gateway-custom-auth"

  api_name             = "my-api"
  lambda_function_name = "my-api-authorizer"
  authorizer_type      = "TOKEN"
  identity_source      = "method.request.header.Authorization"
  stage_name           = "prod"
}

Your plugins receive:

auth_context = {
    "type": "TOKEN",
    "token": "Bearer eyJhbG...",
    "method_arn": "arn:aws:execute-api:..."
}

REQUEST Authorizer

Passes full request context for complex validation scenarios — multi-header checks, query parameter validation, path-based rules.

module "api_auth" {
  source = "github.com/AIOpsCrew/api-gateway-custom-auth"

  api_name             = "my-api"
  lambda_function_name = "my-api-authorizer"
  authorizer_type      = "REQUEST"
  stage_name           = "prod"
}

Your plugins receive everything:

auth_context = {
    "type": "REQUEST",
    "headers": {"Authorization": "Bearer ...", "X-Api-Key": "..."},
    "query_string_parameters": {"tenant": "acme"},
    "path_parameters": {"version": "v2"},
    "stage_variables": {"environment": "prod"},
    "request_context": { ... },
    "method_arn": "arn:aws:execute-api:..."
}

---

Zero External Build Tooling

One design goal we cared about: no Docker, Makefile, or shell scripts in the deployment pipeline. Terraform handles everything.

When you provide a plugins_directory, the module:

1. Copies the core authorizer code (handler, plugin loader, policy generator) into a .build/ directory
2. Copies your custom plugin files into .build/authorizer/plugins/
3. Zips the result using Terraform's archive provider
4. Deploys the ZIP to Lambda

module "api_auth" {
  source = "github.com/AIOpsCrew/api-gateway-custom-auth"

  api_name             = "my-api"
  lambda_function_name = "my-api-authorizer"
  plugins_directory    = "${path.module}/my_plugins"
  stage_name           = "prod"

  lambda_environment_variables = {
    JWT_SECRET = "use-ssm-or-secrets-manager-in-production"
    JWT_ISSUER = "https://auth.example.com"
  }
}

If you don't specify a plugins_directory, the module deploys with a built-in example validator — useful for testing and getting familiar with the plugin interface.

---

Caching and Performance

API Gateway authorizer caching can dramatically reduce Lambda invocations on high-traffic APIs. The module exposes this with a single variable:

authorizer_ttl_seconds = 300  # Cache auth results for 5 minutes (default)

For a TOKEN authorizer, API Gateway caches the policy response keyed by the token value. For REQUEST authorizers, the cache key is derived from the identity sources you configure.

At 1,000 requests/minute with a 5-minute TTL, you go from 1,000 Lambda invocations per minute to roughly 1 — per unique token. That's a significant cost and latency reduction.

---

Module Configuration

The module exposes 24 input variables covering API Gateway, Lambda, plugins, deployment, and logging. Here are the most commonly used:

Variable	Default	Description
api_name	—	Name for the REST API
authorizer_type	"TOKEN"	TOKEN or REQUEST
plugins_directory	""	Path to custom plugin files
lambda_function_name	—	Name for the authorizer Lambda
lambda_runtime	"python3.13"	Python runtime version
lambda_memory_size	128	Lambda memory (128-10240 MB)
lambda_timeout	10	Lambda timeout in seconds
authorizer_ttl_seconds	300	Auth result cache TTL
stage_name	"prod"	API Gateway stage name
endpoint_type	"REGIONAL"	EDGE, REGIONAL, or PRIVATE
lambda_layers	[]	Lambda layers (e.g., PyJWT)
lambda_vpc_config	null	Optional VPC configuration
tags	{}	Tags applied to all resources

And 11 outputs for wiring the API Gateway and authorizer into your downstream resources:

# Use these to attach your own routes and integrations
module.api_auth.rest_api_id
module.api_auth.rest_api_root_resource_id
module.api_auth.authorizer_id
module.api_auth.stage_invoke_url
module.api_auth.lambda_function_arn
module.api_auth.lambda_execution_role_arn

---

Real-World Example: Adding Routes Behind the Authorizer

The module gives you the API Gateway and authorizer. You add your own resources and methods:

module "api_auth" {
  source = "github.com/AIOpsCrew/api-gateway-custom-auth"

  api_name             = "orders-api"
  lambda_function_name = "orders-authorizer"
  plugins_directory    = "${path.module}/auth_plugins"
  stage_name           = "prod"

  # Defer deployment until methods are created
  create_deployment       = true
  deployment_dependencies = [
    aws_api_gateway_integration.orders_get.id,
    aws_api_gateway_integration.orders_post.id,
  ]
}

# /orders resource
resource "aws_api_gateway_resource" "orders" {
  rest_api_id = module.api_auth.rest_api_id
  parent_id   = module.api_auth.rest_api_root_resource_id
  path_part   = "orders"
}

# GET /orders — protected by custom authorizer
resource "aws_api_gateway_method" "orders_get" {
  rest_api_id   = module.api_auth.rest_api_id
  resource_id   = aws_api_gateway_resource.orders.id
  http_method   = "GET"
  authorization = "CUSTOM"
  authorizer_id = module.api_auth.authorizer_id
}

# Wire to your backend Lambda
resource "aws_api_gateway_integration" "orders_get" {
  rest_api_id             = module.api_auth.rest_api_id
  resource_id             = aws_api_gateway_resource.orders.id
  http_method             = aws_api_gateway_method.orders_get.http_method
  integration_http_method = "POST"
  type                    = "AWS_PROXY"
  uri                     = aws_lambda_function.orders_handler.invoke_arn
}

The deployment_dependencies variable ensures the API Gateway deployment happens after your methods and integrations are created — avoiding the common race condition where the deployment snapshot is empty.

---

Writing Your Own Plugins

Creating a plugin takes minutes. Here's a template:

# plugins/01_my_validator.py

def validate(auth_context):
    """
    Validate the request.

    Args:
        auth_context: Dict with request details.
            TOKEN type: {type, token, method_arn}
            REQUEST type: {type, headers, query_string_parameters, ...}

    Returns:
        True - validation passed, no context to add
        dict - {"principal_id": "...", "context": {...}}

    Raises:
        Exception - request denied (returns 401)
    """
    token = auth_context.get("token", "")

    if not token:
        raise ValueError("Missing token")

    # Your validation logic here
    # Call a database, validate a JWT, check an API key...

    return {
        "principal_id": "user-123",
        "context": {
            "validated_by": "my_validator",
            "user_tier": "premium"
        }
    }

Context variables you return are passed to your backend Lambda as event["requestContext"]["authorizer"]["user_tier"] — no extra API calls needed to look up user details downstream.

Need external libraries like PyJWT? Package them as a Lambda layer:

lambda_layers = [
  aws_lambda_layer_version.pyjwt.arn
]

---

VPC Support

If your authorizer needs to reach private resources — a database for API key lookups, an internal identity provider, or a Redis cache — the module supports VPC configuration:

lambda_vpc_config = {
  subnet_ids         = ["subnet-abc123", "subnet-def456"]
  security_group_ids = ["sg-789012"]
}

The module automatically attaches the AWSLambdaVPCAccessExecutionRole policy when VPC config is provided.

---

Getting Started

Prerequisites

• AWS account with appropriate permissions
• Terraform >= 1.3
• Basic familiarity with API Gateway and Lambda

Step 1: Get the Module

Want to try the module? Enter your email to get the GitHub repository URL.

Step 2: Basic Deployment

module "api_auth" {
  source = "github.com/AIOpsCrew/api-gateway-custom-auth"

  api_name             = "my-api"
  lambda_function_name = "my-api-authorizer"
  stage_name           = "dev"
}

terraform init
terraform plan
terraform apply

# Get your API URL
terraform output stage_invoke_url

Step 3: Add Custom Plugins

Create a plugins/ directory next to your Terraform config, add your .py files, and point the module at it:

plugins_directory = "${path.module}/plugins"

Run terraform apply again — the module rebuilds the Lambda package automatically.

---

Contributing

We welcome contributions! Here's how you can help:

Bug Reports: Open an issue on GitHub with your Terraform version, module version, error messages, and expected vs. actual behavior.

Feature Requests: Describe your use case and why the feature would help.

Pull Requests: Fork the repository, create a feature branch, and submit a PR with a clear description of your changes.

Share Your Plugins: Built a useful authorizer plugin? We'd love to feature community plugins in the documentation.

---

About AI Ops Crew

We build production-ready Terraform modules for AWS operations. Our mission: make infrastructure automation accessible to every engineering team.

Our Modules:

• CloudWatch AI Agent (Premium - $5/mo): AI-powered alarm investigation with real-time AWS analysis
• n8n Fargate Cluster (Free): Workflow automation platform on AWS
• Jenkins Sentinel (Free): AI-powered pipeline failure analysis
• API Gateway Custom Auth (Free): Plugin-based Lambda authorizer for API Gateway
• More coming soon...

Follow our journey as we open-source more infrastructure tools. Subscribe to our newsletter for updates.

---

Ready to simplify your API authorization? Get the module and have a plugin-based authorizer running in minutes.

---

Have questions about the module or want to share your plugins? Email us at info@aiopscrew.com or open a discussion on GitHub.