Circuit braker and ratelimiting

 

APIM -Circuit braker and rate limiting

Azure API Management (APIM) provides built-in capabilities for implementing both circuit breaker and rate limiting patterns to enhance the resilience and stability of your APIs.

Circuit Breaker Implementation:

The circuit breaker pattern in APIM helps protect your backend services from being overwhelmed or repeatedly called when they are experiencing issues.

• Configure Backend Circuit Breaker:
  • Navigate to your APIM instance in the Azure portal.
  • Select "Backends" under "APIs" in the left menu.
  • Choose the backend you want to protect and go to its "Settings."
  • Under "Circuit breaker settings," add a new rule.
  • Configure parameters like:
    • Failure count: Number of failures within a specified interval that will trip the circuit.
    • Failure interval: The time window for counting failures.
    • Failure status code range: Which HTTP status codes indicate a failure (e.g., 500-599 for server errors, or 429 for rate limiting).
    • Trip duration: How long the circuit remains open after tripping.
    • Accept Retry-After header: If enabled, APIM will honor the Retry-After header from the backend for dynamic trip duration.

Rate Limiting Implementation:

APIM offers policies to control the rate at which consumers can call your APIs, preventing abuse and ensuring fair usage.

• Using rate-limit policy:
  • This policy sets a global limit on the number of calls allowed within a specific time period for all users accessing an API or operation.
  • Apply this policy at the API or operation scope within the inbound processing section of your policy.
  • Example:

Code

        <rate-limit calls="100" renewal-period="60" />

This allows 100 calls per minute.

• Using rate-limit-by-key policy:
  • This policy allows you to apply rate limits on a per-key basis, enabling different limits for different users or applications.
  • The key can be derived from various sources using policy expressions, such as a subscription ID, user ID, or IP address.
  • Example:

Code

        <rate-limit-by-key calls="5" renewal-period="10" counter-key="@(context.Subscription.Id)" />

This limits each subscription to 5 calls every 10 seconds.

When rate limits are exceeded, APIM automatically returns a 429 Too Many Requests HTTP status code to the caller.

--------------------

Oclote  

Ocelot, an API Gateway for .NET Core, provides functionalities for both rate limiting and circuit breaking to manage user requests and enhance system resilience.

Rate Limiting with Ocelot:

Ocelot allows the configuration of rate limits on specific routes to control the number of requests a client can make within a defined period. This is achieved by adding a RateLimitOptions section to your Ocelot ReRoute configuration in ocelot.json.

Code

{
  "ReRoutes": [
    {
      "UpstreamPathTemplate": "/gateway/articles",
      "DownstreamPathTemplate": "/api/articles",
      "DownstreamScheme": "http",
      "DownstreamHostAndPorts": [
        {
          "Host": "localhost",
          "Port": 5000
        }
      ],
      "RateLimitOptions": {
        "EnableRateLimiting": true,
        "Period": "10s",
        "PeriodTimespan": 10,
        "Limit": 3,
        "HttpStatusCode": 429,
        "ClientWhitelist": []
      }
    }
  ]
}

• EnableRateLimiting: Set to true to activate rate limiting for the route.
• Period: Defines the time window for the rate limit (e.g., "10s" for 10 seconds).
• PeriodTimespan: Specifies the duration in seconds after which the client can retry after exceeding the limit within the Period.
• Limit: Sets the maximum number of requests allowed within the Period. 
• HttpStatusCode: The HTTP status code returned when the rate limit is exceeded (commonly 429 Too Many Requests).
• ClientWhitelist: An optional array of client identifiers that are exempt from rate limiting.

Circuit Breaker with Ocelot:

The circuit breaker pattern in Ocelot prevents repeated calls to a failing downstream service, protecting both the client and the service from cascading failures. This is configured within the QoSOptions section of your ReRoute.

Code

{
  "ReRoutes": [
    {
      "UpstreamPathTemplate": "/gateway/products",
      "DownstreamPathTemplate": "/api/products",
      "DownstreamScheme": "http",
      "DownstreamHostAndPorts": [
        {
          "Host": "localhost",
          "Port": 5001
        }
      ],
      "QoSOptions": {
        "ExceptionsAllowedBeforeBreaking": 5,
        "DurationOfBreak": 5000,
        "TimeoutValue": 3000
      }

    }
  ]
}