Support

Local Development Stage

• Setup the source code repository (GitLab)
  • https://gitlab.com/nyu-central/edm/api-project/
• Create a new project space under the following wiki section
  • Projects – Documentation
• Setup local development environment: JDK, Maven, Anypoint Studio, SCM Client (Git)
  • Developer Workstation Setup
• Developers make changes to their local repository’s copy of the application branch of a deployment project
• Internal testing using Studio embedded Runtime target environments
• Configure Mule Maven pom files on your applications
  • Mule Applications Configurations for CI/CD

CloudHub/Hybrid Development-Sandbox Stage

• CI/CD setup (Jenkins), create a sandbox job to enable applications automated deployment, following the defined standards
  • Automated Deployment – Best Practices
• Deployment to Sandbox environments using Jenkins
• Integration testing on Sandbox environment
• Define application/API functional and non-functional Test Cases, document them under the specific-project wiki section
  • Projects – Documentation
• Functional and non-functional testing over application/API in the corresponding environment (CloudHub, Hybrid)

UAT Stage

• Promote Sandbox applications to UAT
  • Promote an API/Application from a lower environment
• CI/CD setup (Jenkins), create a UAT job to enable applications automated deployment, following the defined standards
  • Automated Deployment – Best Practices
• Execute the CI/CD job
• Execute Performance testings
  • Performance Test – Best Practices
• Users to perform validation on agreed test cases

Go Live (Production) Stage

• Promote the Application/API from the UAT environment
  • Promote an API/Application from a lower environment
• CI/CD setup (Jenkins), create a Production job to enable applications automated deployment, following the defined standards
  • Automated Deployment – Best Practices
• Execute the CI/CD job
• Health check on applications deployment on Production
• Perform non-functional testing as required
• Monitor applications behaviour using Analytics metrics

SLA Tiers

• Despite you can define SLA tiers per API, as a best practice the SLA definitions should be homogeneous across all APIs or across defined groups of APIs.
• Define SLA tiers for each API to enforce the approval workflow and access limits (limit the number of requests an application can make to the API). 
• The Names and values of the tiers should be standard across all APIs following a naming-convention

• The limits are purely descriptive and should be enforced by using policies. Enforce the SLA tiers with SLA-based policies such as rate-limiting and throttling

For more information: https://docs.mulesoft.com/api-manager/defining-sla-tiers

API Policies

• The policies should be applied homogeneous across all environments
• Apply at least one of the following security-related policies
  • Client ID enforcement
  • Open ID token enforcement (OAuth2)
• Apply an SLA rate-limiting policy defining the limits based on the Performance Testing results for each API implementation

API Alerts

• Despite you can define Alerts per API, as a best practice the Alert definitions should be homogeneous across all APIs or across defined groups of APIs.
• See: Notifications and Alerts#APIAlerts

Analytics

• Use the default dashboard to see API consumption parameters
• Create a custom dashboard with custom charts if needed

Introduction

To understand the resources and vCores/cores size that you will need for each project/use case you have 2 paths:

– Reliable: Create a performance benchmarking/test, execute load tests monitoring the resources, start at light load then increase. All the methodology is explained in this document

– Estimated: Pre-Sales sizing procedure (based on relative complexity and fixed parameters). Please contact MuleSoft Customer Success or Pre-Sales representative for the account

Concepts

Types of tests

Load Testing

Load testing is performed to determine a system’s behaviour under both normal and anticipated peak load conditions.

Stress Testing

Testing activity that determines the robustness of software by testing beyond the limits of normal operation, often to a breaking point, in order to observe the results.

Soak Testing

Is a type of performance test that verifies a system’s stability and performance characteristics over an extended period of time. It is typical in this type of performance test to maintain a certain level of user concurrency for an extended period of time.

Metrics/Values

Concurrency

Refers to the ability of an application to be executed out-of-order or in partial order, without affecting the final result. Concurrency is typically a stated requirement – eg. “application must handle up to 10,000 concurrent users with a 90th percentile round-trip time of 100ms maximum”. The goal of testing is to measure what concurrency a given system will support; this means that concurrency is a variable parameter.

Throughput

Refers to the number of transactions or requests that can be made in a given period of time. It is a useful measurement to check the load capacity of the server.

Methodology

Define the Scope

• The final goal of the test (e.g. check if the system remains stable during xx hours of load)
• Systems/Nodes that are going to be included in the test
• Representative environment, should have the same resources and specs of the production environment
• Application(s), resources and data to be used

• Define the “redline” % of system resources in production – e.g. “do not allow CPU% to exceed 70%”
• Always monitor the system resources under every load test, at a minimum capture these metrics
  • CPU%, CPU load 
  • mem%   
  • disk%, IOPS
  • network%, network socket
  • GC log  

• Considerations and limitations

Define the monitoring tools to use

• Define the tools that you are going to use to monitor the systems (e.g CloudHub Dashboard, Analytics dashboard, Database Analytics, jConsole, etc)

Prepare your test plan based on the defined scope

Create a representative workload (to begin, make this static – repeatability is everything)

• Request methods (GET, PUT, POST)
• API(s) that are part of the scenario.
• Payload size
  • Set this as your new baseline for capacity planning
  • Use this baseline to mathematically find out the metrics you are looking for
    • Start at a light load, if resources are plentiful, increase the load
    • If TPS increases past the value of the previous test, latency stays relatively flat and no resources are close to “redline”, increase the TPS
    • If TPS does NOT increase and none of the resources get close to full capacity, it means there is likely thread contention – profile the application to determine where thread-contention is occurring
    • If TPS does NOT increase and latency starts to increase linearly, and one of the resources gets close to full capacity, the system is saturated. Reduce load slightly and rerun the load test, set this as the baseline

Ensure differentiation between performance of: proxy, back-end, or proxy+back-end

This applies specifically to use cases where proxies to legacy back-ends are being designed and deployed. It is important to plan ahead for ability to test performance of only the proxy – to prevent possibly poor-performance of legacy back-ends from tainting accurate impressions of performance. While overall business-performance is most important, nothing is helped if the API Gateway is blamed for performance for slowness.

It is important to establish the baseline performance of proxy-only, however artificial that is in a business-sense. Comparison of overall system performance to proxy-only performance gives a valuable baseline and will aid in diagnosis and rectification of performance problems.

Run Load tests to find the baseline

• Run the load test until the resources get close to the “redline”

Run Soak tests

• Define a representative unit of time to see the stability of the system (e.g. 4-8 hrs)
• Run the test based on the baseline load
• Store the results/metrics

Build a report

• Include the defined scope, configurations, monitoring tools and test plan 
• Include the base line
• Include the results
• Include conclusions based on the results