Application Performance Methodology

From Guidance Share

Jump to: navigation, search

- J.D. Meier, Srinath Vasireddy, Ashish Babbar, Alex Mackman, and Rico Mariani


Contents

Engineering for Performance

image:PerformanceConceptualFramework.gif


Approach

Performance must be given due consideration up front and throughout the life cycle. The guide promotes a structured and repeatable approach to performance that you can embed into your application life cycle. This enables you to mitigate performance risk from the onset of your project. You work toward defined performance objectives, design for performance, and test, measure, and tune throughout the life cycle.


image:PerformanceApproach.gif


Set Performance Objectives

Think carefully about the performance objectives for your application early during requirements analysis. Include performance objectives with your functional requirements and other nonfunctional requirements, such as security and maintainability.


Performance objectives should include the following:

  • Response time. This is the time it takes your system to complete a particular operation, such as a user transaction.
  • Throughput. This is the amount of work your system can support. Throughput can be measured in terms of requests per second, transactions per second, or bytes per second.
  • Resource utilization. This is the percentage of system resources that are used by particular operations and how long they are used. This is the cost of server and network resources, including CPU, memory, disk I/O, and network I/O.
  • Workload. This is usually derived from marketing data and includes total numbers of users, concurrently active users, data volumes, transaction volumes, and transaction mix.


Quality Attributes

Performance and scalability are quality of service attributes. You need to balance performance with other quality of service attributes, including security, maintainability, and interoperability.


image:BalancingQualityAttributes.gif


Your performance objectives and other quality of service attributes are derived from your business requirements. Metrics (captured by measuring) tell you whether you are trending toward or away from your performance objectives.


Design for Performance

Give performance due consideration up front. Performance modeling is a structured approach that supports performance engineering, in contrast to the haphazard approaches that characterize many projects. The performance and scalability frame promoted by this guide also enables you to apply structure and organization to the performance problem domain.


Performance and Scalability Frame

The performance and scalability frame helps your organize and prioritize performance and scalability issues.

Category

Key Considerations

Caching

Per user, application-wide, data volatility

Communication

Transport mechanism, boundaries, remote interface design, round trips, serialization, bandwidth

Coupling and Cohesion

Loose coupling, high cohesion among components and layers

Concurrency

Transactions, locks, threading, queuing

Data Structures / Algorithms

Choice of algorithm, Arrays vs. collections

Resource Management

Allocating, creating, destroying, pooling

State Management

Per user, application-wide, persistence, location


The categories in the frame are a prioritized set of technology-agnostic, common denominators that are pervasive across applications. You can use these categories to build evaluation criteria where performance and scalability decisions can have a large impact.


More Information


Performance Modeling

Performance modeling helps you evaluate your design decisions against your objectives early on, before committing time and resources. Invalid design assumptions and poor design practices may mean that your application can never achieve its performance objectives.


image:PerformanceModeling.gif


The performance modeling process consists of the following steps:

  1. Identify key scenarios. Identify those scenarios in which performance is important and the ones that pose the most risk to your performance objectives.
  2. Identify workloads. Identify how many users, and how many concurrent users, your system needs to support.
  3. Identify performance objectives. Define performance objectives for each of your key scenarios. Performance objectives reflect business requirements.
  4. Identify budget. Identify your budget or constraints. This includes the maximum execution time in which an operation must be completed and resource utilization such as CPU, memory, disk I/O, and network I/O constraints.
  5. Identify processing steps. Break your scenarios down into component processing steps.
  6. Allocate budget. Spread your budget determined in Step 4 across your processing steps determined in Step 5 to meet the performance objectives you defined in Step 3.
  7. Evaluate. Evaluate your design against objectives and budget. You may need to modify design or spread your response time and resource utilization budget differently to meet your performance objectives.
  8. Validate. Validate your model and estimates. This is an ongoing activity and includes prototyping, testing, and measuring.


Measuring Performance

You need to measure to know whether your application operates within its budget allocation and to know whether your application is trending toward or away from its performance objectives.


Know the Cost

You need to measure to know the cost of your tools. For example, how much does a certain application programming interface (API), library, or choice of technology cost you? If necessary, use prototypes to obtain metrics. As soon as development begins and you have real code to use, start measuring it and refine your performance models.


Validate

Validate your model and estimates. Continue to create prototypes and measure the performance of your application scenarios by capturing metrics. This is an ongoing activity and includes prototyping and measuring. Continue validating until your performance goals are met.

The further on you are in the application life cycle, the more accurate the validation will be. Early on, validation is based on available benchmarks and prototype code, or even proof-of-concept code. Later, you can measure your actual code as your application develops.


Testing Performance

Performance testing is used to verify that an application is able to perform under expected and peak load conditions, and that it can scale sufficiently to handle increased capacity.


Load Testing

Use load testing to verify application behavior under normal and peak load conditions. This allows you to capture metrics and verify that your application can meet its performance objectives.


image:LoadTestingProcess.gif


The load testing process involves the following steps:

  1. Identify key scenarios. Identify application scenarios that are critical for performance.
  2. Identify workload. Distribute the total application load among the key scenarios identified in Step 1.
  3. Identify metrics. Identify the metrics you want to collect about the application when running the test.
  4. Create test cases. Create the test cases, in which you define steps for conducting a single test along with the expected results.
  5. Simulate load. Use test tools to simulate load in accordance with the test cases. Capture the resulting metric data.
  6. Analyze results. Analyze the metric data captured during the test.


You begin load testing with a total number of users distributed against your user profile, and then you start to incrementally increase the load for each test cycle, analyzing the results each time.


Stress Testing

Use stress testing to evaluate your application's behavior when it is pushed beyond its breaking point, and to unearth application bugs that surface only under high load conditions.


image: StressTestingProcess.gif


The stress testing process involves the following steps:

  1. Identify key scenarios. Identify the application scenarios that need to be stress tested to identify potential problems.
  2. Identify workload. Identify the workload that you want to apply to the scenarios identified in Step 1. This is based on the workload and peak load capacity inputs.
  3. Identify metrics. Identify the metrics that you want to collect about the application when you run the test, based on the potential problems identified for your scenarios.
  4. Create test cases. Create the test cases, in which you define steps for conducting a single test along with the expected results.
  5. Simulate load. Use test tools to simulate the required load for each test case. Capture the resulting metric data.
  6. Analyze results. Analyze the metric data captured during the test.

The load you apply to a particular scenario should stress the system sufficiently beyond its threshold limits. You can incrementally increase the load and observe the application behavior over various load conditions.

Tuning Performance

Performance tuning is an iterative process that you use to identify and eliminate bottlenecks until your application meets its performance objectives. You establish a baseline and then collect data, analyze the results, identify bottlenecks, make configuration changes, and measure again.


image:TuningProcess.gif


Performance tuning consists of the following set of activities:

  1. Establish a baseline. Ensure that you have a well-defined set of performance objectives, test plans, and baseline metrics.
  2. Collect data. Simulate load and capture metrics.
  3. Analyze results. Identify performance issues and bottlenecks.
  4. Configure. Tune your application setup by applying new system, platform or application configuration settings.
  5. Test and measure. Test and measure to verify that your configuration changes have been beneficial.

Performance Throughout the Life Cycle

Performance begins during requirements gathering and continues throughout the application life cycle.


image:PerformanceThroughoutLifecycle.gif


The following list summarizes how performance is integrated throughout the entire life cycle:

  • Requirements gathering. You start to define performance objectives, workflow, and key scenarios, and begin to consider workloads and estimated volumes for each scenario. You begin the performance modeling process at this stage, using early prototyping if necessary.
  • Design. Working within your architectural constraints, you start to generate specifications for the construction of code. Design decisions should be based on proven principles and patterns, and your design should be reviewed from a performance perspective. Measuring should continue throughout the life cycle, starting from the design phase.
  • Development. You start reviewing your code early during the implementation phase to identify inefficient coding practices that could lead to potential performance bottlenecks. You can start to capture "real" metrics to validate the assumptions made in the design phase.
  • Testing. You conduct load and stress testing to generate metrics and to verify application behavior and performance under normal and peak load conditions.
  • Deployment. During the deployment phase, you validate your model using production metrics. You can validate workload estimates and also resource utilization levels, response time and throughput.
  • Maintenance. You should continue to measure and monitor when your application is deployed in the production environment. Changes that can impact system performance include increased user loads, deployment of new applications on shared infrastructure, system software revisions, and updates to your application to provide enhanced or new functionality.


Performance Engineering Adoption/Implementation Checklist

Topic

Checkpoint

Performance Modeling

Create performance models for your application.

Prototyping

Prototype early to validate your design assumptions. Measure prototype performance to determine whether or not your design approach enables you to meet your designed performance objectives.

Architecture and Design Review

Review the designs of new and existing applications for performance and scalability problems.

Code Review

Educate developers about how to conduct performance-based code reviews. Perform code reviews for applications in development.

Measuring

Know the cost of design decisions, technology choices, and implementation techniques.

Load Testing

Perform load testing to verify application behavior under normal and peak load conditions.

Stress Testing

Perform stress testing to evaluate your application's behavior when it is pushed beyond its breaking point.

Capacity Testing

Perform capacity testing to plan for future growth, such as an increased user base or increased volume of data.

Tuning

Tune your application to eliminate performance bottlenecks.

Personal tools