Development Productivity
Development Productivity Dashboard
The Development Productivity dashboard provides insights into the productivity of development teams within the software development life cycle.
The dashboard helps answer the following questions:
- How many software changes are being produced in a release or sprint?
- How long does it take to complete software changes?
- What is slowing development teams down?
We will delve further into how the dashboard helps answer these questions, but first, it’s important to understand some of the concepts that the dashboard is based on.
Concepts
Life Cycle
The life cycle in Code Pipeline represents a software change model: a hierarchy of levels where each level represents one stage in the cycle. At the top of the hierarchy is the production level. The life cycle of a change, e.g. modifying a COBOL program begins by checking out a component version and is considered completed when the component version is promoted to production.
The dashboard only tracks activity that begins with a normal checkout. Activity related to checkouts with special action codes, e.g. compile only, are not tracked.
Phases and Levels
The life cycle has two distinct phases that changes travel through before being promoted to production: the development phase and the testing phase. The development phase begins when a component version is checked out to the lowest level in the life cycle. When the code changes are completed in the development phase, they are promoted to the testing phase. The testing phase begins upon promotion to the testing phase and ends upon promotion to production.
Typical levels might include:
- Testing phase: integration testing (INT), user acceptance testing (UAT), pre-production (PREP), and emergency fix hold (EHLD)
- Development phase: n development levels (DEV1, DEV2, etc.) and an emergency fix level (EFIX)
Regressions
A regression is an operation taken in the testing phase that deletes source and executables at a testing level and sends the related components back to the development phase.
Components and Assignments
An Code Pipeline assignment contains component versions (also known as tasks). As a best practice, assignments are created to represent a logical unit of work, e.g. a story or bug fix, and should be closed once the work is complete, so that all of the changed components in the assignment are promoted to production.
For more information on Code Pipeline concepts and terminology, see the Code Pipeline User Guide in the BMC Support Center.
Insights
The Development Productivity dashboard provide insights into the development process and answers to common questions, including the following:
How many software changes are being produced in a release or sprint?
The metric, component versions completed, conveys how many software changes are made for the given time range. Component versions completed are measured by these dimensions: life cycle, phase, level, application, assignment, and user. Component versions that are promoted to production are considered completed. Component versions that have not been completed are considered in progress and are excluded from the dashboard.
How long does it take to complete software changes?
The time elapsed while in various phases and levels is calculated as a component version travels through phases in the life cycle. Average time in phase/level is calculated based on the number of completed component versions. In this example, 2,619 component versions were completed taking an average time of 21.3 days to complete each component.
What is slowing development teams down?
In addition to the average time in phase or level (noted above), the total time a component version spends in a phase is calculated. There are visualizations that show the completed component versions that spent the longest time in the life cycle as well as the development and testing phases. Review the component versions that spent the longest time in the life cycle or phases to find potential areas for improvement.
Improving Development Productivity
Areas for Improvement
- Time: Time in life cycle, testing phase, or development phase is longer than desired
- Component versions: Number of completed component versions is lower than desired
- Regressions: Ratio of regressions to promotions is higher than desired
Possible Causes
The following are some possible causes of areas identified for improvement:
- Batch size: (amount of work) transported between levels in the life cycle is too large
- Automated test coverage: coverage is too low
- CI triggers missing: Continuous integration triggers are missing or are too late in the life cycle
- Tech debt: Technical debt is unaddressed
Taking Action
The dashboard Refactoring Candidates can help identify components with unaddressed technical debt.
Filtering the Dashboard
Filters help provide context by setting the scope of the dashboard. Setting filters can help your analysis process and answer specific questions you may have of the dashboard.
Time Filter
Use the time filter to change the time range. By default, the time filter is set to the last 90 days.
The time filter updates the dashboard to show data for component versions completed in the time range. It's important to note that most timeline visualizations will likely show timelines with a start date earlier than the time filter start date. The earlier start dates are included in the following timelines visualizations in order to show when significant events occurred related to the completed component versions, e.g. when a completed component version was promoted from the development to the testing phase.
- Testing Phase: Promotions from Testing Levels - Regressions: Regression of components back to the development phase - Development Phase: Promotions to the Testing Phase |
Filter Visualization
Filter drop-downs are provided for easy filtering on common fields:
- Applications: Filtering on Code Pipeline application IDs helps focus only on applications of interest, e.g. one or more teams that work on those applications.
- Users: Filtering on users is a typical method for focusing on a team's productivity. Important. Filtering on users impacts time calculations. If you want to analyze time metrics, be sure to include all users who are involved in the development, testing, and promotion process. To understand how time might be affected by user filters, consider the following scenario. Time spent is calculated by subtracting the time between a given operation on a component version and the previous operation on that component version. In this case, a release engineer with user ID RELENG1 issued a promote operation of a component version to PROD. The time of 1 hr 31 seconds that is attributed to user RELENG1 is the _time elapse_d since the last operation, which was a promote operation to level EHLD by user DEVELOP1. It is not the time spent by RELENG1 performing the promote operation.
Component Version | User | Operation | Target Level | Time |
|---|---|---|---|---|
APP1-ASGN000555-PROGA-CBL-000025 | RELENG1 | Promote | EHLD | 0:12:00 |
APP1-ASGN000555-PROGA-CBL-000025 | DEVELOP1 | Promote | EHLD | 0:12:00 |
APP1-ASGN000555-PROGA-CBL-000025 | DEVELOP1 | Generate | EHLD | 0:12:00 |
APP1-ASGN000555-PROGA-CBL-000025 | DEVELOP1 | Checkout | EHLD | 0:12:00 |
- Assignments: Filtering on assignments sets the dashboard scope for software changes related to those assignments.
- Component Names: Filtering on component names should be paired with a component type filter if you are interested in focusing on unique components.
- Component Types: Filtering on component type can helps focus on types of components, such as COBOL programs and copybooks, JCL, etc.
Dashboard Layout and Visualizations
The dashboard is divided into three sections:
- Life Cycle: The visualizations in the Life Cycle section represent the software change process from checkout to promotion to production.
- Testing Phase: The visualizations in the Testing Phase section represent operations (e.g. promotions or regressions) in the testing levels of the life cycle.
- Development Phase: The visualizations in the Development Phase section represent operations (e.g.checkouts or promotions) in the development levels of the life cycle.
Life Cycle Visualizations
The life cycle in Code Pipeline represents a software change model. In other words a hierarchy of levels where each level represents one stage in the cycle. At the top of the hierarchy is the production level. The life cycle of a change, such as modifying a COBOL program, begins by checking out a component version and is considered completed when the component version is promoted to production.
Component Versions Completed
The total number of component versions completed in the given time range.
Average Time in Life Cycle
Average time in life cycle is calculated based on the number of completed component versions. In this example, it took an average of 21.3 days to complete each component.
Life Cycle: Component Versions Completed (Timeline)
This timeline visualization shows the number of component versions completed (promoted to production) per day.
Life Cycle: Velocity by Level
This visualization shows the velocity of each level in the life cycle, i.e. the number of component versions traveling through each level and the average time component versions spent in that level.
Life Cycle: Component Versions Completed by Application
This visualization shows the number of component versions completed by application.
Life Cycle: Assignments
This visualization shows assignments having completed component versions. Each row in the table shows the number of component versions in the assignment that were completed in the given time range and the number of users who have performed operations on those completed component versions.
Life Cycle: Users
This visualization shows the number of completed component versions by user. It also shows how many assignments were involved in the completion of those component versions.
Longest Time in Life Cycle
This visualization shows the completed component versions that took the longest time to complete, which may signify areas for improvement.
Testing Phase Visualizations
The testing phase begins upon promotion to the testing phase and ends upon promotion to production.
Average Time in Testing Phase
Average time in the testing phase is calculated based on the time that completed component versions spent in the testing phase. In this example, it took an average of 13.2 days for each completed component to travel through the testing phase.
Testing Phase: Component Versions Completed (Timeline)
This timeline visualization shows the number of component versions completed in the testing phase (promoted from a testing level to another testing level or to production) per day. Note that the y-axis scale displays as square root rather than linear, so that days with fewer promotions are visible.
Testing Phase: Velocity by Level
This visualization shows the velocity of each level in the testing phase, i.e. the number of component versions traveling through each level and the average time component versions spent in that level.
Longest Time in Testing Phase
This visualization shows the completed component versions that took the longest time to travel through the testing phase, which may signify areas for improvement.
Regressions and Regressions Ratio
This visualization shows the number of regressions that occurred in the testing phase. A high ratio of regressions to promotions can signify areas for improvement.
Regression (Timeline)
This visualization shows the timeline of the number of regressions that occurred per day.
Most Regressions
This visualization shows the completed component versions that had the most number of regressions, which can signify areas for improvement.
Development Phase Visualizations
The development phase begins when a component version is checked out to the lowest level in the life cycle. When the code changes are completed in the development phase, they are promoted to the testing phase.
Average Time in Development Phase
Average time in the development phase is calculated based on the time that completed component versions spent in the development phase. In this example, it took an average of 8.1 days for each completed component to travel through the development phase.
Development Phase: Promotions to the Testing Phase (Timeline)
This timeline visualization shows the number of component versions completed in the development phase (promoted to the testing phase) per day. Note that the y-axis scale displays as square root rather than linear, so that days with fewer promotions are visible.
Development Phase: Velocity by Level
This visualization shows the velocity of each level in the development phase, i.e. the number of component versions traveling through each level and the average time component versions spent in that level.
Longest Time in Development Phase
This visualization shows the completed component versions that took the longest time to travel through the development phase, which may signify areas for improvement.
