The Metropolitan Transportation Commission (MTC) is the developer of the StreetSaver pavement management software with more than 500 users nationwide, consisting predominantly of cities and counties. In addition to making StreetSaver available to local agencies, MTC uses the software to perform regional street and road condition and maintenance needs assessments for the 109 cities and counties in the San Francisco Bay Area. Since 2008, StreetSaver has been used exclusively to perform the California local street and road statewide needs assesment. The statewide assessment is used to inform the Governor and Legislature of the critical funding shortfall for maintaining local streets and roads, and the consequences of deferring or reducing transportation funds for cities and counties.

To ensure quality data from our Pavement Management Technical Assistance Program (P-TAP) consultants for our analysis, MTC has developed procedures and guidelines for managing the quality of pavement data collection activities.

MTC’s Data Quality Management Plan includes three components:

  1. Vendor Certification Program (VCP)
  2. Quality control plan–before, during and after production
  3. Data acceptance

The VCP is part of the consultant selection and is an integral part of our data quality plan. The Rater Certification Program; however, ensures that raters are capable of providing the desired level of accuracy on pavement condition ratings. Some StreetSaver users at the local agency level, within or outside the Bay Area, may require that consultants’ raters be certified or the firm is vendor-certified by MTC to perform data collection.

I. Vendor Certification Program

Vendor certification of contractors is the first component of the pavement distress data quality plan. The VCP ensures that the consultant selected to participate in the P-TAP contracts are capable of collecting distress data that is reasonably close to what would be collected by an “expert” rater.

Visit Vendor Certification Program for more information.

II. QUALITY CONTROL PLAN

Each vendor-certified firm is responsible for providing quality data to the project sponsor and MTC. In the Quality Control Plan (QCP), each firm will be required to describe the qualifications of each rater who will collect pavement distress data, including training and experience. If new raters will be assigned to collecting pavement distress data, the data collection contractor will need to describe the training and supervision that will be exercised during their “internship” period. The qualifications of the new raters must be provided to and approved by MTC before they will be allowed to conduct ratings without the direct supervision of experienced raters.

All vendor-certified firms should describe the data verification processes that they will employ to validate accuracy, resolution and precision of the data collected. “Accuracy” indicates that the distress is identified correctly, e.g., alligator cracking is identified as alligator cracking instead of block cracking. “Resolution” indicates that severity levels are identified accurately, e.g., low severity longitudinal cracking is identified and recorded rather than omitted. “Precision” indicates that repeated ratings produce reasonably similar recorded distress type, severity and density combinations.

Data verification processes must include:

  1. Periodic re-survey of “control” sections by survey teams at least once every two weeks.
  2. Re-survey of at least 5 percent of the sections previously rated within one month of completing surveys. The same survey team may do the re-surveys.
  3. Re-survey of at least 5 percent of the rated sections by a supervisor.
  4. Checks of collected data against prior survey data and checks of calculated PCI values against PCI values based on prior survey data projected to the survey date for the same section if no treatments have been applied since the prior survey. All of those outside plus or minus 15 PCI point differences should be checked by a supervisor or other person approved by MTC.

The QCP must define what results will be considered acceptable; at a minimum, results must meet the acceptance criteria as defined in the pre-qualification. The firm must also describe which remedial actions will be taken if the results of the data checks are not acceptable. The Plan must identify when, in what format, and how often the results of the quality control plan checks, and corrective actions will be submitted to MTC. The Plan must also identify when the survey results will be entered into StreetSaver software, the PCI values calculated, and the results made available to MTC. Vendor-certified firm must submit a Quality Management Report for acceptance by the project sponsor.

Rater Certification Program

Under the P-TAP, even if a firm is vendor-certified, all of the firm’s raters must be certified by MTC through the Rater Certification Program (RCP). The RCP consists of passing a pavement distress manual survey exam, and an online written exam. These exams are designed to evaluate the skills and knowledge of raters on pavement condition rating based on the MTC’s seven-distress protocol. A rater must rate up to 24 test sites and achieve the required level of accuracy performing ratings as defined by the Acceptance Criteria (see above). Raters from non-P-TAP consulting firms and local agencies can be certified. The rater certification exam will be conducted at least once a year or as needed. Upon successful completion of the exams, a certificate will be issued that is valid for two years. Renewal is good for another two more years upon successful completion of an online refresher course. Every four years, all certified raters must take the field and written exams to be re-certified.

Visit Rater Certification Program for more information.

III. QUALITY ACCEPTANCE PLAN 

MTC staff will need to communicate with the data collection contractors about issues found such as the sections for which the calculated PCI values do not match the projected PCI values to determine how to approach further steps. If the data collection contractor does not meet the requirements of the data collection quality control plan or if MTC determines that the collected data does not meet the requirements established in the prequalification requirements, MTC can issue a stop work order and require corrective actions. This may include requiring the data collection contractor to re-certify all of the survey teams being used to collect data under the P-TAP contract, and re-surveying all sections rated since the last checks that showed the contractor was meeting contract requirements. Other less onerous corrective actions may be imposed by MTC on the data collection contractor. Multiple violations could result in termination of the data collection contract. MTC or its designated staff, the California State University at Chico (CSUC) will administer the Quality Acceptance Plan.

1) Administer Rater Certification Program

Because vendor-certified contractor for the P-TAP does not ensure that all raters are capable of rating with the desired level of accuracy, all raters employed by the contractor will need to be rater-certified.

The Rater Certification Program consists of a field pavement distress survey test and an online written test. CSUC shall facilitate pavement distress surveys and online written tests for raters up to twice a year. When needed, CSUC shall establish ground truths for pavement distress survey testing. CSUC ’s services shall include administering registration, field and online tests, issuance and renewal of certification, and database tracking.

2) Conduct Audits of Contractor’s Quality Control Plan

CSUC shall verify that the quality control plans (QCP) adopted by the data collection contractors are being completed prior to project sponsor acceptance of survey results. CSUC shall conduct on-project site audits. CSUC shall conduct audits of the QCP results to ensure that the data collection contractors are meeting the requirements established in their plans.

The requirements of the QCP are:

  1. Re-survey of “control” sections by survey teams at least once every two weeks.
  2. Re-survey of at least five (5) percent of the sections previously rated within one month of completing surveys. The same survey team may do the re-surveys.
  3. Re-survey of at least five (5) percent of the rated sections by a supervisor.
  4. Checks of collected data against prior survey data and checks of calculated PCI values against PCI values based on prior survey data projected to the survey date for the same section if no treatments have been applied since the prior survey. All of those outside plus or minus 15 PCI point differences should be checked by a supervisor or other person approved by MTC.

CSUC shall spot check and/or conduct full audits of the QCP from selected projects as directed by MTC.

3) Verify Data Collected by Contractors

CSUC shall conduct data verification actions including at least some of the following:

  1. Checks of collected data against prior survey data and checks of calculated PCI values against PCI values based on prior survey data projected to the survey date for the same section if no treatments have been applied since the prior survey.
  2. Survey of sections previously rated by the data collection contractor.

For more information about the MTC Data Quality Management Plan, please contact ​us.