Evaluation Advisory Group

Background

Prior to adopting its MTI Evaluation Framework, CalMTA and the Market Transformation Advisory Board (MTAB) had extensive discussion regarding the appropriate roles of CalMTA versus CPUC Energy Division staff in MTI third-party evaluations, which are different than those established for investor-owned utility (IOU) impact evaluations, as delineated in Specifically, D.19-12-021 recognizes the value of timely, ongoing evaluation to the success of market transformation programs and calls for CalMTA to administer third-party market progress evaluations, which include assessment of outcomes.

While D.19-12-021 authorizes CalMTA to administer third-party evaluations, modeled after the Northwest Energy Efficiency Alliance’s successful approach, CalMTA and its advisory board agreed that it is vitally important to establish a management and oversight structure that ensures credibility and appropriate independence, avoids any potential conflict of interest, and ensures timely evaluation that can effectively inform decision-making. Toward that end, CalMTA and the Energy Division decided to form the Evaluation Advisory Group, which will function as described in Appendix D of the Evaluation Advisory Group Charter.

Overview

Pursuant to D.19-12-021, CalMTA will bid out evaluation, measurement, and verification (EM&V) activities to independent third-party evaluators via competitive solicitations, as described in the MTI Evaluation Framework. The Evaluation Advisory Group provides expertise and unbiased recommendations to CalMTA’s evaluation management team regarding evaluation plans for upcoming market transformation initiatives (MTIs) as well as planning, selection, and review of third-party MTI evaluation activities. Through its work, the Evaluation Advisory Group helps to ensure high quality, credible, and appropriately independent evaluations.

Member responsibilities and areas of expertise

The Evaluation Advisory Group is not a decision-making body. Member responsibilities include:

• Review and advise on MTI evaluation plans
• Review and advise on request for proposals (RFP) for third-party MTI evaluators
• Review and score third-party MTI evaluator proposals
• Review evaluation deliverables
• Advise the Evaluation Management Team, as requested

The Evaluation Advisory Group offers diverse evaluation expertise across customer sectors and initiative types. Members have expertise and experience in the following areas:

• Energy efficiency program EM&V
• Evaluation of market transformation programs
• Knowledge of national and California-specific market and program data sources
• Knowledge of energy efficiency and market transformation policies in California
• Metrics and measurement approaches to assess equity of program impacts
• Working as a constructive member of advisory groups

Current Evaluation Advisory Group members

The Evaluation Advisory Group is comprised of five members, including designated CPUC Energy Division Evaluation Lead, Christie Torok, and Karen Horkitz, CalMTA Market Research and Evaluation Lead.

Term of Appointment

Members were selected through an open recruitment and application process and began their three-year terms on October 1, 2024. Members have agreed to three-year terms with the option to extend for an additional term lasting one to three years.

Learn more about CalMTA’s MTI Evaluation Framework