Evaluation Advisory Group
Background
Prior to adopting its MTI Evaluation Framework, CalMTA and the Market Transformation Advisory Board (MTAB) had extensive discussion regarding the appropriate roles of CalMTA versus CPUC Energy Division staff in MTI third-party evaluations, which are different than those established for investor-owned utility (IOU) impact evaluations, as delineated in Specifically, D.19-12-021 recognizes the value of timely, ongoing evaluation to the success of market transformation programs and calls for CalMTA to administer third-party market progress evaluations, which include assessment of outcomes.
While D.19-12-021 authorizes CalMTA to administer third-party evaluations, modeled after the Northwest Energy Efficiency Alliance’s successful approach, CalMTA and its advisory board agreed that it is vitally important to establish a management and oversight structure that ensures credibility and appropriate independence, avoids any potential conflict of interest, and ensures timely evaluation that can effectively inform decision-making. Toward that end, CalMTA and the Energy Division decided to form the Evaluation Advisory Group, which will function as described in Appendix D of the Evaluation Advisory Group Charter.
Overview
Pursuant to D.19-12-021, CalMTA will bid out evaluation, measurement, and verification (EM&V) activities to independent third-party evaluators via competitive solicitations, as described in the MTI Evaluation Framework. The Evaluation Advisory Group provides expertise and unbiased recommendations to CalMTA’s evaluation management team regarding evaluation plans for upcoming market transformation initiatives (MTIs) as well as planning, selection, and review of third-party MTI evaluation activities. Through its work, the Evaluation Advisory Group helps to ensure high quality, credible, and appropriately independent evaluations.
Member responsibilities and areas of expertise
The Evaluation Advisory Group is not a decision-making body. Member responsibilities include:
- Review and advise on MTI evaluation plans
- Review and advise on request for proposals (RFP) for third-party MTI evaluators
- Review and score third-party MTI evaluator proposals
- Review evaluation deliverables
- Advise the Evaluation Management Team, as requested
The Evaluation Advisory Group offers diverse evaluation expertise across customer sectors and initiative types. Members have expertise and experience in the following areas:
- Energy efficiency program EM&V
- Evaluation of market transformation programs
- Knowledge of national and California-specific market and program data sources
- Knowledge of energy efficiency and market transformation policies in California
- Metrics and measurement approaches to assess equity of program impacts
- Working as a constructive member of advisory groups
Current Evaluation Advisory Group members
The Evaluation Advisory Group is comprised of five members, including designated CPUC Energy Division Evaluation Lead, Christie Torok, and CalMTA evaluation lead, Karen Horkitz, CalMTA Market Research and Evaluation Lead, Senior Market Transformation Advisor to the Cadmus Group.
Note: Evaluation Advisory Group members must be excluded from CalMTA’s competitive solicitation process for third-party evaluators and must sign a Conflict of Interest statement.
Term of Appointment
Members were selected through an open recruitment and application process and began their three-year terms on October 1, 2024. Members have agreed to three-year terms with the option to extend for an additional term lasting one to three years.
Learn more about CalMTA’s MTI Evaluation Framework