Evaluating market transformation progress
The California Market Transformation Framework, created via CPUC Decision 19-12-021, calls for setting clear savings goals and other market transformation initiative (MTI) metrics to ensure a high level of accountability, and ongoing evaluation to reduce program performance risk. To ensure this outcome, CalMTA has developed an Evaluation Framework that describes the policies, principles, and high-level approaches that CalMTA will use to assess its portfolio of MTIs. The approaches are consistent with the Decision and reference the substantial existing body of literature on the best practices and lessons learned for market transformation evaluation.
The draft Evaluation Framework was available for public comment Oct. 17-Nov. 2, 2023 on the California Public Utilities Commission (CPUC) public documents site. CalMTA has compiled a comprehensive list of comments received and provided a response to each comment in the Evaluation Framework Feedback Response Memo.
The critical role of evaluation
Evaluation is essential to ensure that market transformation programs are making progress and delivering cost-effective energy savings for California’s ratepayers and incremental system benefits to California’s electrical grid. CalMTA will oversee implementation of rigorous and strategically focused evaluation, measurement, and verification (EM&V) practices, which will allow us to gauge the performance of MTIs, verify incremental impacts, and improve the design and success of future work.
Evaluation informs effective evolution of initiatives and ongoing investment decisions and supports strong management accountability, which in turn can enhance stakeholder trust and collaboration. The graphic below illustrates the importance and purpose of evaluation.
“Real-time” market evaluation provides the continual feedback that MTI program managers and implementers need to understand whether the intended market changes are occurring and, if needed, pivot strategies to improve MTI performance. Tracking of near- and longer-term market progress indicators that are tightly aligned with the MTI’s market transformation theory will reduce MTI performance risk and support timely decisions regarding ongoing investment or termination of an MTI.
Watch the video
View our October 18, 2023 presentation to the Market Transformation Advisory Board (MTAB) on the Evaluation Framework:
You can also see the January 8, 2024 Evaluation Framework webinar, which provides additional clarification about the proposed evaluation approach, oversight, and management.
Want to learn more?
Read our blog post to learn more about CalMTA’s Evaluation Framework for Market Transformation Initiatives.