ML Performance without Labels: Comparing Performance Estimation Methods (Webinar Replay)
소스 코드
- 게시일 2024. 04. 26.
- This talk will discuss the advances in Performance Estimation for classification models without access to the ground truth data. Once you've deployed your model to production, you want to ensure it performs well. However, this is often hard to do if you predict an event far in the future or automate certain tasks.
We will introduce three algorithms: Confidence-Based Performance Estimation (CBPE), Importance Weighting (IW), and Multicalibrated Confidence-Based Performance Estimation (M-CBPE). We show that all three give better performance estimates than the test set performance.
We will explain how they work, the intuition behind the algorithms, and their strengths and weaknesses. We will also showcase typical scenarios where Performance Estimation is vital and how to use it in ML Monitoring and Root Cause Analysis.
Timestamps:
0:00 - Introduction
0:33 - Paper results overview
4:02 - Agenda
4:50 - Why is performance estimation vital?
6:21 - How is performance estimation possible?
13:15 - Uncertainty to Performance metric
32:20 - Importance weighting
39:25 - M-CBPE - 과학기술
![토끼와 거북이 (The Tortoise and the Hare) #1｜[SKZ CODE] Ep.49](http://i.ytimg.com/vi/L5pYlIc_Fp8/mqdefault.jpg)
![[데드풀과 울버린] 안녕? 오늘 밤 기대해도 좋아..🔥｜새로운 예고편의 예고 영상](http://i.ytimg.com/vi/AKwjy4qLghM/mqdefault.jpg)
