Closing Remarks: What can we do with multiple diverse solutions?

Size: px

Start display at page:

Download "Closing Remarks: What can we do with multiple diverse solutions?"

Janis Harrison
5 years ago
Views:

1 Closing Remarks: What can we do with multiple diverse solutions? Dhruv Batra Virginia Tech

2 Example Result CRF Diverse Segmentations. Now what? (C) Dhruv Batra 2

3 Your Options Nothing User in the loop (Approximate) Min Bayes Risk Use solutions to estimate the distribution and optimize Bayes Risk Re-ranking Pick a good solution from the list Increasing Side Information (C) Dhruv Batra 3

4 Interactive Segmentation Setup Model: Color/Texture + Potts Grid CRF Inference: Graph-cuts Dataset: 50 train/val/test images Image + Scribbles MAP 2 nd Best MAP Diverse 2 nd Best 1-2 Nodes Flipped Nodes Flipped (C) Dhruv Batra 4

5 Interactive Segmentation Better 96% +3.62% Segmentation Accuracy 95% 94% 93% 92% 91% +0.05% +1.61% 90% 89% MAP M-Best-MAP Confidence DivMBest (Oracle) (Oracle) (Oracle) (C) Dhruv Batra 5 M=6

6 Your Options Nothing User in the loop (Approximate) Min Bayes Risk Use solutions to estimate the distribution and optimize Bayes Risk Re-ranking Pick a good solution from the list (C) Dhruv Batra 6

7 Statistics 101 Loss PCP, Pascal Loss, etc True Distribution L(y gt, ŷ) P (y gt x) = P (X ) X X MAP Expected Loss: BR(ŷ) =E P L(y gt, ŷ) Min Bayes Risk min y Y y gt Y L(y gt, ŷ) P (y gt ) (C) Dhruv Batra 7

8 Structured Output Problems Min Bayes Risk min y Y y gt Y L(y gt, ŷ) P (y gt ) Two Problems Intractable Intractable Approximate MBR: min ŷ DivMBest y gt DivMBest L(y gt, ŷ) P (y gt ) P (X ) X (C) Dhruv Batra 8 X MAP

9 Semantic Segmentation Setup Models: Hierarchical CRF [Ladicky et al. ECCV 10, ICCV 09] Second-Order Pooling [Carreira ECCV 12] Inference: Alpha-expansion Greedy Dataset: Pascal Segmentation Challenge (VOC 2012) 20 categories + background; ~1500 train/val/test images (C) Dhruv Batra 9

10 Large-Margin Re-ranking Diverse Segmentations CRF (C) Dhruv Batra 10

11 Semantic Segmentation Input MAP Best of 10-Div (C) Dhruv Batra 11

Model 47% 44% 1 2 3 4 5 6 7 8 9 10 Rand (Re-rank)

12 Semantic Segmentation Better 59% DivMBest (Oracle) PACAL Accuracy 56% 53% 50% 15%-gain possible Same Features Same Model 47% 44% Rand (Re-rank) #Solutions / Image MBR MAP [State-of-art circa 2012] (C) Dhruv Batra 12

13 Your Options Nothing User in the loop (Approximate) Min Bayes Risk Use solutions to estimate the distribution and optimize Bayes Risk Re-ranking Pick a good solution from the list (C) Dhruv Batra 13

14 Large-Margin Re-ranking Diverse Segmentations CRF (C) Dhruv Batra 14

15 Large-Margin Re-ranking Diverse Segmentations CRF ψ(, ) (C) Dhruv Batra 15

16 Large-Margin Re-ranking Diverse Segmentations CRF α ψ(, ) α ψ(, ) (C) Dhruv Batra 16

17 Large-Margin Re-ranking Diverse Segmentations CRF min α,ξ i Discriminative Re-ranking of Diverse Segmentation α 2 + C i ξ i [Yadollahpour et al., CVPR13, Wednesday Poster] α ψ(, ) α ψ(, ) 1 ξ i loss i (C) Dhruv Batra 17

18 Semantic Segmentation Better 59% DivMBest (Oracle) PACAL Accuracy 56% 53% 50% DivMBest (Re-ranked) [Y.B.S., CVPR 13] 47% 44% MBR MAP [State-of-art circa 2012] Rand (Re-rank) #Solutions / Image (C) Dhruv Batra 18

19 Qualitative Results: Success (C) Dhruv Batra 19

20 Qualitative Results: Success (C) Dhruv Batra 20

21 Qualitative Results: Success (C) Dhruv Batra 21

22 Qualitative Results: Failures (C) Dhruv Batra 22

23 Qualitative Results: Failures (C) Dhruv Batra 23

24 Qualitative Results: Failures (C) Dhruv Batra 24

25 Summary All models are wrong Some beliefs are useful Diverse Multiple Solutions A way to get useful beliefs out. DivMBest + Reranking Big impact possible on many applications! (C) Dhruv Batra 25

26 Summary What does my model believe? Posterior Summary (C) Dhruv Batra 26

27 Thanks! (C) Dhruv Batra 27

ECE 5424: Introduction to Machine Learning

ECE 5424: Introduction to Machine Learning Topics: (Finish) Model selection Error decomposition Bias-Variance Tradeoff Classification: Naïve Bayes Readings: Barber 17.1, 17.2, 10.1-10.3 Stefan Lee Virginia