Visual Domain Adaptation in the Deep Learning Era
Published on 6. June 2022
XXI, 168 pages
978-3-031-79175-8 (ISBN)
Description
Solving problems with deep neural networks typically relies on massive amounts of labeled training data to achieve high performance. While in many situations huge volumes of unlabeled data can be and often are generated and available, the cost of acquiring data labels remains high. Transfer learning (TL), and in particular domain adaptation (DA), has emerged as an effective solution to overcome the burden of annotation, exploiting the unlabeled data available from the target domain together with labeled data or pre-trained models from similar, yet different source domains. The aim of this book is to provide an overview of such DA/TL methods applied to computer vision, a field whose popularity has increased significantly in the last few years. We set the stage by revisiting the theoretical background and some of the historical shallow methods before discussing and comparing different domain adaptation strategies that exploit deep architectures for visual recognition. We introduce the space of self-training-based methods that draw inspiration from the related fields of deep semi-supervised and self-supervised learning in solving the deep domain adaptation. Going beyond the classic domain adaptation problem, we then explore the rich space of problem settings that arise when applying domain adaptation in practice such as partial or open-set DA, where source and target data categories do not fully overlap, continuous DA where the target data comes as a stream, and so on. We next consider the least restrictive setting of domain generalization (DG), as an extreme case where neither labeled nor unlabeled target data are available during training. Finally, we close by considering the emerging area of learning-to-learn and how it can be applied to further improve existing approaches to cross domain learning problems such as DA and DG.
More details
Other editions
Additional editions
Gabriela Csurka | Timothy M. Hospedales | Mathieu Salzmann
Visual Domain Adaptation in the Deep Learning Era
Book
04/2022
Springer
€80.24
The article will not be published
Gabriela Csurka | Timothy M. Hospedales | Mathieu Salzmann
Visual Domain Adaptation in the Deep Learning Era
Book
04/2022
Springer
€53.49
Shipment within 15-20 days
Persons
Gabriela Csurka is a Principal Scientist at NAVER LABS Europe, France. Her main research interests are in computer vision for image understanding, 3D reconstruction, visual localization, as well as domain adaptation and transfer learning. She has contributed to around 100 scientific communications, several on the topic of DA. She has given several invited talks and organized a tutorial on domain adaptation at ECCV'20. In 2017 she edited the Springer book Domain Adaptation for Computer Vision Applications. Timothy M. Hospedales is a Professor at the University of Edinburgh; Principal Researcher at Samsung AI Research Centre, Cambridge; and Alan Turing Institute Fellow. His research focuses on lifelong machine learning, broadly defined to include multi-domain/multi-task learning, domain adaptation, transfer learning, and meta-learning, with applications including computer vision, language, reinforcement learning for control, and finance. He has co-authored numerous papers on domain adaptation, domain generalization, and transfer learning in major venues including CVPR, ICCV, ECCV, ICML, ICLR, NeurIPS, and AAAI. He teaches computer vision at Edinburgh University and has given invited talks and tutorials on these topics at various international venues, renowned universities, and research institutes. Mathieu Salzmann is a Senior Researcher at EPFL and, since May 2020, a part-time Artificial Intelligence Engineer at ClearSpace. His research focuses on developing machine learning algorithms for visual scene understanding, including object recognition, detection, semantic segmentation, 6D pose estimation, and 3D reconstruction. He has published articles on the topic of domain adaptation at major venues, including CVPR, ICCV, ICLR, AAAI, TPAMI, and JMLR. Furthermore, he has been invited to present his domain adaptation work at various venues and internationally renowned universities. Tatiana Tommasi is Associate Professor at Politecnico di Torino, Italy and an affiliated researcher at the Italian Institute of Technology. She pioneered the area of transfer learning for computer vision and has large experience in domain adaptation, generalization, and multimodal learning with applications for robotics and medical imaging. Tatiana received the best paper award at the 1st edition of Task-CV workshop at ECCV'14 and since then she has been leading the organization of the following workshop editions. She also organized a workshop on similar topics at NIPS'13 and '14 and taught a tutorial at ECCV'14 and '20.
Content
- Cover
- Copyright Page
- Title Page
- Contents
- Preface
- Figure Credits
- Motivation
- Introduction
- Real Application Need
- Dataset Bias
- DA Benchmarks and Challenges
- Book Organization
- Theoretical Background
- Basic Concepts
- Cross-Domain Generalization
- Distances Between Distributions
- Maximum Mean Discrepancy
- f-Divergences
- Wasserstein Distance
- Traditional Methods
- Parameter Adaptation for DA
- Metric Learning for DA
- Aligning Data Representations
- Subspace Representations
- Other Representations
- Aligning Data Distributions
- Re-Weighting Instances
- Learning Transformations
- Deep Domain Adaptation
- Using Deep Features in Traditional DA
- Aligning Data Representations
- Aligning Data Distributions
- Marginal Feature Distribution Alignment
- Class-Conditional Distribution Alignment
- Network Parameter Adaptation
- Parameter Fine-Tuning
- Domain Specific Batch Normalization
- Domain-Specific Network Weights
- Pixel Level Domain Style Transfer
- Self-Based Learning for DA
- Self-Training
- Pseudo-Labels for Adaptive Learning
- Labeling Confidence and Curriculum Learning
- Entropy Minimization
- Self-Ensembling
- Self-Supervision and Test Time Training
- Beyond Classical Domain Adaptation
- Multi-Source Domain Adaptation
- Shallow MSDA
- Deep MSDA
- Multi-Target Domain Adaptation
- Moving Toward Domain Generalization
- Reducing Source Knowledge
- Class Label Mismatch Across Domains
- Partial DA
- Open Set DA
- Universal DA
- Conclusions and More Settings
- Domain Generalization
- Architectural Approaches to DG
- Regularisation Approaches to DG
- Data Augmentation Approaches to DG
- Discussion and Outlook
- Learning to Learn Across Domains
- Learning-to-Learn Preliminaries
- Meta-Learning for Domain Adaptation
- Meta-Learning for Multi-Source UDA
- Discussion
- Meta-Learning for DG
- Multi-Source Domain Generalization
- Meta-Representations for DG
- Discussion
- Cross-Domain Few-Shot Learning
- Cross-Domain Few-Shot Learning Objective
- Meta-Representations for CD-FSL
- Discussion
- Conclusion
- Bibliography
- Authors' Biographies
