An Adaptive Training Method for Continual Semantic Segmentation with Task Boundary Detection

摘要

Background and Motivation: Continual Semantic Segmentation (CSS) aims to enable models to sequentially learn new knowledge in dynamic environments, akin to human learning, without forgetting previously acquired capabilities. However, existing methods commonly face the dual challenges of "catastrophic forgetting" and "semantic drift." The root cause lies in the model's inability to distinguish the boundaries between new and old tasks, preventing adaptive adjustments in learning strategies and resulting in a trade-off between stability and plasticity.Method: This paper proposes an Adaptive Training method for Continual Semantic Segmentation with Task Boundary Detection (AT-TBD). The core of this method is the introduction of a Task Boundary Detection (TBD) module, which dynamically determines if a new task has been introduced into the data stream by monitoring changes in the entropy of the model's prediction outputs.Implementation: Based on the judgments from the TBD module, the model adaptively switches its training strategy between a "stable learning phase" and a "boundary adaptation phase." During the stable learning phase, the model focuses on optimizing performance for the current task. When a task boundary is detected, it switches to the boundary adaptation phase, prioritizing the consolidation of old knowledge through techniques such as parameter freezing and enhanced knowledge distillation, thereby achieving intelligent control over the learning pace.Core Conclusion: Experimental results on the PASCAL VOC 2012 and ADE20K datasets demonstrate that the proposed AT-TBD method significantly outperforms several state-of-the-art CSS baseline methods across multiple continual learning scenarios. It shows particular strength in mitigating catastrophic forgetting, achieving a 3% to 5% improvement in mean Intersection over Union (mIoU).Significance and Value: This research offers a new perspective on resolving the stability-plasticity dilemma in continual learning. It proves that explicitly perceiving task changes and adaptively adjusting learning strategies can more effectively manage the model's knowledge update process. This method not only enhances the performance and robustness of CSS models but also provides theoretical and practical insights for building more intelligent lifelong learning systems that more closely mimic human learning mechanisms.