Continual learning models face distinct challenges based on their fine-tuning regimes, which dictate the trainable parameter subspace. Researchers have found that the fine-tuning regime significantly impacts a model's ability to acquire new tasks while retaining previous knowledge1. By varying the fine-tuning regime, different continual learning problems emerge, each requiring unique solutions. This discovery highlights the importance of considering the fine-tuning regime as a key evaluation variable in comparative studies of continual learning methods. The trainable parameter subspace defines the scope of adaptability for a model, and its configuration can significantly influence the model's performance on subsequent tasks. As a result, practitioners must carefully consider the fine-tuning regime when designing and evaluating continual learning systems, as it can make or break a model's ability to learn and adapt over time. This matters to practitioners because it requires a more nuanced approach to continual learning, one that takes into account the complex interplay between fine-tuning regimes and model performance.