Many organizations have shipping products, generating ample revenue with a sizable customer base. A majority of such products have system software as the building block that is stable to a high degree and hardly needs feature enhancements. However, one topic that keeps getting discussed in the context of such products is Sustenance. Teams keep grappling with customer escalations related to operating system upgrades, newer versions of system libraries, security fixes, etc.
My discussions with engineering leaders and business executives lead to below top three reasons why it has become a hard problem to solve today:

  1. Complexity: System software implementations have multiple components and dependencies. Usually, C/C++ is the programming language used for compatibility and performance. It allows developers to write complex software systems. Without deep understanding and knowledge, the code is difficult to understand and modify.
  2. Legacy Code: The software is written many years ago. Original developers may no longer be available. This makes it difficult to understand the code and make changes without introducing new bugs or breaking existing functionality. Moreover, documentation is rare.
  3. Portability: The code is not always portable between different platforms and architectures. Re-compilation is needed with suitable platform-dependent modifications. This needs a build farm, which can be imagined as an ever-growing rack of servers to build the code and maintain released versions with hotfixes and patches.

Thus, stable, revenue-generating products have their own share of good to have problems. My experience has taught me that one of the effective solutions is to have a team of engineers with deep expertise and a tenacious, persevering attitude.