If you are interested in smart software, you may want to read “Machine Learning: the High Interest Credit Card of Technical Debt.” I like the credit card analogy. It combines big costs with what some folks see as a something-for-nothing feature of the modern world. … The paper identifies specific cost points which most MBAs happily ignore or downplay in post mortems of failed search and content processing companies. The whiz kids, both boys and girls, rationalize their failure to deal with shifting boundaries, “dark dependencies,” expensive spaghetti, and the tendency of smart software to sort of drift off center.
Abstract: Machine learning offers a fantastically powerful toolkit for building complex systems quickly. This paper argues that it is dangerous to think of these quick wins as coming for free. Using the framework of technical debt, we note that it is remarkably easy to incur massive ongoing maintenance costs at the system level when applying machine learning. The goal of this paper is highlight several machine learning specific risk factors and design patterns to be avoided or refactored where possible. These include boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, changes in the external world, and a variety of system-level anti-patterns.
Phi Beta Iota: The human factor continues to be denigrated despite its vast superiority to machines. This is because the techno-industrial-financial complex understands how to spend vast amounts of money on technical projects for which accountability is rarely achieved, and they do not understand how to spend small amounts of money empowering humans at the edges of the network.