The growing influence of industry in AI research
Co-authors: Nur Ahmed and Muntasir Wahed
Citation: Nur Ahmed et al. ,The growing influence of industry in AI research.Science379,884-886(2023).DOI:10.1126/science.ade2420
For decades, artificial intelligence (AI) research has coexisted in academia and industry, but the balance is tilting toward industry as deep learning, a data-and-compute-driven subfield of AI, has become the leading technology in the field. Industry’s AI successes are easy to see on the news, but those headlines are the heralds of a much larger, more systematic shift as industry increasingly dominates the three key ingredients of modern AI research: computing power, large datasets, and highly skilled researchers. This domination of inputs is translating into AI research outcomes: Industry is becoming more influential in academic publications, cutting-edge models, and key benchmarks. And although these industry investments will benefit consumers, the accompanying research dominance should be a worry for policy-makers around the world because it means that public interest alternatives for important AI tools may become increasingly scarce.
Here’s How to Make Deep Learning More Sustainable
Co-authors: Silviu-Marian Udrescu and Stephan ZhengSubstituting equations for neural network operations can yield whole new efficiencies
The Computational Limits of Deep Learning
Co-authors: Kristjan Greenewald, Keeheon Lee, and Gabriel Manso Citation: Thompson, Neil C., Kristjan Greenewald, Keeheon Lee, and Gabriel F. Manso. 2020. The Computation Limits of Deep Learning. Online at arxiv: https://arxiv.org/abs/2007.05558.
Citation: Thompson, Neil C., Kristjan Greenewald, Keeheon Lee, and Gabriel F. Manso. 2021. Deep Learning’s Diminishing Returns: The Cost of Improvement is Becoming Unsustainable. IEEE Spectrum.
Top 84 most-interesting machine learning articles in 2020
Wired, VentureBeat, Discover, The Next Web, Interesting Engineering, Tech Gig
Deep learning’s recent history has been one of achievement: from triumphing over humans in the game of Go to world-leading performance in image recognition, voice recognition, translation, and other tasks. But this progress has come with a voracious appetite for computing power. This article reports on the computational demands of Deep Learning applications in five prominent application areas and shows that progress in all five is strongly reliant on increases in computing power. Extrapolating forward this reliance reveals that progress along current lines is rapidly becoming economically, technically, and environmentally unsustainable. Thus, continued progress in these applications will require dramatically more computationally-efficient methods, which will either have to come from changes to deep learning or from moving to other machine learning methods.
How Fast do Algorithms Improve ?
Co-authors: Yash Sherry Citation: Sherry, Yash and Neil C. Thompson. 2021. How Fast do Algorithms Improve? Proceedings of the IEEE.
Algorithms are one of the fundamental building blocks of computing. But current evidence about how fast algorithms improve is anecdotal, using small numbers of case studies to extrapolate.
In this work, we gather data from 57 textbooks and more than 1,137 research papers to present the first systematic view of algorithm progress ever assembled. There is enormous variation. Around half of all algorithm families experience little or no improvement. At the other extreme, 13% experience transformative improvements, radically changing how and where they can be used. Overall, we find that, for moderate-sized problems, 30% to 45% of algorithmic families had improvements comparable or greater than those that users experienced from Moore’s Law and other hardware advances
Building the algorithm commons: Who discovered the algorithms that underpin computing in the modern enterprise?
Co-authors: Shuning Ge and Yash Sherry Citation: Thompson, Neil C., Shuning Ge, and Yash M. Sherry. 2021. Building the algorithm commons: Who discovered the algorithms that underpin computing in the modern enterprise? Global Strategy Journal. February 2021 issue, pp 17-33.
Recent work has revealed rapid improvement in the algorithms that underpin modern computing. For many computations, these algorithmic innovations have been more important than those in computer hardware (including Moore’s Law, which is known to have substantially improved firm productivity). In this article, we analyze who built the “Algorithmic Commons”. We find that the United States has been the largest contributor of these public goods, with universities and large private labs (e.g. IBM) playing the biggest role. More broadly, we find a historical pattern of contributions consistent with world geopolitics, where the United States took algorithmic leadership in the post-war period, but that this has faded in recent decades as Europe recovered and then Asia grew.