Journal of Information Technology
Document Type
Research Article
Abstract
The scientific method is based on the rigorous testing of falsifiable conjectures. Data mining, in contrast, puts data before theory by searching for statistical patterns without being constrained by prespecified hypotheses. Artificial intelligence and machine learning systems, for example, often rely on data-mining algorithms to construct models with little or no human guidance. However, a plethora of patterns are inevitable in large data sets, and computer algorithms have no effective way of assessing whether the patterns they unearth are truly useful or meaningless coincidences. While data mining sometimes discovers useful relationships, the data deluge has caused the number of possible patterns that can be discovered relative to the number that are genuinely useful to grow exponentially—which makes it increasingly likely that what data mining unearths is likely to be fool’s gold.
DOI
10.1177/0268396220915600
Recommended Citation
Smith, Gary
(2020)
"Data mining fool’s gold,"
Journal of Information Technology: Vol. 35:
Iss.
3, Article 9.
DOI: 10.1177/0268396220915600
Available at:
https://aisel.aisnet.org/jit/vol35/iss3/9