When building AI, is simpler better? New research challenges assumptions

When trying to solve problems, artificial intelligence often uses neural networks to process data and make decisions in a way that mimics the human brain.

In his latest research, Binghamton University Assistant Professor Sadamori Kojaku challenges a fundamental assumption in AI circles—that more complex neural networks are always better.

The paper, published in Nature Communications, shows that simple neural networks can find communities in complex networks with theoretical optimality, questioning the common view that more complex models outperform simpler ones.

“What we found was that the training matters, not the programming architecture itself,” said Kojaku, who joined the faculty of the Thomas J. Watson College of Engineering and Applied Science’s School of Systems Science and Industrial Engineering in Fall 2023.

“There are many ways to teach a neural network, but we found that one best teaching method is contrastive learning, where you present real data and fake data so the neural network is trained to differentiate the two. This simple training achieves optimal performance.”

Understanding how AIs work is fundamental to establishing trust when it makes decisions in critical areas such as health care or electrical grids.

Right now, the exact route that AIs use to get their conclusions is inside what programmers call a “black box.” Data input leads to a result, but the pathway between those points can be mysterious.

“Our work unboxes the neural networks and then tries to interpret how it works to provide a guarantee that this neural network works optimally for this specific task,” Kojaku said. “This is our first work that tries to hammer on the black box.”

Also contributing to the paper are Professors Filippo Radicchi, Yong-Yeol Ahn and Santo Fortunato from Indiana University, where Kojaku served as a postdoctoral fellow after earning his Ph.D. at Hokkaido University in Japan and before coming to Binghamton.