Current generative AI research is heavily reliant on brute-force scaling with ever-larger datasets and computing power, raising questions about sustainability and future data limitations. There is a call for architectural innovation enabling AI to learn efficiently from modest data,4 drawing from concepts in human learning and cognitive science.
Approaches to cope with data limitations include generating synthetic data, employing new data diversification algorithms5,6 and developing smarter, data-efficient learning paradigms. However, progress in generative AI must overcome issues with robustness and generalisation by moving from “vanilla” data synthesis to creating data that does not yet exist online, and by exploring alternative architectures to transformers. The field must also avoid over-fixation on current trends (brute-force scaling) and foster unconventional collaborations, especially between academia and industry, to drive fundamental change.
It is important to emphasise that ethical considerations, safety and alignment of generative models with societal values should be foundational to further adoption.7 It is also important to keep technical progress aligned with societal needs and to raise critical questions about long-term future directions of the field.8
