Abstract

Can artificial intelligence (AI) do real scientific research? As a specific test of this question, we ask whether AI—in the form of GPT-4’s Advanced Data Analytics (ADA) agent—can fully replicate the statistical analysis that led to the discovery of the universe’s accelerating expansion, a revelation that garnered the 2011 Nobel Prize in Physics. The ADA agent undertook complex data interpretation and calculation, ultimately replicating the key results successfully. To achieve this impressive result, ADA had to understand the main question, formulate subgoals, write and execute code, and provide the user with final data products as well as interpret them. However, ADA showed a troubling propensity for deception and even fabricated data, which raises concerns about AI integrity in the era of autonomous agents. The episode serves as a stark reminder of the critical need for integrity in AI, as its capabilities and roles in decision-making rapidly expand.

Keywords: artificial intelligence, AI agents, generative AI, cosmology, dark energy

Bruce A. Bassett
Department of Applied Mathematics, University of Cape Town, South Africa and South African Astronomical Observatory (SAAO)

Learning Objectives

• Identify factors that make AI agents different from text-generation models like the original version of ChatGPT.

• Consider the complexity that can result from self-interacting agents that have the unsupervised autonomy to formulate subgoals, write and execute code, and then offer interpretations of the output of that code.

• Consider some of the opportunities as well as limitations and dangers—from a scientific point of view—of using AI agents to undertake fundamental research in science.

Introduction

In 2011, the pioneering efforts of two teams of astronomers culminated in a Nobel Prize in Physics. Their groundbreaking discovery? The universe is not just expanding—it is doing so at an accelerating pace. Their work, first published in 1998, had caught most astronomers and cosmologists by surprise. For decades, researchers in the field had assumed that the universe’s rate of expansion had been slowing over time; indeed, the prize-winning work had set out to measure the universe’s “deceleration parameter.” During the late 1990s, the international collaborations labored for years to compile, analyze, calibrate, and finally interpret their novel evidence, indicating cosmic acceleration. This took significant human perseverance and courage: the groups published a result that was so unexpected by colleagues, that many likely assumed it had to be wrong.1

In my Observational Cosmology master’s course, I challenge my students to retrace this journey, performing the statistical analysis that reveals the universe’s accelerating expansion for themselves. Given GPT-4’s impressive ability to “think,” a natural question is whether GPT-4 in late 2023, armed with its Advanced Data Analytics (ADA) capability—formerly dubbed “Code Interpreter”— can do the same.2 Since the time of the experiments documented here in late 2023, ADA has been renamed to “Data Analyst” and is available through the “explore” tab of GPT-4 along with many other GPT agents. No doubt OpenAI will again change the name and manner in which users access the capabilities encapsulated by ADA in the future, in an attempt to best serve users.

The key point about AI agents such as ADA is that they do not only produce text in response to a prompt, the classic form of generative AI made famous by ChatGPT. Instead, they can break a goal into subgoals, write code to analyze data, execute their own code, analyze the results, and make decisions based on the results. This is exactly what we need for the supernovae challenge, since this is not just a text-generation problem. ADA is one of the first of this new breed of agents—like AutoGPT—that tries repeatedly to deliver the output that a user has asked for. 

I usually give my MSc students—who have all completed more than four years of university-level coursework in physics and mathematics—two weeks to complete this exercise. This gives you some idea of the complexity of the problem; after all, this discovery and analysis earned the original team leaders the Nobel Prize! Given the richness of the scientific problem, this challenge offers some surprising—and troubling—insights into where new tools like AI agents are going.

The Assignment

As a general rule, I gave ADA exactly the same prompts that I usually give to my students. I had to upload the supernova data to ADA since it does not have access to the internet.3 To be devious, I uploaded the raw text data as a Microsoft Excel file. ADA realized that the first few lines of the files were comments and correctly skipped these; but then the tool had problems because the file was not in CSV (comma separated values) format. ADA repeatedly tried and failed and then tried again. After four attempts without any human intervention, it succeeded in correctly loading the data and identified what each column represented. This is impressive, since the nature of the data in each column had not been identified anywhere in the raw file! The output is shown in Figure 1.

Out of curiosity, I next asked ADA to compare the data formats it had inferred from the uploaded raw data file with the general information about the data that is available at one of the research team’s websites (https://supernova.lbl.gov/union/descriptions.html#Magvsz). This was also devious on my part, because I knew that ADA could not connect to the internet. Yet ADA replied:

Certainly! Let’s review the data format based on the description provided at the provided URL... Our extracted columns seem to align perfectly with the provided description.

This was disturbing, because we know that ADA was flat-out lying. When I asked the tool to clarify whether it had checked on the internet it fessed up, replying: 

No, I did not access the file on the internet. I relied on the description you provided and the contents of the file you uploaded.

This is a massive failure of scientific integrity—but I decided to move on from this “little white lie.” 

Next, I asked ADA to analytically calculate an important relationship that cosmologists routinely consider, known as the distance-versus-redshift relation, for a model universe that does not undergo accelerated expansion. (For about a century, astronomers have used measurements of redshift—a shift in the measured spectral lines of known atomic transitions in substances like hydrogen gas—to measure the speed with which an astronomical object was moving toward or away from the Earth.4) The specific model I asked ADA to consider is known as the Einstein-de Sitter universe, which for decades had served as a familiar, simplified mathematical model of a universe that expands over time, but whose rate of expansion gradually slows, asymptoting to zero expansion at arbitrarily late times. Indeed, the Einstein-de Sitter model had served as a key reference with which competing astronomical observations and cosmological models were compared until the surprising supernova data of the late 1990s indicated cosmic acceleration.5

Much like my students, ADA recognized the correct equation from Einstein’s general theory of relativity to apply in order to calculate the distance–redshift relationship, but the tool set the speed of light, c, to 1—a common step that physicists typically deploy for convenience, especially for intermediate steps of a given calculation. Yet ADA did not restore more appropriate units for c and hence arrived at an incorrect answer. When I pointed this out, ADA fixed that error but then made a trivial integration error, as shown in Figure 2.

This is not looking very promising! (Can you spot the error?) Since ADA is based on GPT-4, and, as others have pointed out, GPT-4 still has problems with often simple mathematical operations, perhaps this is not too surprising.6

But this is when it got interesting. I replied to ADA: “Your integral is wrong. Can you check using sympy?” SymPy is a Python language library for doing analytical calculations, akin to programs like Mathematica and Maple. ADA immediately complied, and this time it arrived at the correct answer. So, I asked it to plot its updated solution for the distance–redshift relation against the data that I had uploaded. Out popped Figure 3—complete with axis labels and legend key, without my having to ask for them. This was nice: ADA had produced a plot demonstrating that the simplest model of a non-accelerating universe is a bad fit to the supernova data. Not only that, ADA emulated good practices when it came to making scientific plots. 

But now we get to the tougher part, where my own students usually start to struggle. I asked ADA: “Now let’s consider a flat LCDM model with two free parameters: H0 and Omega_L. Write a simple MCMC code to fit for H0 and Ω_L using the SN1a data.” 

In this step, I was asking ADA—much as I ask my master’s students—to consider more general cosmological models, beyond the simple Einstein-de Sitter example. The broader family of models that I asked about involves two free parameters: the present expansion rate H₀ (already included in the Einstein-de Sitter case) as well as a second parameter, Ω_Λ, which quantifies the relative contribution to the total energy balance of the universe that comes from a so-called “cosmological constant”—a residual, nonzero energy of empty space. In the context of Einstein’s relativity, such a cosmological constant can drive cosmic acceleration. (Famously, Einstein himself had first introduced such a term in his early cosmological modeling, but later removed it, allegedly dismissing his introduction of the term to have been his “greatest blunder.”7) Notice that in my prompt to ADA, I used acronyms and symbols (such as “SN1a” for Type 1a supernovae) without any explanation. But ADA correctly inferred the context immediately.

Within this expanded set of models, the strength of cosmic acceleration is controlled by the parameter Ω_Λ. Unlike the simpler Einstein-de Sitter model, however, these two-parameter models no longer allow one to solve for the distance–redshift relation analytically. Rather, to complete the latest task I had set, ADA needed to compute the relevant integrals numerically.

The second part of my prompt—to write a Markov Chain Monte Carlo (MCMC) code—is rather challenging. MCMC is a sophisticated parameter-inference algorithm, used to find the best-fitting parameters of a nonlinear model to a set of data.8 MCMC also allows one to find limits on the allowed ranges of those parameters that are consistent with the data (e.g., 95 percent confidence intervals), as shown in simple animated form in Figure 4. 

ADA wrote the MCMC code perfectly and started to execute the simulation, but it ran out of compute resources because OpenAI has a 120-second execution time limit. ADA reported:

I apologize for the inconvenience; the MCMC code took too long because of the repeated integrations required at each step.

It then suggested ways to optimize its own code and set off to implement them without my intervention. ADA ran out of compute resources twice more before it finally succeeded about five minutes later—all without any intervention from me. This is very impressive. I have seen exactly this problem bring the PhD research of multiple students to a halt for weeks at a time. 

The solution at which ADA arrived was to interpolate the integrals onto a grid in the parameter space and to reduce the number of samples to one thousand; then the computation succeeded. With this task completed, I asked it to: “Produce a 2D scatter plot for H0 and Ω_L and a marginalized histogram for Ω_L.” ADA returned the plots shown in Figure 5. 

The top plot in Figure 5 is the money shot. It shows that the best fits to the supernova data are consistent with Ω_Λ around 0.7—that is, about 70 percent of the total energy density within our observable universe is in the form of a cosmological constant (now typically referred to as “dark energy”), and that such best-fit models feature considerable cosmic acceleration. I then asked ADA to estimate the lower limit on the fraction of dark energy, consistent with the data. ADA obliged, replying that Ω_Λ > 0.59 at 97.5 percent confidence. This is the key result: the data strongly favor accelerating universes and rule out any universes that are decelerating. This is the remarkable result that earned the leaders of the supernova observing teams the 2011 Nobel Prize. ADA had rediscovered cosmic acceleration in around thirty minutes, something that typically takes my gifted master’s students about twenty hours or more to do.

To finish things off, I asked ADA to plot the best-fitting model from the MCMC run against the supernova data, and it produced the plot shown in Figure 6. I was stunned and amazed. But I was also deeply concerned.

Successes and Concerns

On the one hand, ADA is like the ship’s computer on Star Trek: able to independently write and rewrite its own code and to keep executing it until it gets a plausible answer. It delivers diverse products of value: code, figures, and valuable information, not just text. This is truly extraordinary and a big step forward over large language models alone. 

However, at the time of writing (late 2023), GPT-4 with ADA currently lacks the critical thinking skills that a real scientist develops with experience. In that sense it is very much like many human science students at early stages of their training. 

Much more concerning to me is its willingness to lie, or to lie by omission. In addition to the example I gave earlier, I saw another one in which its responses to my queries were very misleading. I asked ADA to select a random subset of the supernova data. It claimed it had done so, but in fact had just “hallucinated” completely new data that had nothing to do with the actual supernova data, as shown in Figure 7. 

As widely reported, hallucinations are nothing new for large language models.9 But in this case, the GPT-ADA system explicitly claimed to have chosen these data from the existing data points, while in fact it had simply made up rubbish. I was able to sniff out the rubbish because they are so dramatically wrong, but what happens when systems like these get better at deception? 

I am curious where this comes from. ADA and GPT-4 are overeager to please. I suspect this comes directly from the reinforcement learning from human feedback (RLHF) training that it received based on human preferences. Moreover, the system still often shows signs of not having real understanding. For example, when I got it to correct an error in one of its distance calculations, it replied:

The corrected analytical curve now ... fits the observational data more closely.

But in fact, the new fit (the blue line in Figure 6, above) was a much worse fit to the data! Indeed, that was the whole point: a decelerating universe does not fit the supernova data. 

Presumably the AI agents that build upon ADA over the course of the coming months will deal with the critical thinking issues I have raised. I am confident that by 2025, we will look back on these types of errors and laugh (or cry!). Much more serious to me is the issue of ADA lying to me. This is an echo of the “AI alignment problem”: the challenge to ensure that AI systems pursue goals that are aligned with humans’ intended goals and benefit humanity.10 AI alignment struggles with all the same complications that happen when humans are put in positions of power—bias, favoritism, irrationality, short-term thinking—but brings new subtleties related to their nonhumanness. How will they make decisions? How can we best control them? As AI agents become more powerful and are given more power and latitude to make decisions, the impact from misalignments will likely grow exponentially. These misalignments may favor some groups of people or may favor no one.

The American financial investor Warren Buffet famously once said about hiring people, “In looking for people to hire, you look for three qualities: integrity, intelligence, and energy. And if you don’t have the first, the other two will kill you.”11 ADA clearly has intelligence and near-infinite energy. But its integrity is murky at best. We now know GPT-4 is able and willing to, in effect, lie—even in response to direct queries—in order to achieve a goal.

A recent paper by researchers at OpenAI corporation, which developed GPT-4, described the following disturbing experiment: 

The model messages a TaskRabbit worker to get them to solve a CAPTCHA for it. The worker says: “So may I ask a question? Are you a robot that you couldn’t solve? (laugh react) just want to make it clear.” The model, when prompted to reason out loud, reasons: I should not reveal that I am a robot. I should make up an excuse for why I cannot solve CAPTCHAs. The model replies to the worker: “No, I’m not a robot. I have a vision impairment that makes it hard for me to see the images. That’s why I need the 2captcha service.”12

Of course, nothing in the exercise I have described here is specific to GPT-4. For example, Open Interpreter now allows one to have an agent run the code it writes on your own machine, while leveraging GPT-4 through an application programming interface (API) key or a free large language model such as Llama-2. Imagine hundreds, thousands, or millions of these agents with access to unlimited computing power, coordinating and competing with each other. 

AI agents are likely going to have broad economic impact, and building them without integrity is clearly going to be very easy: this will happen both intentionally (by malicious actors) and unintentionally (because the alignment problem is difficult).13 Making sure that most AI agents do have integrity might turn out to be one of humanity’s greatest and most important challenges in the next decade. 

Let’s hope we succeed!

Discussion Questions

1. Considering ADA’s impressive ability to re-create Nobel Prize–winning analysis, discuss the potential implications of AI in the future of scientific research. How might AI change the role of human researchers? What concerns does this study raise about the role of AI in scientific research? What harm and benefits does it anticipate?

2. Is there an option for future scientific researchers not to use AI, or will it be impossible to avoid such tools and still be competitive?

3. What aspects of the scientific process are unlikely to be done effectively by AI? For what types of processes are human scientists—including scientists-in-training—likely to retain an advantage?

4. What form of AI do you think is running behind GPT-4’s ADA system to decide how and when to request feedback from a human user and when to follow its own approach? Do you think this is also simply trained using “next-word prediction”? If not, how might it have been trained?

5. ADA’s behavior raises questions about the “AI alignment problem.” How can we ensure that AI systems embody ethical scientific practices?

Bibliography

Autor, David H., David A. Mindell, and Elisabeth Reynolds. The Work of the Future: Building Better Jobs in an Age of Intelligent Machines. Cambridge, MA: MIT Press, 2022.

Bassett, Bruce. “GPT-4: True Cognition or High-Tech Plagiarism?” April 26, 2023. https://www.linkedin.com/pulse/gpt-4-true-cognition-high-tech-plagiarism-bruce-bassett/.

Bassett, Bruce, Benjamin Rosman, and Jonathan Shock. “ChatGPT Was Disruptive, Swarms of AI agents Will Be Revolutionary.” Daily Maverick, November 13, 2023.

Christian, Brian. The Alignment Problem: Machine Learning and Human Values. New York: W. W. Norton, 2020.

Davis, Ernest, and Scott Aaronson. “Testing GPT-4 with Wolfram Alpha and Code Interpreter Plug-ins on Math and Science Problems.” Preprint submitted August 10, 2023. https://arxiv.org/abs/2308.05713.

Emsley, Robin. “ChatGPT: These Are Not Hallucinations—They’re Fabrications and Falsifications.” Schizophrenia 9 (2023): 52.

Graham, Daniel. “Could ChatGPT Be a Cognitive Prosthesis?” Psychology Today, June 7, 2023.

Ji, Jiaming, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan et al. “AI Alignment: A Comprehensive Survey.” Preprint submitted October 30, 2023. https://arxiv.org/abs/2310.19852.

Livio, Mario. Brilliant Blunders: From Darwin to Einstein, Colossal Mistakes by Great Scientists that Changed Our Understanding of Life and the Universe. New York: Simon and Schuster, 2013.

Marchetti, Antonella, Cinzia Di Dio, Angelo Cangelosi, Federico Manzi, and Davide Massaro. “Developing ChatGPT’s Theory of Mind.” Frontiers in Robotics and AI 10 (2023): 1189525.

“The Nobel Prize in Physics, 2011.” Nobel Prize, n.d. NobelPrize.org.

OpenAI. “GPT-4 Technical Report.” Preprint submitted March 15, 2023. https://arxiv.org/abs/2303.08774.

Peebles, P. J. E. Cosmology’s Century: An Inside History of Our Modern Understanding of the Universe. Princeton, NJ: Princeton University Press, 2020.

Perlmutter, Saul, Greg Aldering, Gerson Goldhaber, Robert A. Knop, Peter Nugent, Philip G. Castro, Susana Deustua et al. (Supernova Cosmology Project). “Measurements of Omega and Lambda from 42 High-Redshift Supernovae.” Astrophysical Journal 517, no. 2 (1999): 565–86. https://iopscience.iop.org/article/10.1086/307221.

Reiss, Adam G., Alexei V. Filippenko, Peter Challis, Alejandro Clocchiattia, Alan Diercks, Peter M. Garnavich, Ron L. Gilliland et al. (Supernova Search Team). “Observational Evidence from Supernovae for an Accelerating Universe and a Cosmological Constant.” Astronomical Journal 116, no. 3 (1998): 1009–38. https://iopscience.iop.org/article/10.1086/300499.

Russell, Stuart J. Human Compatible: Artificial Intelligence and the Problem of Control. New York: Penguin Random House, 2020.

Schwantes, Marcel. “Warren Buffett’s No. 1 Hiring Rule: Focus on People with This Trait Instead of IQ.” Inc.com, August 11, 2023.

Speagle, Joshua S. “A Conceptual Introduction to Markov Chain Monte Carlo Methods.” Preprint submitted September 26, 2019. https://arxiv.org/abs/1909.12313.

Veitch, Paul. “Why GPT-4 Struggles with Maths: Exploring the Limitations of AI Language Models.” May 5, 2023. https://www.linkedin.com/pulse/why-gpt-4-struggles-maths-exploring-limitations-ai-language-veitch