Thanks to Hetan Shah and Margot Finn on Bluesky, I came across this article by Gillian Tett in the Financial Times. It’s a discussion about the rhetoric and reality of AI adoption in business with, as you’d expect, a focus on the financial sector and its regulators. The bit that caught my attention was the description of a New York financier evaluating, for the first time, summer interns who had grown up using AI. While they appeared impresive initially,

… when senior financiers later probed their ideas they found them alarmingly shallow.

Consequently this person’s company made fewer return offers and is now focusing less on graduates in science, technology, engineering and mathematics — and more humanities students instead.

Now, I am speaking as a physics professor and a physicist of course. In my experience, there is a lot of critical thinking involved in science, engineering and maths, and I get a bit testy – grumpy even – if I think my humanities colleagues are trying to claim it as their thing.

However, there is also a prevalent “get stuff done” imperative. I learned my ways of working in a world where if you didn’t think critically at the start, you could waste a lot of time. I mean, years. A career even. I suppose the FT equivalent would be buying up sub-prime mortgages.

As I watch the students and postdocs around me exploit the productivity gains of rapidly-improving AI tools to solve problems, write code, test hypotheses, develop and validate new techniques – “get stuff done” – I have two concerns.

One is simply that I can’t compete at certain tasks any more and probably shouldn’t try. This has always happened to professors ever since there have been students, but I enjoy coding so it makes me a bit selfishly sad.

The other concern is reflected in the FT article. I hope it is just a transitional concern, but it is real. It is this: The metrics we use to assess success need to change, and quickly.

It would be possible now to be so “productive” that even if most of your produce was garbage, you could initially out-perform a previous generation who didn’t have access to AI productivity tools. But you could still be “alarmingly shallow” in your understanding of the subject in which you were supposed to be becoming an independent researcher.

The problem is that if speed and efficiency at solving problems is your only USP, what will happen if and when you become responsible for deciding which problems are the most important ones to tackle? If you have never had to think this through carefully, with months or years of potentially wasted effort at stake, will you have the critical thinking and contextual knowledge required?

Maybe you will, maybe this is all fine. There’s certainly a possibility that my worries are on a par with complaining that life’s too easy now we have compilers. Or debuggers. I never even learned assembler, never mind used punch cards, and I’ve done ok.

Is AI really qualitatively different?

Well, lots of people seem to think it is, and the FT article crystalised something that has been bugging me a bit, so I thought it was worth adding to the welter of AI thinkpieces by posting here. Comments, counterpoints welcome, especially from “AI natives”.

This comment came in from Peter Hobson after the comment window had automatically closed…

Hi, I learned to code with punch cards (Algol68 and FORTRAN 4) and did a little assembler for CAMAC (Pascal) in the late 1970s and early 1980s (PhD). For my PhD project I wrote a lot of FORTRAN 4 to produce a UI for the holographic film scanning machine I was building (charm physics) and here I did learn a great deal of real world coding challenges, not lest because of the tiny space, 28k words that any subroutine could occupy without swapping.

Thus I had to look really closely at the logical structure and use of my code blocks and learn ODL which allowed my programmes to be larger than the 11/44 main memory.

I think I learned some very generic coding skills this way, though of course I wouldn’t want to go back to the days with primitive support for the “end product”.

In a different but perhaps related area, I was having dinner at an instrumentation conference with the UK CEO (or possibly CTO) of a well known manufacturer of what I will for reasons of anonymity, just call radiation sensors. At the time I was a Professor in a Department that taught degrees in Electronic Engineering. He confided that his experience of employing UK EE graduates was that their design and simulation skills were good, but when the populated prototype pcb came back it was to French and German graduates they turned to to get the system to work as designed. It was my experience in my UK university that all the practical EE was taught by physicists! My “proper” EE colleagues worked with powerful simulations and FPGA board programming – all good work, but without that hands-on insight you get from wondering why your amplifier design is now a rather poor 125 MHz oscillator in the lab.

A really scary example of shallowness (pre AI boom!) was showing some new MSc students, all with EE degrees, a piece of copper wire and asking them how much DC current they thought it could carry safely. In some cases their shallowness of real-world knowledge let to overestimates of more than 100. I do hope they stuck to simulation where things do not go up in flames.