‘We must be able to trust artificial intelligence’

Nil Ayday is a scholarship holder of the Konrad Zuse School of Excellence in Reliable AI (relAI).

Attracting young, international talent is vital to Germany’s success as a hotbed of AI innovation. Three DAAD-funded graduate schools provide the perfect conditions for study and research and a gateway to a career in Germany. We took a tour of the Zuse School relAI in Munich with master’s degree ­student Nil Ayday where she told us about her plans.

Technical University of Munich’s (TUM) natural sciences and engineering campus is located in Garching, just to the north of Munich. Nil ­Ayday gets there on the metro. ‘It’s so easy to get from one place to another here,’ says the ­master’s degree student, laughing. ‘Nothing like Istanbul!’

Nil Ayday was 17 when she came to study in ­Germany. Her parents encouraged her to study engineering and she enrolled on a bachelor’s ­degree course in electrical and information technology. ‘Back then I had no idea what arti­ficial intelligence was,’ the 23-year-old tells us. The talented mathematician’s interested was aroused by one of her professors. ‘All of a sudden I understood that artificial intelligence is no more than maths and statistics, and I love working with those,’ she says, so she decided to dig into the topic more deeply.

On her master’s degree programme at the ­Konrad Zuse School of Excellence in Reliable AI (relAI), Nil is now grappling with questions of trust and security in artificial intelligence. 
The graduate school is one of three Konrad Zuse Schools for artificial intelligence which the DAAD supports with funding from the Federal Ministry for Education and Research. Zuse School relAI  is a joint project between TUM and LMU Munich, with activities taking place at both sites. Nil is one of the first cohort of relAI students who started their studies in September 2022. ‘Women are still in the minority in AI research,’ she tells us. ‘But that was also one of the reasons why I applied for the programme.’

Responsible AI

Nil Ayday spends most of her time at the extensive research centre campus in Garching. On our way to the Mathematics and IT building, Nil runs into some friends, some of whom she has known since her bachelor’s degree days, and they arrange to meet for lunch. ‘It’s a green campus,’ she tells us as we approach the building, where courses in her master’s degree course in robotics, cognition and intelligence are taught. On our way she points out a small park. ‘We ­often hang out there in the summer,’ she tells us. We ask what it was about the concept of a Zuse School that particularly caught her interest. ‘I can attend courses, lectures and events at both universities which are designed around the topic of trustworthy AI,’ she says. Combined with the IT focus of her master’s degree programme, she says it’s a perfect fit.

When it’s time to study, Nil Ayday likes to use the Zuse School rooms at the Munich Data Science Institute (MDSI). Located on the fifth floor above the new conference centre building at the heart of the Garching Campus, MDSI is home to the relAI graduate school. ‘I can always find a space here where I can really concentrate on my work,’ she says, sitting down on a brightly coloured cube in the lounge. In October 2023 50-or-so Zuse School master’s degree and PhD students attended the second Welcome Day where they got to know one another. ‘They really encouraged us to connect and talk to one ­another,’ says Nil. What she particularly values, she says, is that ‘We all come from different pathways and we all benefit from one another’s research.’

AI research is about questioning a result, even if it delivers correct values.

The Clever Hans effect

Nil Ayday opens her laptop and a conference poster appears on her screen. It displays the results of her research into explainability for ­improving model reliability at the Fraunhofer Institute for Telecommunications in Berlin, where she works as a research assistant. She points at the black-and-white photo of a horse, which seems out of place next to the complex ­diagrams and models. It shows the horse known as Clever Hans, which was famous for his supposed ability to do sums. When given a sum to complete, Hans would tap his hoof the right number of times. In reality, though, Hans was not a horse of exceptional cognitive ability: he was just extremely good at observing others. By noting the unconscious hints given off by people around him, he knew when he had reached the right answer and stopped tapping.

In AI research, the ‘Clever Hans Effect’ is used to refer to a phenomenon in machine learning. ‘It’s about digging into results, even if they give you the right answer, as that may be based on false correlations and unwanted assumptions in the training data,’ Ayday explains. Techniques used in ‘explainable AI’ cast light on how this happens. ‘In medical applications, for example, it’s vital to be able to explain how AI models reach certain conclusions,’ Ayday adds.

Gil Ayday tauscht sich mit einer Koordinatorin der Zuse School relAI an der TU München aus.c

Leading international partners

Nil Ayday has presented her poster several times, such as at a meeting in Dresden of all three Zuse Schools and the major opening event at the Munich Residence in July 2023, attended by high-profile guests from the worlds of politics, business and science in attendance. Through the Zuse School the two universities work with both regional and leading international partners. ‘One visitor was really scrutinising my poster and she asked me a really demanding question. I was so excited,’ says Nil. It turned out that the person showing an interest in her poster was conducting research into a similar area for Google.

Nil Ayday is one of the next generation of AI ­researchers. She explains how, ‘Having access to events like this and being able to talk to people from fascinating companies is fantastic.’ Some of her fellow relAI students are throwing themselves into these opportunities, but Nil Ayday has other plans. For now she plans to stay in academia after completing her master’s degree to do a PhD in theoretical AI at TU Munich. And then? Nil Ayday laughs. That’s too far in the ­future to say, she replies, ‘But who knows!’ She would certainly be open to the idea of an internship during her PhD. ‘Maybe that would give me an idea of the direction I could go in,’ she says, but after five years in Munich, she feels right at home here, too.

This article was first published in the DAAD Annual Report 2023.

Miriam Hoffmeyer (30 June 2024)


 

Related Topics

DAAD - Deutscher Akademischer Austauschdienst - German Academic Exchange Service