The potential of generative AI is immense, but while there are many exciting avenues for software developers to explore, any new technology should be handled with an appropriate amount of caution. 

Ahead of her talk at LeadDev Berlin, Scott Carey (SC), checked in with Principal Consultant at Thoughtworks, Birgitta Böeckeler (BB), to hear more about the realities of dealing with AI.  

The below conversation has been edited for clarity and brevity. Birgitta’s full talk will be delivered at LeadDev Berlin on December 6.  

SC: What will you be talking about at LeadDev Berlin later this year? 

BB: The title of my talk is AI for software development: A reality check and I will mostly be focusing on generative AI and large language models (LLMs). 

I want to talk about what I'm currently seeing in my role and speaking about with my colleagues and clients across the globe. I want to also talk a bit about what has potential and is working well, and also the things that may sound good in principle, but present a lot of issues after having scratched at the surface.  

SC: What are some of the typical things that people ask you about generative AI? Is there anything they are concerned about or excited by?

BB: The AI trend has really resurfaced on a lot of the age-old software delivery questions. 

Most of our clients ask us how they can assess the effectiveness of coding assistance tools like GitHub co-pilot and the implementation of generative AI. This is the most frequently asked question, but it's not a question that is specific to AI or introducing an AI tool. Software delivery: how do we measure if we're effective? is a perennial question. 

An aspect that clients need to think about is if they are looking to leverage ChatGPT and other LLMs to make software delivery more effective, they have to understand how the models work. This requires you to think about how highly effective teams function, where the sources of waste are, and how LLMs can actually help. That's something that people don't ask about as much, but should.

Finally, people are raising questions about creating software. A lot of people right now are focusing on coding assistants and the potential of creating code 50% faster with AI. But in reality, writing code constitutes less than 40-30% of what a team does.  

SC: Is generative AI going to reduce work or make it easier to be a software engineering manager?

BB: If you compare how we build software today, and deliver it, versus how we did 20 years ago, it's like a huge boost. But we’ve never run out of work because software is being used for so many things now; there are always more things to build, there are always more things to make secure, and there are always more ways to make things repeatable. 

I don’t believe that LLMs are the new category of tools that can help us with this. Historically, we've tried to make ourselves more effective, raising the abstraction level and reducing the level of technical details we have to deal with. But then we always hit a ceiling because we need more control and we want more customization, which, then calls for us to deal with details again. 

To me, generative AI is not there to raise that abstraction level but rather to assist us in doing so. For instance, helping with writing Java code or creating low-code applications to help make engineers more effective.

SC: Do the skills of being an effective engineer or leader always stay the same?

BB: For as long as we will need to collaborate to create software, certain values and principles will probably stay the same. It will always be multiple people and stakeholders who deliver products. And as long as we need to collaborate as humans, there will be complexities for which engineering leaders are necessary. 

One of the frequently asked questions from our clients is whether AI can replace pair programming. But pair programming is actually a practice to make the team better as a whole and not just an individual coder. So while AI-based coding assistants can definitely take some of the weight off pair programming, in the end, it's all about the collaboration, the collective code ownership, and the flow of the team. So getting AI to do it is not the same. It's a team sport. 

SC: What do you hope people will take away from your talk?

BB: My goal is to give a bit of a reality check. At the moment, because it's such a big hype, there are two extreme ways people react to this, and then some people in between. 

On one end of the spectrum are people who are very skeptical i.e., those who think AI isn’t actually helping at all, and I hope that this talk can open up these people a bit more to the potential. 

On the other end of the spectrum, there are a lot of people who are very excited about this, who believe AI can be applied to everything and make teams 50% faster. I would maybe like to bring those people down to Earth a little with more realistic expectations.