A welcome to LeadDev London 2024 from the host Tanya Reilly

Anchoring on a case study about driving observability platform adoption in a multi-product organization, we’ll review the organizational implications that impeded adoption.

After providing some context on the problem space (production readiness in the context of General Election Preparation), we’ll walk through the blameless post-mortem analysis of where our technical vision and strategy fell short due to organizational issues we overlooked. More specifically, we’ll review organizational issues like top-down and bottom-up buy-in, organizational structure, technical processes, and culture around blamelessness and accountability.

The growing importance of streaming applications in today’s tech landscape is driven by the need for real-time data processing, scalability, enhanced user experiences, and competitive advantage.

However, developing these applications is far from trivial because of the engineering challenges—ranging from subscribing to various data sources, defining complex streaming transformations, to integrating with multiple data sinks. 

To address the complexities of building these applications, a declarative interface offers a robust solution. A declarative approach, where developers specify what to achieve rather than how, simplifies the development process by making code more modular, reusable, and understandable across different teams. It also enhances system maintainability and debuggability. 

We pioneered a declarative interface rooted in GraphQL at Airbnb to significantly ease and standardize the development of streaming applications. This methodology has been embraced by over 20 teams within Airbnb for them to more easily develop, maintain, and scale streaming applications, staying ahead in the fast-paced world of real-time data.

While reflecting on my journey to becoming a staff engineer, I realized many of the practices I apply to my day-to-day work come from my time working with open source software maintainers, as a contributor and fellow maintainer.

Guiding OSS projects can teach us about scaling ourselves, influencing technical direction, and collaborating effectively across disparate team members.

In this presentation, I’ll share the lessons learned by me and other open source maintainers that impact how we perform as staff+ engineers and leaders today.

Staff engineers can make for an excellent advance party, someone you give a problem like “build a new product” and know they’ll turn that ask into an actionable plan, eventually releasing something great.

Sounds exciting, right? But if you’re the person given this massive – and often extremely important – ask, you’ve got some hard work ahead of you. 

You may be the first to scope out the project, but you’ll need to build a team to help you deliver on it. Once you have a team you’ll need a vision that can excite and inspire them, as it’s unlikely you’ll deliver the quality of outcome within the timelines needed without one. Complicating things, it’s probably a poor choice for you to take an ‘official’ leadership position within a team: you’ll need to steer, support and drive the team all from within. 

I’ve been this person many times over, and I know how big a challenge it can be. But if you’re the type of staff engineer who has deep technical expertise, is unafraid of big problems and knows the business well, the impact you can have doing this type of work can be company-altering. 

In this talk, I’ll share what I’ve learned from my experiences bootstrapping teams. I’ll draw from my experience at GoCardless as a Principal Engineer when leading efforts to build a new Open-Banking payment scheme, and more recently as the engineer who helped go from zero-to-release of the [incident.io](http://incident.io) Status Pages and Catalog products. 

I’ll cover what worked and what – sometimes painfully – didn’t, and ask whether the most important outcome of this work is the product or the team you end up building around it.

Wherever you are in the organization, you can help foster inclusivity, but your opportunities for doing so increase exponentially with your level.

Your responsibility to work toward this change also increases dramatically with your privilege. I’ll talk through some concrete ways to magnify the change you effect in your work while doing things you already do anyway: 

• Individual change: onboarding new interviewers, being a good example of inclusive meeting facilitation 
• Team change: formalizing equitable access to mentorship and opportunities
• Org change: not just joining or creating Diversity/Equity/Inclusivity groups, but also bringing software solutions to recruiting & people teams, start cross-team initiatives i.e. accessibility, bringing equity considerations to data analysis conversations, advocating for pay equity analysis 

This talk will give real-world examples of action to take, and the outcomes they’ve had at organizations. 

Change doesn’t have to be a top-down initiative; and in fact, inclusive and equitable change seldom starts that way, considering those at the top are seldom from disempowered groups. As managers and individual contributors in positions of relatively large scope and authority, we’re in a great position to bring change to our organizations. As a bonus, the work we do toward these goals often benefits our own careers, and often doesn’t require a ton of time or financial investment.

Tech stacks vary widely from company to company.

The technologies available to us are constantly evolving. Likely your tech stack isn’t the most modern it could be. By extension maybe you, as an IC, aren’t as exposed to those technologies as you’d like. If you’re facing a tech stack that needs a long-overdue refactor should you commit to technologies you haven’t been as exposed to? In this talk I’ll share my experiences with leading your team through those uncharted waters and the lessons I learned along the way. 

The phrase of the day is “prompt engineering”—developing the skills to make LLMs productive for generating code, tests, and documentation.

But we have a bigger challenge on our hands: the need for better *substrate* engineering. A substrate is the layer an organism grows and lives on and is supported by. For LLM-based tools, that means the frameworks and programming languages we use every day. 

Put it this way: Would you trust LLM-generated C code to be free of memory vulnerabilities? As a rule, human-authored C has many vulnerabilities. By definition, so will Copilot-style tools. The programming language itself is no help here, and LLM-based tools can only be as good as the data they were trained on. What’s more, they can only be as good as the frameworks and programming languages they target. So in a world where these tools let programmers author far more code, far more quickly, our foundations matter more than ever. 

We need to start investing much more in migrating to better programming languages, building better tooling, and authoring new frameworks where correctness is built in. What does that look like for your engineering organization today?

In the dynamic landscape of software engineering, there’s a pervasive allure to what I call “Generic Goodstuff™” – universally lauded practices that seem like silver bullets to team improvement.

Yet, reality often falls short of expectations. This talk delves into the pitfalls of advocating for Generic Goodstuff without consideration for the context we’re working in. First, let’s talk about what Generic Goodstuff is and isn’t. There are plenty of great tools in software: Testing, documentation, process improvements, type systems (and more) all are good stuff – when applied to the right problems! The trouble arises when we stop thinking about the context we’re working in and apply these tools as generic, one-size-fits-all solutions to our software woes. Eager software engineers at every level want to improve things, but, lacking context, end up falling into the Generic Goodstuff trap. They end up driving initiatives that fizzle out without impact because great tools are applied in the wrong situation. It’s up to us Staff+ engineers to fuel that enthusiasm with context around the mission of the organization and the specific challenges our organizations are facing. We’ll address strategies for navigating Generic Goodstuff conversations, keeping the focus on the desired outcome, bringing valuable context to the table, and gaining a deeper understanding of what decisions brought us to our current situation.

It seems like every engineering team reaches a tipping point where bottoms-up technical steering where everyone “just knows who to talk to” no longer scales.

It becomes increasingly difficult to maintain shared mental models for how systems operate and should evolve over time. As a result, technical debt accumulates that no one feels empowered to take on, and cross-cutting initiatives stall without centralized strategy, accountability, and clear decision-makers, especially due to the historically flat nature of the org. Resolving technical disagreements often relies on either organizationally empowered EMs (violating our principle that technical leadership should come from ICs) or the accumulated social capital of the proposers (a non-inclusive system), rather than folks who are clearly empowered and accountable for making decisions.

This talk will cover how we overcame these challenges at Plaid through the creation of a technical leadership structure parallel to management. Since 2020, we have had a few iterations of our program. The first attempted to empower ICs by carving out time and accountability for engineers to focus on technical excellence, separate from a lot of day-to-day work on their teams. Based on learnings from this first iteration, we set up later iterations to better align with team structures and responsibilities.

We will discuss our learnings from these iterations and why we settled on our current approach. This includes our strategies for advocating for this type of program, what type of buy-in we needed from engineering management in order to make it happen, and how we knew that we had finally reached the tipping point that warranted a larger change. In our experience, creating a successful program is highly dependent on specific properties of your organization, so we will also share the specific attributes of Plaid that informed our choices, and how this might vary at other companies.

This talk by Ainsley will cover when to know you’ve outgrown your monolith and what to do about it.

We are all shaped by the sum of our life experiences. Canon events are the most formative: those responsible for building character and shaping us into who we are.

Companies can experience canon events too – most often in the form of a high-severity incident.

Incidents are unavoidable; they will happen. However, how you choose to respond to them is what matters and for better or worse, it will shape your engineering organization. In this talk, I’ll take you through a journey of a few canon events I have experienced throughout my career, including ones with names such as “The Great Reliability Crisis”. I’ll also share some things I’ve learned and how best to guide teams through them.