From Lone Architects to Team Players: Rethinking System Design
Exploring why system design thrives as a collaborative effort and how embracing teamwork leads to better architecture.
👋 Hi, this is Thomas. Welcome to a new edition of Beyond Runtime, where I dive into the messy, fascinating world of distributed systems, debugging, AI, and system design. All through the lens of a CTO with 20+ years in the backend trenches.
QUOTE OF THE WEEK:
“[…] You have to take yourself out of just the code for your single component and go higher level, to the 10,000 foot view and look down and understand how things work together” - me, Thomas Johnson 🙂
I was recently interviewed for a three-part series on MonkCast, RedMonk’s podcast, where I discussed the evolving landscape of system design with KellyAnn Fitzpatrick.
Here’s my summary — and a few more thoughts — of that conversation.
System Design is a Team Sport
In various written content, I have made the assertion that system architecture is a team sport, meaning (in part) that it is a concern not just of someone in an "architect" role, but of folks in various roles on an engineering team. This, of course, suggests that things may not have always been this way.
Indeed, for years the software architecture practice has been seen as the scope of a single Software Architect (or an Enterprise Architecture team) who would produce detailed system designs upfront and in isolation, and then “impose” it on the development teams without soliciting or incorporating their input.
As I discussed in the first episode of the MonkCast, software development has evolved, and so has our understanding of how architecture works.
Software architecture, same as any other aspect of software engineering, is an inherently social kind of work, requiring close collaboration to coordinate efforts, integrate different parts of the software, and ensure that the final product is cohesive and functional.
Software architecture is a core engineering skill, possessing system design experience and considering architectural requirements when coding makes you a better engineer and results in a better product.
Software system are much more complex and continuously evolving. Modern software systems are very complex and require the collaboration of the entire eng team from frontend, backend, DevOps, etc. No single person can track every change or foresee every implication.
Therefore, there is a shift from software architecture as a title to software architecture as role or an activity that is performed throughout the entire project lifecycle.
A team sport approach to system design fosters:
A more integrated and cohesive final product. A thorough grasp of system design enables engineers to:
(1) have a better overall understanding of the project's objectives,
(2) effectively collaborate with their peers
(3) successfully contribute to the project’s development and implementation, and
(4) proactively identify potential design flaws or bottlenecks.
Thoughtful technical decisions. Technical decisions should be made collaboratively, considering not only the optimal solutions to meet functional and non-functional requirements but also the team's existing technical knowledge, skills, and capacity. This avoids the risk of uncoordinated work where devs make uninformed decisions due to siloed knowledge, the system tends to sprawl, becoming increasingly chaotic and leading to redundancies and more complex dependencies.
Shared responsibility. The entire team collectively shoulders the responsibility for the final output. There is no separation between the decision-making authority from the responsibility for results.
Resilience to change. When every developer within your team knows and understands the system architecture, this significantly reduces the risk of architectural knowledge loss when developers leave. This also leads us to the value of accurate and up-to-date documentation, although that’s a longer conversation.
Why Individual Practitioners Should Learn System Design
Setting aside the importance of tacking system design as a team, even from an individual practitioner point of view, it’s important to know system design. A basic understanding of system architecture can greatly benefit engineers in multiple aspects of their day to day work. For example:
Navigating complexity. Proficiency in system design is critical for making informed coding decisions and understanding the impact of their work on the entire system.
Integrating technologies: With businesses integrating a mix of new and legacy systems across various platforms, the ability to design seamless system integrations is vital. Not to mention that due to the higher proportion of brown-field vs green-field projects, those developers that can understand, navigate and improve upon legacy architectures, are advantaged.
Building cost-conscious software: Cost-cutting is still at the top of engineering managers’ agendas in 2024 (and likely next year too). A knowledge of system design avoids over-engineering and allows devs to perform conscious architectural trade-offs.
Embedding security: In an era of escalating cyber threats, incorporating robust security measures into the system architecture from the outset is paramount. System designers must be adept at identifying and mitigating potential security risks at all levels of the architecture.
Moreover, system design skills are increasingly important in job interviews, where interviewers often ask “how would you scale this system? how would you build this app? how would you solve this problem?…” all system design questions.
The Invisible Artistry of Backend Design
Maybe this is me geeking out as a backend developer but I also think that system design is a creative act. We tend to see front-end development as inherently creative, but fail to see the artistry in, say, a really well-designed API.
If you don’t have a backend system that’s as beautiful as the frontend —and “beauty” means a great design in this case—that frontend app might not be able to do anything.
Backend design decisions ripple through the entire organization. A poorly designed backend hinders scalability and innovation, while a well-designed system enables seamless user experiences, product growth, and long-term stability.
Good system architecture is like art—an invisible, yet beautiful, foundation that defines the success of the entire system. When backend and frontend align in their elegance and functionality, the result is something truly extraordinary.
This is the full conversation with KellyAnn Fitzpatrick:
💜 This newsletter is sponsored by Multiplayer.app.
Full stack session recording. End-to-end visibility in a single click.
📚 Interesting Articles & Resources
Sharding v/s Partitioning - Vivek Bansal
Sharding refers to splitting a dataset into independent subsets, typically to handle scaling and performance in distributed databases. Partitioning, often a broader term, includes strategies for dividing data based on business logic or operational needs.
Understanding how the choice between these strategies impacts system scalability, fault tolerance, and complexity.
How Apple built iCloud to store billions of databases - Engineer's Codex
Apple's iCloud architecture is a masterclass in scaling for billions of users and devices. The system achieves seamless synchronization by using a robust backend infrastructure that leverages partitioning and redundancy for fault tolerance. With billions of files and daily operations, the architecture balances user experience, security, and performance.
Migrating from AWS to Self-Hosting - Zig Engineering Team
The Zig programming language recently completed its migration from cloud services to a single-server solution, allowing for greater flexibility for future development, and ensureing better cost-effectiveness.
Multi Cloud Architecture: Tutorial & Best Practices - Thomas Johnson
Multi-cloud architecture has its benefits—e.g. no vendor lock-in, improved scalability, regulatory compliance, etc.—and its challenges, including increased complexity and potential security concerns. This article dives into when and how to choose this type of architecture.