My TDD Pilgrimage

Posted on 2024-07-15
extreme-programming test-driven-development software-engineering
In 2010, I was working at a startup and a colleague did a show-and-tell about unit tests. It was my first job, I had never seen unit tests before, and in all honestly, I completely disliked it. Initially, thought it was a waste of time. Today in 2024, I have a very unsettling feeling if I end up writing a code without writing tests first. I have always wondered why I had an allergic reaction at the beginning and why it took me the time it did to warm up to this practice. So I started to roadmap key milestones I had in the last 14 years and the agents that brought about this change for me.
How I adopted TDD and What I felt...
When I was first introduced to writing tests, it was a Learning and Development session. As an outcome from the session everyone (20ish devs) were supposed to go and try out writing unit tests for the rest of the month and all PRs are required to have tests. We all did as said, but since none of us had truly written any unit tests before, so the quality and conventions around the tests varied. Furthermore, we were working with a sizable monolith which never had a single unit test before and hence despite being a pretty well written modular monolith it still required refactoring to allow code to be fully testable. Finally, we were writing unit tests after we had written the code, so we weren't actively thinking of all cases (happy path + edge cases) that should be unit tested. Most engineers, including myself, found this a tedious process. was nearly disastrous. We had unit tests which varied in convention across the codebase, some unit tests were fairly flaky, and the team was motivated to just do enough to get their PRs approved. At that stage in 2010, I hated unit testing.
TDD Popart
I joined ThoughtWorks in 2011, and I got a chance to attend the ThoughtWorks University which was a bootcamp for the graduate and younger developers to extreme programming practices, Agile software development methodologies, and the ThoughtWorks culture. We picked up a pro bono project to help a local non-profit organisation. We followed all the processes and steps any ThoughtWorks project would follow. Here I was introduced to TDD. We were constantly pairing and writing unit tests first for all code we were supposed to write. We were pairing with industry leaders (our trainers) to learn and then other colleagues to practise as we picked up user stories. I would say I didn't really hate unit tests anymore, but I was still finding TDD very tedious. Almost as if I had an idea, and I need to first write a test over quickly working on my “fantastic” idea. I struggled to see the balance in the trade-off. I couldn't shake the feeling that I was moving slower.
After the 2 months of bootcamp, we moved to client facing projects, and it was apparent to me TDD was “The Way of Working” in ThoughtWorks. My brain started understanding the value, however, it had not yet become muscle memory. That is every time I was picking up a story on which I wasn't pairing I was writing the test after I wrote the code. Though it wasn't long when I started realising that when I followed TDD (with my pair), I needed to refactor less after the fact since the code was written with best practices like dependency injection, and single responsibility principle. Whereas when I was writing code first, I had to refactor a fair bit to make my code testable. The realisation that I am going faster, introducing fewer bugs and feeling far more confident on my code quality was getting clearer and clearer. As this feeling settled in, TDD started becoming more of a muscle memory to me. And since then it has always been the case.
Fast forward 2024, I am an Engineering lead. The percentage time I spend on tools has changed to a fairly low percentage mark. But when I write code, I can’t get my brain to work in any other way than writing tests first.
Bringing others on the journey
All of the above was about my adoption journey, as I grew in my role - I saw more people following a similar path to mine - starting confused & frustrated, then realisations followed and finally came the adoption. I wanted to help others reduce the frustrations and get to adoption faster - in my search for the answer I stumbled upon the article Canon TDD by Kent Beck. The step to list the test scenarios was a crucial find for me. I started encouraging people to think of all the test scenarios first and write empty test bodies with just the test names indicating the scenario and then fill them one by one. For example, I have a domain object called Blog. The Blog has a field called Content which stores the blog’s html body. To extract the summary I want only the first html tag contents. We start by first writing the list of unit test scenarios. See example in golang below:
            
    func TestBlog_GetSummary(t *testing.T) {
        t.Run("should return empty string for empty content", func(t *testing.T) {
            //Input: ""
            //Output: ""
        })

        t.Run("Should return only the first html tag", func(t *testing.T) {
            //Input: "

Some content

Some more content

" //Output: "

Some content

" }) t.Run("Should return only the first html with all sub tags tag", func(t *testing.T) { //Input: "

Captain Kathryn Janeway

Some more content

" //Output: "

Captain Kathryn Janeway

" }) }
Then you turn ONLY the first item on the list into an actual failing unit test. Once the test is written, you write the code. Let’s pick up the first test in the list above. Failing since we haven’t written the code yet.
            
    func TestBlog_GetSummary(t *testing.T) {
       t.Run("should return empty string for empty content", func(t *testing.T) {
            b := &Blog{Title: "Some Title", Content: "", State: Draft}
            if got := b.GetSummary(); got != "" {
                t.Errorf("GetSummary() = %v, want %v", got, "")
            }
        })

        t.Run("Should return only the first html tag", func(t *testing.T) {
            //Input: "

Some content

Some more content

" //Output: "

Some content

" }) t.Run("Should return only the first html with all sub tags tag", func(t *testing.T) { //Input: "

Captain Kathryn Janeway

Some more content

" //Output: "

Captain Kathryn Janeway

" }) }
Next, you write just enough code to get the first test to pass.
            
    func (b *Blog) GetSummary() (string) {
	    return ""
    }
            
        
Once the test passes, we move on to the next test in our list. We write the test and let it fail first.
            
    func TestBlog_GetSummary(t *testing.T) {
       t.Run("should return empty string for empty content", func(t *testing.T) {
            b := &Blog{Title: "Some Title", Content: "", State: Draft}
            if got := b.GetSummary(); got != "" {
                t.Errorf("GetSummary() = %v, want %v", got, "")
            }
       })

        t.Run("Should return only the first html tag", func(t *testing.T) {
            b := &Blog{
                Title:   "A New Title",
                Content: "

Some content

Some more content

", State: Draft, } if got := b.GetSummary(); got != "

Some content

" { t.Errorf("GetSummary() = %v, want %v", got, "

Some content

") } }) t.Run("Should return only the first html with all sub tags tag", func(t *testing.T) { //Input: "

Captain Kathryn Janeway

Some more content

" //Output: "

Captain Kathryn Janeway

" }) }
Again, you write just enough code to get the second test to pass.
            
    func (b *Blog) GetSummary() (summary string) {
        reader := strings.NewReader(b.Content)
        tokenizer := html.NewTokenizer(reader)

        for {
            tokenType := tokenizer.Next()
            token := tokenizer.Token()
            switch tokenType {
            case html.StartTagToken:
                summary += token.String()
            case html.TextToken:
                summary += token.Data
            case html.EndTagToken:
                summary += token.String()
                return summary
            }
        }
    }
                
        
You follow the process until you write up all your unit tests (one by one), get them red and then get them green. (Do see this comic strip by Uncle Bob giving a bit of idea on how the process feels like.)
A look at the benefits
By far, I have been talking more about the process, and I touched on the benefits briefly. Let’s dive in the benefits in more detail now. The list below is my reason to follow TDD:
  • Improved code quality: You start by writing a list of test scenarios listed out first. You are thinking about inputs and edge cases first. This leads to decreased chances of missing requirements, and introducing bugs.
  • Upholding the test pyramid: Unit tests are always cheaper to write and run. When you focus so much on writing unit tests before code you are absolutely building a solid base of the testing pyramid.
  • Maintainable and extensible code: The chances of writing “clean code” significantly increase when you write tests first. Why do I make this claim - to allow your code to be testable you need to start thinking of how you would design your code first? When you consider code design before diving into coding, you start thinking about cohesion, testability, and low coupling, leading to patterns such as dependency injection and the single-responsibility principle. All this leads to maintainable, testable and extensible code blocks at core.
  • Better documentation: Well written test scenarios are live documentation of the code. They need to get updated to keep the builds green. These documents seldom go out of date. Additionally, they make on-boarding and context sharing easier for teams.
  • Speed: There is a perception that TDD slows us down. On the contrary, TDD saves time by delivering higher quality code upfront. Additionally, it brings design thinking first, leading to code which is highly extensible and maintainable in future. All these are efficiencies you need to go faster.
Let's wrap it up!
I called this blog "My TDD Pilgrimage" because it truly has been a long and transformative journey for me. From my initial scepticism and resistance to the concept of unit testing to now, where TDD is an integral part of my coding process. For me, TDD is a “silver bullet” to keep your systems from turning into legacy, to ensure you have living documentations, and to get better code quality.
It is natural to feel frustrated and overwhelmed, especially when a process/practice challenges your ways of working; and TDD is much of a mindset shift as well. As an engineering lead, my focus has shifted from writing code to enabling others to write better code. Seeing others go through similar phases of frustration and realisation, I strive to help them shorten the learning curve and embrace. I hope my journey and insights can inspire others to embrace TDD and experience its profound impact on their development process.