How a Developer Productivity Engineering (DPE) approach can speed up writing, running and debugging tests.
Abstract
Testing is a Good Thing, right? Especially automated testing. But "Good things come to those who wait" is not something that's going to appeal to the busy developer. You want results, and you want them now. You're in The Zone working on a problem, and the last thing you want is to break your flow wrestling with your testing framework or waiting for the tests to finish running.
More code means more tests. More coverage means more tests. More tests mean more time. Time that you want to spend being productive, creative, innovative. How can you balance the need for quality with the need for speed?
In this talk, Trisha will identify issues that slow down developers when writing, running and debugging tests, and look at tools that can help developers with each of these problems. There will be live coding, analysis of social media poll results, an overview of solutions in this space, "best practice" recommendations, and machine learning will be mentioned at some point.
Video
Other videos
- Virtual JUG presentation (2024)
- Devoxx Belgium (2023)
Slides
Resources
Writing Tests
IntelliJ IDEA tips for writing code
- Code Completion (video)
- Code Generation (video)
- Writing Tests with JUnit 5 (blog & video)
- Creating a test Live Template (blog)
- IntelliJ IDEA Guide - full of tips and tutorials
Using ML & AI
- JetBrains AI Assistant
- GitHub Copilot
- Paper on LLMs for software testing (paper) - I have not read this, but thought maybe it may be relevant for those looking for information on the topic
- Coding Session With ChatGPT - How Helpful Is It? (video) - good examples of how to use ChatGPT, with pointers on the pros and cons
- Challenging The Myths of Generative AI - there's some good stuff in here about "The Productivity Myth" and "The Creativity Myth" which is relevant to our jobs as developers.
Debugging / Troubleshooting Tests
- IntelliJ IDEA for Debugging (YouTube playlist)
- Remote Debugging (reference documentation)
- Remote Debugging tutorial
- Analyze stack trace (reference documentation)
- Using a Build Scan to troubleshoot Maven and Gradle builds
Flaky tests
- Your tests are failing you (video) - my rant on the Continuous Delivery channel about flaky tests
- 5 causes of test intermittency (video) - Dave Farley gives excellent pointers on why your tests might be flaky
- Non Determinism (Martin Fowler)
- Why fix your flaky tests (blog) - I wrote this blog because I thought perhaps people might not understand why flaky tests are a problem
- How to use Develocity to identify your flaky tests (blog) - I wrote this product-specific blog not only to help sales of Gradle's Develocity tool, but also to talk about how to use tooling to find your problem tests
- How to fix your flaky tests (blog) - I use a bunch of ideas from Dave's video on test intermittency (above) to give pointers on how to fix your flaky tests
- How Top Banks Handle Flaky Tests (blog) - a case study of four banks' approaches, and i also inserted some extra tips on addressing flaky tests
- Develocity Flaky Test Detection (reference documentation)
- If you want to show your disdain for flaky tests, you can literally buy the t-shirt.
- Surveying the Developer Experience of Flaky Tests (PDF)
Running tests
- Running JUnit 5 tests in parallel (Baeldung tutorial)
- Running JUnit tests in parallel with Maven (Baeldung tutorial)
- Test Distribution with Gradle's Develocity (reference documentation)
- Case Study: Netflix and Develocity test distribution (blog)
- Testing@LMAX – Distributed Builds with Romero (blog)
- Running tests with IntelliJ IDEA (reference documentation)
- Predictive Test Selection (Meta Research paper)
- Develocity Predictive Test Selection (reference documentation)
- Speeding up Builds with Predictive Test Selection (overview video)
- Testing@LMAX – Time Travel and the TARDIS (blog)
Cleaning up your test suite
- Automatically detecting redundant tests (blog) - on using mutation testing to detect duplicate tests
- Java Mutation Testing systems
General information about testing
- Kent Beck and Dave Farley talk testing (video) - highly recommended! A conversation between two people who are passionate about automated testing and software engineering
- Testing is bad for Developer Productivity (video) - my video for the Continuous Delivery channel about some complaints developers have about testing
- Desirable Unit Tests (Kent Beck) - Laying out 12 properties that Kent calls "sliders" to consider when writing unit tests
- Programming with GUTs (Good Unit Tests) (Kevlin Henney)
- Test Cancer (Martin Fowler) - an anti-pattern where tests are no longer maintained
- Test Shapes (Martin Fowler)
- Test Pyramid (Martin Fowler)
- Practical Test Pyramid (article)
- Testing in 2021 (Tim Bray)
- Why acceptance test (Dave Farley)
- We need to talk about testing (Daniel Terhorst-North)
- Lisa Crispin: Holistic Approach to Testing (podcast with transcript)
- When developers say writing tests slows them down | Episode #24 (podcast, Lisa Crispin & Janet Gregory with Trisha Gee)
- Kelsey Hightower on Developer Experience and Testing in Production (podcast)
- TDD, Where Did It All Go Wrong (video) - great talk about how to deal with brittle tests, why you shouldn't use mocks, and why you should be writing social unit tests
- Separating Structural and Behavioural changes (blog, Kent Beck) - we kinda know we should do this, but we don't
- Three Mindsets of Test Driven Development (TDD) (video, Dave Farley)
- How many test failures are acceptable? (blog, Dave Farley)
- All about test automation: best practices with Angie Jones from Applitools (podcast)
- TCR: test && commit || revert (blog, Kent Beck) - really interesting idea to keep your changes extremely small
- How Practicing TCR (Test && Commit || Revert) Reduces Batch Size (article)
- Which tests should we automate? (video, Angie Jones)
- All Things Automated Testing With Specialist Emily Bache (video)
- Test Maintainability (blog, Dave Farley)
Other
- Organisational Physics personality types
- Gradle Performance (reference documentation) - including information on how to configure parallel execution, and the local build cache
- Top 10 Performance Mistakes (Martin Thompson)
Books & courses
- Modern Software Engineering - Dave Farley
- Tidy First? - Kent Beck
- Test Driven Development: By Example - Kent Beck
- Agile Testing Condensed: A Brief Introduction - Lisa Crispin & Janet Gregory
- Getting to Know IntelliJ IDEA - Trisha Gee & Helen Scott - 20% off with this link (or see this page for more options, including paperback and hardback)
- Setting a Foundation for Successful Test Automation (video course)
- Developing Applications with IntelliJ IDEA (Leanpub course) - yes this is a shameless plug for my own course, but we do cover running and debugging tests in it. It's free if you have a Leanpub membership.
- Working With Code in IntelliJ IDEA (Leanpub course) - another one of my courses (also free if you have a Leanpub membership). This one teaches you how to use code completion and generation, live templates, etc, in order to create code quickly and safely.
- How to Use Build Scan® (DPE University) - free course I created for Gradle that shows how to use Build Scans to troubleshoot your builds and tests
- Introduction to Develocity (DPE University) - a free course I created for Gradle about using Develocity, contains a chapters on: troubleshooting tests; identifying flaky tests; and speeding up tests with Predictive Test Selection.
Attributions
If you like my stuff, buy my books and courses!