Published: 2023-03-30
Retrospective: Winter of Pipelines
My theme for the year is ‘Focus’, and as part of that whole process for the first time I am trying out seasonal themes. This last Winter was ‘Winter of Pipelines’..
I have chosen a theme every year since 2016. (https://mastodon.social/@sarahjamielewis/109487396062363269)
This is a brief retrospective of the Winter of Pipelines season, things that worked, things that didn’t work, and a look forward into the start of Spring!
Successes
From practically all perspectives “Winter of Pipelines” was a very effective season. During the last 13 weeks I:
- Wrote a concrete plan for moving towards Cwtch Stable (and then implemented it)
- Tackled Reproducible Builds for libCwtch
- Open sourced Repliqate
- Developed a way for libCwtch to be automatically built from a specification
- Integrated UI Tests in the Cwtch UI Pipeline
- Dramatically improved documentation coverage for Cwtch
- Released Cwtch 1.11
- Wrote and published 12 Cwtch Development Logs - (One will be published the day after this post)
- Produced a Youtube Video about some of my Fuzzing Research
This list doesn’t include all the small bug fixes, ongoing projects, and active research that I also made progress on during the last few months. Some of that will start paying off early in Spring.
Focusing on Constraints
“Pipelines” was really a misnomer, while I set out to optimize a few processes what I really wanted to do was reduce bottlenecks, specifically:
- The effort it takes to review pull requests in Cwtch. One of our most valuable resources is people-time and reviewing PRs takes up most of that time. Any improvements to automated testing, automated review or code minimization has a large impact on the amount of other work that Open Privacy can do.
- The gap between work and writing about work. We know that people like reading about the work that we do, and we used to publish a fortnightly summary of ongoing projects. However that work was divided between 3 people, with one person writing something every 6 weeks or so. Now that there is only one person doing the bulk of new work that strategy makes little sense. Because of that, my secondary goal was to make it easier to publish about ongoing work.
These two tracks went hand in hand and, as you can see from the summary, a lot of the work that I wrote about during the last few weeks was directly aimed at minimizing review effort:
- Automated UI tests now mean that every Cwtch UI PR triggers dozens of checks on functionality across the application
- Automated library builds meant a huge reduction in the amount of code needed in libCwtch, and an ongoing reduction in review effort - to expose a new library function now takes a single specification line instead of 10-15 lines of boilerplate
Secondly, many of the other development logs are related to ongoing-design discussions. Many of these documents already existed in one form or another, and the major change was deciding to prepare them for wider publication. By making these documents more accessible we’ve seen an uptick in new people being interested in Cwtch development - even aspects unrelated to the main content of the development log.
My most effective weeks started with drafting the development log, and then guiding work towards that goal.
Getting Sidetracked in the Understory
Despite all the success I do want to note that I didn’t get everything I wanted to do done. And I did do several things I didn’t intend to do.
I had originally planned to get at least two Cwtch releases out during the season, and I had planned to spend some time working on my in-draft book.
On the flip side:
I definitely did not plan on completely automating Cwtch library bindings, nor did I expect to get sidetracked working on some, currently unreleased, awesome features.
I also spent more time than expected on general organization maintenance like accounting - which in hindsight should have been predictable.
Looking Forward to the Spring of Intentionality
Spring has already started, and have already spent some time this week thinking about how I want to move forward.
The theme I settled on is “Intentionality”, acknowledging that while the Winter of Pipelines was successful, that it could certainly have been more effective with some additional, thoughtful, planning.
A few lessons/techniques/strategies I am planning on integrating:
- Daily Time Tracking Works, but I think I should be collecting more data..
- Daily Metrics Tracking is not that useful - over the last few months I diligently spent each work morning updating the status of a number of key metrics. This mostly resulted in a general feeling dread every morning before I open the metrics sheets - even though the metrics themselves are as positive as they can likely be at the moment. I’m making an active decision to limit update tracking to once a week.
- Schedule admin disruptions, they are nearly always predictable - while accounting season is mostly over, there are other admin obligations on the horizon.