Designers have an opportunity to help lead & drive impact through managing the experiment process.
What I learned from a recent experience at Thumbtack is that designers have the opportunity to co-own the experiment process. By helping setup and maintain these 4 tools, design leaders can help their team have more buy-in and how to measure impact. This in turn allows team members to have more ownership.
Last year I joined a newly formed customer experience team as a product design lead. Alongside me was a product designer, product writer, user researcher, product manager, engineer manager, 7 web engineers and 4 native engineers.
Our team had successfully tested a new customer experience. It saw a lot of promise but there was more work needed to make sure it performed just as well, if not better than the baseline. We needed a process that would facilitate quick iteration and improve collaboration with design, pm & engineering.
Over roughly 6 months we created a process to rank, define, resource & track experiments. Our team was able to run 47 experiments that enabled us to ship the new experience!
1. Backlog
At the time we had no shortage of ideas, we tended to be reactive from data analysis that spurred random ideas. Our main problem was prioritizing the right ones that would make an impact our team had set goals on.
With a backlog we are able to remember where an idea came from and roughly what its priority is. This allows anyone to suggest an idea but keeps the working team focused on choosing ideas that make an impact.
For prioritization we use the RICE method. It’s a method that our whole team could agree on, which helps build trust that we’re working on the right thing.
See Sean McBride’s (PM, Intercom) article for more detail.
2. PRD Template
Our team didn’t have a consistent way of kicking off design on a new project. Sometimes the PM would write a PRD (Product Requirements Document), but it was inconsistent and missing the clarity designers needed to help contribute. This caused our team to sometimes become misaligned on what user problem we were solving, the scope of the project and how we were measuring success.
I also was receiving feedback from engineers around not having a forum to express their ideas/thoughts on a project once design took over. For them it felt like a black box and then boom, fully designed mocks were handed over for them to build.
Collaborating with PMs on a PRD template and having shared ownership allowed our designers to feel confident that they understand the user problem, business need and success metrics. This also creates a forum for anyone on the team (and others) to add comments.
3. Weekly Staffing
At the time our team was only tracking engineering tasks in Jira. This made it really hard for me as a design lead to figure out staffing earlier in the process (writing, research & design) especially if we wanted to plan design sprints or research studies. We also needed to make sure experiments wouldn’t overlap and dilute learnings from tests.
By keeping track of projects across functions we were able to plan ahead as a team. This enabled us to keep up with quick tactical work and also make time for deeper generative design sprints.
4. Experiment Tracker
Our team was lacking from a single source of truth. We didn’t know what experiments we had run in the past and to what success. This caused us to re-run similar experiments and be misaligned on the problem areas with the biggest opportunity.
Having an experiment tracker allows the team to keep all the relevant links, dates and retrospective notes in one sheet. This helps with forecasting, timing launches and future resourcing. It also helps to clearly see our success-to-failure ratio and use it to adjust strategy when needed.
Make a copy & share!
By no means is this the solution for all teams. This process was created by collaborating closely with product managers and engineers to fit our team’s needs. It continues to evolved as new people join and bring different perspectives. If your team prefers specific software like Jira or Coda adapt the parts that make sense for you. For us G-Suite works great because it’s easy to enable different levels of access and has great commenting features.
Backlog, Resourcing & Experiment Tracking Docs | PRD Template