02.05.2021   /   duration: 23 min
Featured Episode

A retro for retrospectives

It’s the season finale! In the final episode of season one, Jay leads us through a discussion about product team retrospectives. We review simple strategies for planning and executing retrospectives; we wrap up the season with our own retro of the podcast!

Hosted By

Jay Cosgrove, Senior Product Manager at Digital Scientists
Jay Cosgrove
senior product manager

Episode Transcript

Rob Hall: This is the experience lab, the official podcast of digital scientists from Atlanta, Georgia. We’re an experience lab that explores and builds digital products. My name is Rob Hall, and I’m the Senior Director of Product of DS.


Jay Cosgrove: And I’m Jay Cosgrove, senior product manager.


Rob Hall: Thanks for listening.


Rob Hall: Hey, Jay, what’s up? It’s a beautiful day.


Jay Cosgrove: It is gorgeous.


Rob Hall: It’s a unseasonably warm,


Jay Cosgrove: I think it was 65.


Rob Hall: That’s nice, but you know, global warming.


Jay Cosgrove: I was gonna say this is the reason I live in the south, but maybe I shouldn’t be so happy so quickly.


Rob Hall: Yeah, about that. So what’s new, anything exciting going on in your world? 


Jay Cosgrove: Well, actually, we just had our second kiddo. So that’s pretty exciting.


Rob Hall: That is very exciting.


Jay Cosgrove: Yep. Evelyn Lee. She’s a cutie. Thank God.


Rob Hall: She is adorable.


Jay Cosgrove: Thanks, man.


Rob Hall: Clearly she takes after her mother.


Jay Cosgrove: Yes. Yeah. Thank God for that right. 


Rob Hall: Yeah, exactly. 


Jay Cosgrove: Yeah. Well, hey, I heard you have a new little one in your house. 


Rob Hall: Oh, I do. We have a new puppy.


Jay Cosgrove: A little different. 


Rob Hall: A little different. But also similar. There’s still the late night feedings and so on. Or at the late night potty break. That is slightly less diverse but slightly less, but still exhausting. exhausting. Yes.


Jay Cosgrove: Well, your pup is pretty cute too.


Rob Hall: Well, thank you. Yes, his name is Laird Falcor Finley. And we just call him Finn for short.


Jay Cosgrove: Yeah, I was gonna say he’ll never win that name. 


Rob Hall: Oh, no, no, not a chance. We’ll get a woof out of them. And that’ll be pretty good. So, Jay, what are we talking about today?


Jay Cosgrove: Well, we thought we would cap off season one with a little retrospective.


Rob Hall: Do tell.


Jay Cosgrove: Yeah. So we’re gonna go over today how the podcast has done in 2020. And run a retrospective on it. And we’ll also talk about what retrospectives are in general.


Rob Hall: This is a really good topic, especially for product managers who are looking for a little advice on how to get better at running retrospectives. They’re a very important part of any product team that’s either running in a sprint cadence, or has just completed a big release, any kind of major milestone that you reach, I think it’s very important that you pause and take the time to reflect on how you’ve done and where things are going. Moving forward.


Jay Cosgrove: Yeah, I would agree with that. So let’s let’s break it down. How would you define a retrospective?


Rob Hall: That’s a great question. My personal definition of a retrospective is simply an organized meeting, in which product team, your product manager, if there’s a project manager involved, or developers, designers, you get together in an intentional way to reflect upon the activities that you’ve completed, and in a real and transparent manner, certainly with guardrails and with boundaries. But the idea is to create a safe space where you can realistically talk about what’s gone well, what needs improvement? And what you’re going to do next, what actions are going to follow to either continue improvement to continue delivery, or to course correct.


Jay Cosgrove: I love that. And we talked about this pretty frequently. But one of the things you’ll learn as your tenure expands and product management is that process can’t really be sacred. And really the process is, is just a tool. It’s a tool that we implement to create efficiency. And using retrospectives allows us to analyze our process up to that point, and determine what’s working and what’s not. And even the practice of using retrospectives, I think has to be questioned a little bit to house up. Well, one of the retrospectives I ran in my early days where I was being very faithful to run them after every two weeks sprint. The feedback I got from the team was we do retrospectives too often.


Rob Hall: How often were you doing that? ,


Jay Cosgrove: Just every two weeks at the end of the sprint I was a very good faithful, agile Scrum man at the time. 


Rob Hall: Very faithful.


Jay Cosgrove: Very faithful by the book. And it kind of shocked me, you know, because I had kind of lived and died on the hill that was Scrum up to that point. And then really had to consider that feedback after the retrospective of do I allow my process to be broken? Do I hold so sacredly to Scrum?


Rob Hall: What was the feedback you were getting, though from your team that indicated that they were? Because you’re right. The process prescribes retrospective at the conclusion of every sprint, right? Why did the team feel like it was too much? Too often?


Jay Cosgrove: Well, at that point, I think the team had been set for like six months, it was a long standing project. And we just had really good communication, there was a lot of trust that was developed in the team. And so honestly, during a lot of our other ceremonies that we had the stand ups or even the the, like sprint plannings that we would do, ideas would be thrown out, even in Slack, you know, just this is, Hey, could we tweak this? And I tend to move very quickly on good ideas, you know, so if something got thrown in, I’d evaluate it. And, you know, as early as the next day where we changed pace? Sure. So I mean, the team was smaller, which helped, you know, I think there were three to five people on it, something like that. And yeah, that was the first time I really had a question. And that came out in a retrospective.


Rob Hall: Hmm, my past experience has been not doing enough retrospectives.


Jay Cosgrove: Interesting.


Rob Hall: And mainly, mainly due to, at least in the agency world, have a feeling this anxiety and stress about moving on to the next thing, writing too quickly and allowing that pressure to dictate the dialogue? Essentially, instead of pushing back and saying, nope, this is really important. We need to press pause just for an hour, or even less to reflect and, and get these perspectives out on the table.


Jay Cosgrove: Yes, they’re very important. And I think this is like every answer we give the cadence that you use them and how you use them. The questions you ask really depends on the team, and where you are at in the project and what is needed. Because I 100% agree. Like, I didn’t say any of that to say I don’t recommend retrospectives. I was just saying that there’s a real question that should be asked of what in my process is truly sacred and what isn’t and are we willing to bend?


Rob Hall: So let’s talk about that for a quick second. How do you determine through the makeup of your team? how to best structure a retrospective? What are the types of considerations that are running through your mind when you’re figuring out you know, the right activities or the right way to structure? The retro itself?


Jay Cosgrove: Yeah, that’s a good question. I think the size of the team is going to be a big deal. Like if I’ve got 10 people in the room, then I’ve got to keep the exercises more snappy? Sure. So the number of stickies someone puts up on the board or the virtual board at this point, right? I might limit and therefore I might pick exercises that allow for one or two pieces of input from each team member versus maybe there’s four columns, and everyone fills out five in each column or something like that. So my like my go to is probably the very typical one, which is, you know, what worked, what isn’t working? And what, you know, what do we never want to see again? What do we want more of those kind of things, and there’s about a bazillion ways people word them today, we’re actually going to use, liked, learned, lacked and longed for, for our retrospective. And it’ll be the first time I’ve actually run that one.


Rob Hall: That sounds very nice.


Jay Cosgrove: Yep. So the size of the team. And I would also say, the amount of time that I’ve gotten, how many retrospectives I’ve run with that team, those are all going to go into it. If I’ve done maybe multiple using a certain exercise with the same team, then I’m going to switch it up. If there’s people in the team that contribute less feedback, then I’m very mindful of them and my structure exercises that are really gonna engage them.


Rob Hall: So one, one rule of thumb, I think that’s very important for a product manager to keep in mind is that there is no one single perfect way to structure a retrospective. Yep. I would strongly propose that the only wrong way to structure a retro is by not setting clear boundaries and expectations up front. 


Jay Cosgrove: Yeah, I would agree. 


Rob Hall: You don’t want to run a retrospective without any preparation. And you’d want to let your team know, in clear language, what type of feedback is expected and allowable during the conversation.


Jay Cosgrove: Yeah, I would, I would specifically say there that you want to make it very clear that we steer away from personal tax 100%.


Rob Hall: Absolutely.


Jay Cosgrove: I mean that’s what you’re trying to avoid. Is it getting incredibly personal and keeping it on the work?


Rob Hall: That’s correct. I think there’s in any team whenever there are when there’s a diversity of thought and a diversity of ideas. There’s bound to be some level of personal conflict. And as we talked about last time, in our conversation, about niceness disease there can be a real temptation to avoid dealing with that conflict or, or a fear of engaging with that conflict, to try and understand each other’s point of view. But where I think the team can grow very toxic is when we’re characterizing our remarks towards the individual and their character, and their makeup as a person, right, rather than the outcome that was delivered in the work itself. And so when we resort to making commentary in the retrospective about the style of a person’s communication, or the tone of voice that they use, and and not that those can be points of concern, I would just argue that those are better discussed in the format of a one on one Yep, rather than out in the open in front of the entire team where someone’s feelings could be hurt, or someone could potentially be humiliated. without, you know, really, that being the appropriate forum. Yeah, for that type of feedback to be discussed.


Jay Cosgrove: Yeah. I mean, I think the saying that you praise in public and critique in private is, is a real mark of maturity in the workforce. And I think a lot of times when people resort to personal critique in these types of meetings is because they’re shying away from those direct conversations, and sometimes even trying to like, you know, alley, or get allies in the meeting, to gang up. And that’s really not the culture you want to set. So I agree, set good boundaries at the beginning of your retro.


Rob Hall: Now, at the same time, that does not mean you want to shy away from constructive criticism of the team. But the important thing is to use the Royal we, rather than you.


Jay Cosgrove: Yeah.


Rob Hall: So we as a team failed to hit the mark in X y&z concrete way, you know, relying on your KPIs or your okrs that you’ve set at the outset of the sprint, again, as your Northstar to objectively talk about whether or not you know, the sprint was, or the project as a whole was successful, but avoiding ad hominem attacks when possible is very important. agreed, agreed?


Jay Cosgrove: Well, there’s a couple good books out there in good resources that you can use if you’re new at running retros. Or if you’re stale with your retros. And looking for some different things. I know one we’ve used here are retro cards, I think Edgar Allen is the one. If you Google that you should find them. It’s like a deck of cards. And they’re super cool, super cool, great questions on there that can really spark new ideas. But honestly, if you google retro exercises, you will very quickly get a pretty good list of them.


Rob Hall: Yeah, there’s a couple of other good books that are out there fun retrospectives by Paulo karolyi. That’s a pretty good one. He’s got a lot of nice activities for making your retrospective more engaging and fun. There’s also this is more of a standard book about retrospectives. Generally, within the Agile framework, it’s called agile retrospectives making good teams great by Esther Derby and Diana Larson. It’s from the pragmatic marketing group. Many product managers have learned how to product manage through pragmatic groups, materials and their courses of study. So it’s, it’s a good primer.


Jay Cosgrove: Alright, let’s switch gears here and get into our 2020. Retro on the experience lab podcast.


Rob Hall: Okay, just to be clear, we’re not gonna wax poetic on how terrible 2020 was because I think everybody’s beating the year beating it to a pulp adequately enough.


Jay Cosgrove: Yeah, this is good. Let’s set that as a boundary at the outset of our retrospectives.


Rob Hall: First boundary. Yeah. 


Jay Cosgrove: Number one, 2020 has already been beat down. No comments on the year. Number two, don’t attack me or I will punch back.


Rob Hall: We’ll see. 


Jay Cosgrove: Just kidding. 


Rob Hall: We’re socially distant. So you know.


Jay Cosgrove: Yeah might be a little harder. You got a little more room to move. 


Rob Hall: That’s right. 


Jay Cosgrove: All right. So the first first column, and we’re just going to go in order here. Normally in a retrospective, we take some quiet time at the beginning to write stickies for all the columns. But the first column we’re just going to go in order is liked. So what did you like, Rob, about how we ran the experience led podcast in 2020.


Rob Hall: Liked yeah, it’s the light column. I’m gonna use the word love borrow more, can you I, I deeply enjoyed the opportunity to share with a much broader audience than we have in the past, our experience and our strengths as a team. I think being able to showcase our talent and our expertise, and talk about how we approach doing what we do all day every day has been very, very rewarding.


Jay Cosgrove: Yeah, I would agree. I was pretty nervous about this at the beginning. I mean, the idea of, of speaking, and especially not having, you know, 20 years under my belt in product management made me feel very vulnerable and like who am I to comment on this, but I think it really dug out of both of us. But certainly me. The fact that like, there’s a lot of knowledge that we’ve gained, even in my short career that is worth sharing, that took a lot of time for us to learn, just through the regular work day, and sometimes as mundane as it can be, and being able to share that, like you said, is feels really giving back to the community, you know.


Rob Hall: Yes yeah, I completely agree. 


Jay Cosgrove: I think another thing that was good, was we’ve had some little traction on the podcast, it’s been awesome. And even though we haven’t done a ton of promotion, we’ve received a lot of encouragement from listeners. And that has been something I’ve really enjoyed as well.


Rob Hall: Yeah, I agree. We got a note from a listener just a few days ago, completely out of the blue on LinkedIn. Hey, Rob, just curious when season two’s coming out, I love the show, can’t wait to hear more. It’s things like that that are extremely validating, and very helpful to know that what we’re doing is providing value to people who are trying to get better at their craft.


Jay Cosgrove: All right, the next column learned, what did we learn through 2020 on this podcast.


Rob Hall: I think we learned that it is easy to create a podcast, it is more difficult to create a high quality podcast. And one of the thing one of the values that we kind of set forth at the beginning is that we wanted to focus on quality over quantity. So there’s lots of other product management podcasts, some that are putting out an episode a week, some of they’re very good. But you can also kind of see a correlation between the production value of shows that are coming out more frequently, versus those that are taking the time to refine the narrative, compose music for them. Yeah, and so on. And so we found that it has been a really big effort. Every single episode that we’ve put out has included original compositions, some by other members of our team, Biff, in particular. 


Jay Cosgrove: And by you, man. 


Rob Hall: And some by me, sure, it all takes work. It all requires effort. But it to me is important, as Digital Scientists, relates itself as an experienced lab quality and the depth of our experience is we strive to reflect that in the quality of our work. And it was, you know, kind of our vision that the podcast would also reflect the quality of the work that we do.


Jay Cosgrove: Yeah, I think there’s a yin and yang to that, too. Because I know that internally, especially you hemmed and hawed over this for a while, you know, there, there was probably at least, what two month period three month period before we just set up the mics and started working on it. So I think there’s just like anything, you can over think it sometimes. And sometimes you need to just kind of go for it and learn as you go. And I think we’ve done a good mix of both.


Rob Hall: Yeah, I agree. Jay, I was actually going to say the next thing that I personally learned is that it is easy to allow imposter syndrome to take hold when you’re listening to negative voices saying you can’t do this, or you’re not ready to do this, or you’re not good enough to do this. And not that we had a lot of those voices. But they were there. And it was hard to overcome that. Yeah. And then finally just say, well, dag gummit we’re gonna do it. Jay, get your mic. We’re doing it. And with efforts like this, just getting started is often the hardest part.


Jay Cosgrove: Yeah, it’s much easier once you’re going.


Rob Hall: That’s right. And it has been for sure.


Jay Cosgrove: I think a couple granular things too, to comment on before we’re off the learn section. I think we learned that shorter episodes work better. Yes, you know, purely from analytics, and not just analytics, I mean, friends and family comments from them. And in even listening back through episodes ourselves. I think the ones that come in between 20 and 30 minutes are some sort of a sweet spot for us. 


Rob Hall: Yeah, I agree. I think we’ve definitely seen that we can pack a lot of value into a shorter span of space. And if we have longer form topics, then we can break them into multiple episodes. 


Jay Cosgrove: Yep, I agree. I think that works better. Another thing that is challenging, and I don’t think we’ve quite figured it out fully, is doing remote recording together. lag tends to be an issue that we’re still kind of working through. But we’ve learned that it’s an issue to be aware of.


Rob Hall: Yeah, it’s funny. There’s a lot of podcasts that I mean, I’d say most that have a similar format to our show, or obviously recorded remotely but there’s a chemistry that’s easily lacking. And I think a lot of that is in part to some technical limitations and kind of blockers on the technical side of things. And that’s something we’ll keep getting better at.


Jay Cosgrove: Third column lacked. So this column is things that we saw. And we want more of. So we didn’t have quite enough?


Rob Hall: I agree with the notion that we lacked consistency. And mainly in terms of release schedule. Yeah. as kind of a side project within our marketing organization. 


Jay Cosgrove: It’s our day to day work.


Rob Hall: Yeah, that’s right. It takes time. And it has not been, you know, our primary, our primary responsibility, day in and day out.


Jay Cosgrove: Maybe someday, that’d be cool.


Rob Hall: Maybe someday, right? Not to go back a step. But I think for us to continue pushing on consistency will be helpful.


Jay Cosgrove: Yeah, I would agree with that. I think one thing that we’ve done a decent job, or specifically, our marketing department has done a decent job of his promotion. And we’re really new and just figuring out how to not only market ourselves, but to market the podcast is a different thing. So I think we’ve got some work to do there this year. 


Rob Hall: Agreed. 


Jay Cosgrove: Longed for. What do we want to add that we did not have in 2020? For our podcast, VIP guests?


Rob Hall: Yes. Like really super special, fantastic. Semi celebrities.


Jay Cosgrove: It’s not to say the people that we had on the podcast weren’t super special. They’re VIPs in our heart.


Rob Hall: I still think you’re a VIP Nick.


Jay Cosgrove: Did you call me Nick?


Rob Hall: I still think you’re a VIP Jay.


Rob Hall: This coffees decaf.


Jay Cosgrove: That was like a Ron Swanson moment. 


Rob Hall: That was totally a Ron ahah.


Rob Hall: That’s right.


Jay Cosgrove: I would agree. I think more interviews from people outside of DFS is definitely something that will add some nice spice.


Rob Hall: Yes, I agree, Fred.


Jay Cosgrove: Another thing I would throw in and that I’d written down when I was thinking about this is crowd participation. So this is something we’re gonna definitely strive towards. We’ve already talked about it a little bit. But we’d like to get you guys the listeners more involved with asking questions or adding commentary on podcasts or topics that you would like us to talk about?


Rob Hall: Yeah, I think for me, Jay, something I longed for is the podcast, eventually creating deeper conversations for our organization as a whole. So for it to continue being a path for doors to open.


Jay Cosgrove: Yeah. I think that’s good, man.


Rob Hall: Yep. Thank you for listening to the experience lab from Digital scientists. To learn more about our team in the great work we do or even hire us, visit our website at Digital scientists.com