Wow. This is such a great post, and sums up exactly why every implementation of "Agile" I've seen at previous employers have been completely dysfunctional.
So there's an engineering team and they have a project manager. The post-its and standups and storyboards seem a little weird at first to the engineering team, but they go along with it. It's rarely perfect in the first iteration but they can see how doing this makes them more productive, and it beats getting a bunch of confused requirements from confused business owners, or working off cumbersome and inflexible product requirement documents. And it doesn't ask them to change much in the ways they're already communicating.
The problem is the project manager needs a way to provide "remote views" that the OP mentioned. The CEO needs to know where the implementation hours are being spent. The finance team wants to know which projects they can capitalize. And so forth. It is part of the project manager's responsibility to provide this.
This is where tools can be great. Enter in some data, enter in the stories and tasks and points and hours and projects and generate a dozen charts and graphs with the push of a button. Cool.
The problem is if the project manager doesn't manage this themselves, if they don't do the tedious task of entering in this information themselves, then it's going to be a mess. Because like the OP said, engineers aren't going to update the tool. They want to do work and if the work status needs to be updated to reflect their progress, that should just automatically happen. If they have to manage the updating, then you immediately have a point of failure because the engineer is never going to do this. Why? Because it's pointless. He's working on projects. He's cranking out code. He already has a dozen windows open on his desktop at all times to text editors, terminal shells, documentation on the web, documentation in PDFs. He already communicates to his peers via IM, email, in person, using version control, or even moving post-its on a storyboard. He is already passively indicating his status a dozen times a day. Why does he need to actively manage his status in some tool that has zero value to him or anyone he works with?
But the project manager implores, "just update your hours, guys, it only takes a little bit of time each day. We really want to maintain an accurate burndown chart."
If only it were so simple. Because beyond being completely inconvenient, the tool doesn't even use the same units of measure that he does! "Hours per story?" Does that count the hours brainstorming the solution? The hours actually typing in code? The hours working with the sysadmins to deploy the code? The hours (well, hopefully minutes) spent thinking about the project while taking a dump? If the story is closely related to another one that he basically worked on both at the same time, should he combine his hours and divide by two? And burndown chart? Does the burndown chart mean anything to him? Does it help or improve his life in anyway? We thought this project was going to take a week, but when we started getting into the implementation we realized there's huge hurdle X and now it's going to take two weeks. There's your damn burndown chart.
So the engineer hears, "just wax and shave your balls with vaseline, guys, it only takes a little bit of time each day." Because waxing and shaving his balls seems just as pointless as updating the tool. And even if it only takes a few minutes to wax and shave your balls, it's awkward enough that you will get really frustrated really quickly if this needs to become part of your workflow.
If the project manager wants to the use the tool because of the convenience and power of generating reporting charts and graphs to external teams, then good for them. But if they're going to ask the engineers to update it themselves, then they should be prepared for that to be as successful as if they asked them to shave their balls, because that is literally what they are doing.
First implementation of agile/scrum I worked with was a bunch of printed cards nailed to a board where in the daily meetings the PM used a pen to take hours out of it, or add more if we decided it was going to take more. After the meeting he took the cards to his computer and wrote some kind of report (no idea the format). Second company about the same except PM didn't do the report after but still post its on the wall. And it all worked great, then I got to the third company and no board, just tickets in Team Foundation the programmers had to update every day (heck, they even asked us to update various times a day). We had a daily meeting but not much else.
Truth be told, I usually just forget about the hours update, whenever I closed a ticket I just looked at the estimated time that was there (which wasn't decided by me) and put that as working hours. I was bugged everyday to keep my hours up to date because it was very important some higher up had a general idea on how things were going...
Here is the thing, first two places started using agile/scrum by reading online and books, third one payed a consultant more than 25K to implement this system.
You are absolutely right that manually updating hours is a huge chore and not surprisingly it is universally despised. Making estimates beforehand in scrum is also problematic because it doesn't involve the time spent in previous phases as you pointed out. Kanban, however, solves the problems inherent in scrum, and virtually all digital kanban solutions automatically track the time for you in each phase of your workflow.
So, it is possible to measure productivity relatively painlessly in kanban as it limits work in progress (discourages multitasking, reveals bottleneceks in your workflow). Kanban also takes idle time into account (ex: time spent waiting approval from the management or another dept). See http://flow.io/how-kanban-can-help-you-measure-productivity.... for a brief overview of kanban's benefits.
Disclaimer: I am the author of flow.io, a lean project management app based on kanban.
Wow. This is such a great post, and sums up exactly why every implementation of "Agile" I've seen at previous employers have been completely dysfunctional.
So there's an engineering team and they have a project manager. The post-its and standups and storyboards seem a little weird at first to the engineering team, but they go along with it. It's rarely perfect in the first iteration but they can see how doing this makes them more productive, and it beats getting a bunch of confused requirements from confused business owners, or working off cumbersome and inflexible product requirement documents. And it doesn't ask them to change much in the ways they're already communicating.
The problem is the project manager needs a way to provide "remote views" that the OP mentioned. The CEO needs to know where the implementation hours are being spent. The finance team wants to know which projects they can capitalize. And so forth. It is part of the project manager's responsibility to provide this.
This is where tools can be great. Enter in some data, enter in the stories and tasks and points and hours and projects and generate a dozen charts and graphs with the push of a button. Cool.
The problem is if the project manager doesn't manage this themselves, if they don't do the tedious task of entering in this information themselves, then it's going to be a mess. Because like the OP said, engineers aren't going to update the tool. They want to do work and if the work status needs to be updated to reflect their progress, that should just automatically happen. If they have to manage the updating, then you immediately have a point of failure because the engineer is never going to do this. Why? Because it's pointless. He's working on projects. He's cranking out code. He already has a dozen windows open on his desktop at all times to text editors, terminal shells, documentation on the web, documentation in PDFs. He already communicates to his peers via IM, email, in person, using version control, or even moving post-its on a storyboard. He is already passively indicating his status a dozen times a day. Why does he need to actively manage his status in some tool that has zero value to him or anyone he works with?
But the project manager implores, "just update your hours, guys, it only takes a little bit of time each day. We really want to maintain an accurate burndown chart."
If only it were so simple. Because beyond being completely inconvenient, the tool doesn't even use the same units of measure that he does! "Hours per story?" Does that count the hours brainstorming the solution? The hours actually typing in code? The hours working with the sysadmins to deploy the code? The hours (well, hopefully minutes) spent thinking about the project while taking a dump? If the story is closely related to another one that he basically worked on both at the same time, should he combine his hours and divide by two? And burndown chart? Does the burndown chart mean anything to him? Does it help or improve his life in anyway? We thought this project was going to take a week, but when we started getting into the implementation we realized there's huge hurdle X and now it's going to take two weeks. There's your damn burndown chart.
So the engineer hears, "just wax and shave your balls with vaseline, guys, it only takes a little bit of time each day." Because waxing and shaving his balls seems just as pointless as updating the tool. And even if it only takes a few minutes to wax and shave your balls, it's awkward enough that you will get really frustrated really quickly if this needs to become part of your workflow.
If the project manager wants to the use the tool because of the convenience and power of generating reporting charts and graphs to external teams, then good for them. But if they're going to ask the engineers to update it themselves, then they should be prepared for that to be as successful as if they asked them to shave their balls, because that is literally what they are doing.