That's not Agile!
I cannot tell you how often these words are uttered these days. Arwafn has gone over the cliff, and Nerdhaven has had to publicly walk back from its launch posture of "Wow, look at this!" to the less inspiring message of "Watch this space."
Upper management is involved to an alarming degree.
Faced with the possibility of completely shutting down Arwafn, dev effort was flung at the TURN issue to determine which customers could retain the feature and which could not. The SMEs drew a line in the sand and in the end, about a third of our customers could keep the feature.
Which would be encouraging, if it weren't for the filters. Arwafn has more than a few, and two of them in particular are near total flops. Both are based on fields that customers have never been required to supply. We'd assumed they were, and our limited beta tests looked promising. Data flowed in, and the filters acted on that data.
Trouble was, that data was crap. For some customers, Filter A's key field was being populated only 20% of the time - or was co-mingled with another field, rendering it unusable. In beta, these filters appeared to work, but what they would show was wildly inaccurate.
In beta - our users were not doing a rigorous comparison between their system's data, and what Arwafn was showing. Our users viewed the beta sessions as a preview - play time. And while they surely care about the accuracy of the report - they evaluated Arwafn based on what they imagined it was doing.
Fellow UXers of the world: learn from me. Spell out exactly what you want your beta testers to do. If you need them to do system testing, you need to tell them that. Regardless, you should ensure that your feature has system testing with live data before you green light it for wider release.
But some customers had good data. And there were options for us to explore - an alternate key field might get us there. It would mean an immense amount of re-tooling, though. The thought of it made my mind hurt.
Filter A was the good one.
Filter B was based on a field that not only had issues with being populated only some of the time, when it was populated it was almost always the right kind of data - at the wrong point in time. Filter B is supposed to present its value as of a specific moment - and it was showing a value that was a few days later. And those few days were killing us.
We had a few customers who were happy to get the data at all - even asked for Filter B to be left on despite the time shift - but Arwafn is supposed to follow a standard. A few days later was not the standard. With over 100 clients active on the report, we only found 1 who had a working Filter B.
Worse, when I spoke with Data about our prospects for getting Filter B working, I read out the point in time requirement and Data just groaned. "If we ask the customers to send us an entirely new set of data, we could get closer to what is needed. But the literal criteria for Filter B does not exist in any data element that I know of."
For the millionth time, I ask myself - How is it I am learning this now? How did we get to the point where we have code in production, and I literally just found out that the data point I need does not exist?
And I don't have a good answer for that.
UX was primarily focused on the front end stuff - what the user sees and touches - and the back end stuff (where the digital bodies are buried) was supposedly the realm of other smart people. Our former UXDir encouraged us to provoke the change by asking for things without looking under the hood. "We'll figure it out," they'd say, "we can't design based on just what we have now, or we'll never make new things."
And there is a point there. If every design is vetted against the current codebase, legacy code limits what your customers get.
Fellow UXers of the world: learn from me. If a feature is dependent on a lot of user supplied data, design it to fail gracefully when some or all of those elements don't show up.
That's pretty basic, but its one thing to nod at a good idea in principle, it's quite another to realize that your feature is built on a house of cards while you are working on it. When all of this was happening, it all seemed so...right. And now Arwafn was a pile of roadkill in the middle of the road.
* * *
And everyone was rushing in to help. Which is great, but with upper management calling folks out, the agile culture in NerdHaven was beginning to crack.
Upper management said: Turn off Filters A and B. NOW.
And NerdHaven jumped to obey. Devs created branch code, stopped all other feature work and slammed two shutoff switches for Filters A and B into production. Devs were up in arms, because upper management never tells NerdHaven to work on this feature or that feature - and now they were.
But a fire was burning and this would put it out, so everyone got it done.
Next up was the TURN problem - and dear God what a mess. Telling upper management that the key value for Arwafn is being adulterated by our product was the easy part. Management realized there was a problem and TURN fit that bill. Resources were allocated to fix this issue and commandments were given.
FIX THIS. NOW.
The problem was, as we fixed TURN for Arwafn, we realized that TURN was used all over our product - and fixing it for just Arwafn would mean that other features would be out of synch with Arwafn.
Mind you, I'd always thought of Arwafn as a modest feature - and it is. Just a little report in one corner of the product. But TURN is a value that is used in the NumeroUno feature, and Nachen work, and more besides. I spoke with Sprint when we started fixing TURN and they gave me the full list of what they knew would be affected and it was alarming.
Then, in the middle of one of our situation room meetings for Arwafn, I started spelling that out to the group.
This launched another round of anxiety telephone and soon fixing the TURN issue for Arwafn, became fix TURN everyGoddamnWhere.
Which meant the scope of work just ballooned - which naturally had to be reported up the food chain.
Upper management's response was predictable:
FIX EVERYTHING NOW. AND SWEAR IN BLOOD THAT THIS WILL BE THE LAST OF THE TURN PROBLEMS.
Now, you're an agile dev shop and you get this kind of directive. What do you say to that? There is absolutely no guarantee that we will get everything in one go.
The SMEs are being asked to sign in blood that everything is fine from their perspective - and they are already anxious about the current state of things. They don't see everything and now they want to see EVERYTHING.
We've got SMEs crawling down our necks asking for more access and more analysis. And then Gumby's asked to provide proof that UX has discussed our designs with SMEs (in a meeting, with no warning - so they don't have that information, which itself becomes a thing)...
...and now we're careening towards Waterfall.
Waterfall: an epithet in dev shops. Shorthand for let's add more analysis and more documentation and more signoffs and when that doesn't result in a perfect product - slow down and add still more analysis/docs/signoffs. Lather, rinse, repeat.
All the UX folks hate this idea with a passion, and I pointedly ask if this change is the new way of things going forward, or just until we get Arwafn/TURN fixed.
Gumby is direct, "this is going forward."
I feel like throwing up. Yes, we steered the company in the ditch, but we learned a boatload and absolutely nobody wants to go through that again. We will be more vigilant, and we don't need more goddamn documentation.
I point out (for the umpteenth time) that more SME contact in our process would not have caught our data issues. The SMEs don't go under the hood any more than UX does. But we have experts in house who do, and we will be joined at the hip with them the next time we roll out features driven by client data.
Let's do that, I say, before we get all process heavy. We are already getting berated for being slow, all this bloat is not going to make software happen faster.
Wasn't this why Agile was invented? to prevent this kind of stuff?
But the powers that be are unassailable - we will add process and hope this will get better - or blow over.
Or something.
* * *
Today was a day that threatened to be the end of all things. I was roped into three meetings in a row, one about some future data issue I knew absolutely nothing about, one with the SMEs to discuss their (well aged) laundry list of desired features (that have absolutely nothing to do with Arwafn/TURN), and one that I was dreading most of all: a review of all our Arwafn/TURN work with the SMEs.
The laundry list meeting was a total bust - the SMEs want us to build what they want (as opposed to what Product has prioritized, or what the customers want). Their list of wants will solve issues that the SMEs have experienced, but may not do much to address root causes of user pain. And it is very unclear if all the items on the list should be solved with more computer code. Some might literally be user error. And all of the items are broadly worded concerns that cannot be vetted without specific examples of the behavior in question.
Sprint is in the meeting with Gumby and I and (heroically) volunteers to vet the list for the SMEs. I say (as diplomatically as I can) that this list should be given to our support staff, who can explain the inner workings of the application and perhaps get to the root of the matter without us trying to build a solution based on a speculative issue. I'm not sure the message took, but Gumby and Sprint are clearly way ahead of me. They want to do right by the SMEs, but route support work to support resources.
And then the review meeting. I wanted to keep my hackles in check, but this had all the hallmarks of being a "why did you put the button there?" kind of discussion. One of the SMEs, Ihaq, had ridiculed our design of a new form and I was dreading a new session of "Move that button, because I want you to."
The lead SME was Voldemort: brilliant, hardworking and overworked beyond all reason. They started out with a few probing questions which Gumby fielded for the first few minutes. Ihaq was there and clearly confused about what we would be running through. They had a few questions they wanted to run through before we got started. Ihaq's questions are never brief, and usually involve follow ups.
It was getting awkward.
When we got to the list of feature work, Gumby gave me the podium.
At the risk of appearing arrogant - I am pretty good at explaining things. So long as I understand something, I can transfer that information to pretty much anyone.
And the work we've been neck deep in for the past two months is stuff I can't help but understand. I wake up in the middle of the night thinking about this stuff.
Grok it? Hell, I wrote it.
And I go to town. You have a question, Ihaq? I got you. Voldemort wants clarity on the hideously technical matter of historical TURN values? I got you, too.
And as soon as I realize that the SMEs are (rather predictably, I might add) solid professionals trying to do their jobs - the tension goes out of the meeting. The SMEs are not there to grill us, or re-design our work - they are literally trying to understand where the work is now and where it will be when we are all done.
Gumby asked our new UXer, Dash, to study our stakeholder demos and they pointed out a host of issues with how we present our work and how it is consumed and retained.
Predictably, shortcomings in our presentations leave lasting misunderstandings in the SMEs as well as others in our organization.
Having this Q and A session allowed us to close that gap for Voldemort and Ihaq at least, and at the end of it - it was a really good feeling.
Voldemort was exultant "That was awesome."
And I cannot tell you how much better that made damn near everything.
Yes, we have a metric sh!t-ton of work ahead of us, but we are beginning to round the corner and we've started to mend things with the SMEs.
The technical work is in flight, the release date is set. Our culture has taken a massive hit to the jaw.
But if today was any indication-
We can do this.
No comments:
Post a Comment