Académique Documents
Professionnel Documents
Culture Documents
Or,
08/08/09 1 jbullock@rare-bird-ent.com
Data-Driven, Experi ... Huh?
Lots of ideas about how to do software development
● Many can work "well enough" (& plenty of folks happy to tell you)
● Ideas are the easy part
● Changing how you work is hard
● How you go about changing determines what takes
What?
● Data-Driven, Experiential, (In Context), (as a Team)
How?
● Support the Fact of Change,
● (Realize) It's Knowledge Formation,
● Your Process is What You Do (so start where you are)
● (Align with) Mission & Data
08/08/09 2 jbullock@rare-bird-ent.com
Clue-Stick Stories
08/08/09 3 jbullock@rare-bird-ent.com
Claim Check
What How
Data Driven ? Support the Fact of Change ?
Experiential ? Your Process is What You Do ?
(in Context) ? Knowledge Formation ?
(as Team) ? Mission & Data ?
Seems to Work? ?
Otherwise fails? ?
Implies a Strategy? ?
Thanks to SASQAG for this idea.
08/08/09 4 jbullock@rare-bird-ent.com
NettyCo – From Many, One
Corporate
Net Mobile
Clients
● Consistent IP
● Continuously Available
08/08/09 5 jbullock@rare-bird-ent.com
If at First You Don't Succeed ...
In the Beginning "NPI Process" Third Try
~18 Months
~13 Months of "Oh my god!"
● ATT sez,
● 90% Y/Y Revenue Growth
● Intel sez
● New SW Release
● Motorola sez,
● GE sez
●
1st New HW Prod in 3 Yrs
● At Risk Pilots from 17 to 0
Results FAIL
● >4x Dev Throughput Increase
08/08/09 6 jbullock@rare-bird-ent.com
NettyCo Business Results
Releases ●
●
SW 14 months Late
No New HW for 3 Yrs
●
●
SW Maint Release
New SW Release
● "The best HW we've ever
shipped." - SW Architect
● "Slipping" 1:1 - 2:1 ● New HW Product ● "The best SW we've ever put
Channels ●
●
Babysitting Relationship
Certifications Hellish
●Release Cert with 1 issue
(New Req. on HDWR)
●5 New Channels, Cert w/o
Engineering Knowing
● Threat to Abandon
●
Closed all "at Risk" pilots
Sale-Driven Feature Order
● >90% revenue growth
Product Dev ●
●
Exhausted
Morale Shot
●Having to chase them
home.
●
●
>4x throughput gain
~ 8x reduction in support load
08/08/09 7 jbullock@rare-bird-ent.com
NettyCo Process & Results
Prior
"SPI" "Do the work" – change for results.
Q3 Q4 Q1 Q2 Q3
Prod ● NPI Binders
● All at once Roadmap
● Release Gating
● Release Scope
● Product Definition ●
Product Line
Mgmt
Engr ● Task-Planning
● Product Slideware
● Code Line Mgmt
● Integ Gating
● Tier III Sppt
● Kanban
● Architecture First
● Tech Debt Aware
● Project Slideware ● Mission/Measure ● KM / wiki ● Fix Propagation
● Mission/Measure Assessment(s)
● Articulated Mission
● 2:1 dev slip ● Maint Release w 0 ● Field stability ● Sales up 90% ● SW Release
● >80% install fail Showstoppers ~30x Better ● At-risk Pilots ● New HW Product
● ~ 0% fix success ● 85% Fix Success ● Off "Watch List" From 17 to 0 ● 4 -5 Adtl
● Fantasy Process ● 9 cust saves ● Dev progress 1:1 ● 1-Pass Cert Channels
● Ship Showstoppers ● Channel cert on HW & SW ● Mkt Parity
● Partners Deserting
08/08/09 8 jbullock@rare-bird-ent.com
Successes - Hard & Soft
Who Says? - Dictated from above Who Says? - Guys doing the work
08/08/09 9 jbullock@rare-bird-ent.com
How'd That Happen?
Current pain point - with measure and declaration of better
Each change based on a theory of how stuff works
"Try it" - with time to adopt and willing to adjust prescription
Contextual targets - Coverage! (Nope.) Defect count! (Nope.)
● (Know, shipped defect count actually went up for first 2 quarters)
● Response time for & confidence in point fixes
● ("Time to spaghetti-toss" actually went up initially)
● Throughput of & agility to hit emerging requirements
● Got vs. think we've got. Customer custom vs. product / roadmap
Consistent - Only worked when we agreed. (Sometimes by fiat.)
● "SPI" drove owned agreements vs. theoretical process
08/08/09 10 jbullock@rare-bird-ent.com
Claim Check - NettyCo
How To B4 After
Claim B4 After Support the Fact of Change N Y
08/08/09 13 jbullock@rare-bird-ent.com
A Tale of Two Systems
Basselope-Control
IRS Mgmt
Basselope 116 CIs
Needs a Brain 1,500 Interfaces
10**6 Payloads
>6*10**6 SLOC
● Tend to serialize
08/08/09 14 jbullock@rare-bird-ent.com
The Rigid and The Clueless
IRS for the Basselope Brain IRS Management System
08/08/09 15 jbullock@rare-bird-ent.com
Both, Both "Agile" & "Defined"
IRS Tools
(Expanded, automated & (Incremental, ad-hoc
integrated) organization, dynamic schema
generation)
"Agile" ● Accelerate Interface Changes ● Schema Change on Demand
● Integrate Early ● Evolving Requirements ->adtl attributes
● Easier Communication & Collaboration ● Whole new functions: analytic extracts,
● Closer to the Code (IRS injected into change life-cycle management, online
source files by some sub contractors) browse & update
● Lightest "life cycle" that worked
"Defined" ● Massive, Elaborate Spec - More ● SCM & Change Management on Tools
Elaborate w / Automation ● Formalized ad-hoc db Definition
● Formal Change Process - More Formal ● Integrated with Program CM
08/08/09 16 jbullock@rare-bird-ent.com
What's "Better?" (Whack)
"SEI"-Land "Agile"-Land
More Definition Less Definition
Up-Front Post-Hoc
Impact-Driven Results-Driven
Serious Validation Validation Hostile
Formal Verification Ad-Hoc Verification
IEEE Computer:
Berkeley Breathed – Boom County
http://www.berkeleybreathed.com/ Improving the Development System Model
08/08/09 17 jbullock@rare-bird-ent.com
So, How?
Local agreed measures (directional is OK) – Data Driven
Automated IRS: req scale, change volume, process timing & error rate
DB: design change rate, requirements evolution, scut fraction
Adoption takes work – Experiential Learning
Automated IRS for "Basselope Brain" – Two tries to create it
Once present, multiple updates for sub-contractors to "get it"
DB (Schema Gen) – 3 cycles for dev to adopt. More for x-function
Drive to what matters to you (slogan-free, please) - In Context
Valuable and possible to "get it right"
Different "it" and "right" between IRS and Tools
Valuable and possible to "accommodate change"
Different "change" and "accommodate" between IRS and Tools
Reducing injected defects was secondary at best both times
Finding defects turned out to be a bonus
Shared understanding and practice matters – Change as a Team
08/08/09 18 jbullock@rare-bird-ent.com
Concurrent Big-Brain SPI
"Official" Sponsored Group doing SPI
● Parallel "development practices & etc."
● No constraints on solution
(a) For IRS look at learning across programs & across teams
(b) Direct boss – Larry "the Wounded Gorilla" – got it. Cross-function, not.
(c ) Their knowledge formation (½). Concept w/o testing (½, ½ * ½ =¼)
(d) Knowing what you need not sufficient.
08/08/09 21 jbullock@rare-bird-ent.com
Claim Check III - Basselope
Claim IRS B4 IRS DB B4 DB Hi IQ How To IRS B4 IRS DB B4 DB HI IQ
Data Driven N Y N (a) Y N Support the Fact of Change N ½(a) N ½(b) N
Experiential N 1/2(a) N Y N Your Process is What You Do Y ½(a) N ½(b) N
(a) Crossing scales: IRS across programs, DB single team / tool set
(b) Now have failure after / concurrent with two successes
(c) Know your mission. Realize you are ignorant.
● Learn by doing. Refine with local data.
● Test your solutions with real application.
● Bringing people along is part of the exercise.
08/08/09 22 jbullock@rare-bird-ent.com
Personal SPI (Change) Initiatives
Who? What Practices Strat Win?
Techtran Telephony App Builds & Baselines, Code Stds, Design & Arch Y Y
Carrier Heat Pump Controls Whole SDLC over 3.5 years. Y Y
CSC Vertical IT Change Mgmt, Defect Mgmt, Build / Release, Req Mgmt 1/2 1/2
dbI (Moore) Mfg "Order to Cash" Independent Test, Defect Mgmt, Integration / CM 1/2 N
dbI (Prod) DM Toolkit Hand-waving attempt at anything in an SDLC N N
dbI (Test) Automation SVCS Test design& automation for distributed n-tier systems. Y Y
dbI (DB) Y2K Banking Work packet workflow definition and automation Y Y
Harris Online Polling Proj life-cycle, Integration & CM, Risk-Based Test Strategy, Req Mgm Y Y
"Swamp" e-tailer Defect routing, Unit test & Messaging Auto -> Interface / API Mgmt 1/2 N
"Swamp" Life-Cycle Tools Unit Test, Tools in Repository, Continuous Build, Incremental Dev 1/2 N
"Midwest U" Alumni Assn. Requirements Mgmt. Project Oversight N N
Hightower SEM Appliances Manifest, Acceptance & Regression Testing 1/2 1/2
ARC Membership Org Auto Acceptance Test, CM / Release, Proj Planning, Scope Control & Req N N
"TradingCo" Exchange Auto. Req-DrivenScalable Performance Testing 1/2 1/2
"NettyCo" Multi-Protocol VPN Gating, Workflow, Design & Code Stds, Defect Mgmt, Test Coverage, Y Y
Req Mgmt, Smoke Test, etc.
08/08/09 23 jbullock@rare-bird-ent.com
Public Examples
Who Clue What Improvement Location
Oracle Dist Code Mgmt Custom repository Practice, Protocol, Participation http://queue.acm.org/detail.cfm?id=966796 (Steve Hagan)
Microsoft Xteam APIs Database of calls Practice ?Webcast '05? - Can't relocate
IBM Big-Q Quality Localization of error Process, Procedure, Policy Late 1990s, system SW, since removed ...
Twitter Uptime Matters Migrate to SCALA Practice http://www.artima.com/scalazine/articles/twitter_on_scala.html
Drizzle Code quality! Tune it all Practices, Policy, Participation https://launchpad.net/drizzle
LOLCats Pay attention Ongoing SPI Practice, Process, Policy http://startpad.org/countdown/cheezburgers-and-lolcats-look-
into-lazy-messy-backwards-way-starting-up
Linux / GIT Web of Trust Trusted Dist SCM Practice, Process, Protocol http://www.youtube.com/watch?v=4XpnKHJAok8
FDD Coad / Delivery perf Lifecycle w/controls Process, Practice, Procedure, Feature Driven Develoment,
DeLuca Protocol, Policy, Participation Process chapter in "The Color Book"
SCRUM Delivery perf. Lifecycle & protocols Process, Practice, Protocols The first SCRUM book
Crosstalk Mission / Everything Process, Practice, Procedure, http://www.stsc.hill.af.mil/crosstalk/2009/07/index.html
Context Protocol, Policy, Participation
Cleanroom Design Hard Statistical quality Practice, Procedure, Protocol, http://infohost.nmt.edu/~shipman/soft/clean/
Policy, Participation
Victor Basili Try it Empirical SE Process, Practice, Procedure, http://www.cs.umd.edu/~basili/papers.html
Policy
Bullock Data-Driven Do what it takes. Process, Practice, Procedures, http://www.ddj.com/blog/debugblog/archives/2008/08/five_ques
Protocol, Policy, Participation tions_61.html
08/08/09 24 jbullock@rare-bird-ent.com
What Changes Worked?
Kind of Changes Measures of "Better"
Process - SCRUM / Kanban Quality - Defects Shipped
- Staged Gate - Requirements Accuracy
Practice - TDD / Most Tools - Design Accuracy
- Languages / IDEs - "Technical Debt"
Procedure - Checklist (or Template) Perf - Throughput
- Coding Standards - Responsiveness
Protocol - Inspection Vote - Precision
- Stand Up - Efficiency
Policy - "Don't break the build." Control - Transparency
- Showstoppers - Estimates / Projections
Participation - On-Site Customer - Alignment
- Feature Team
08/08/09 25 jbullock@rare-bird-ent.com
Change That Works?
Institutionalized Cluelessness
Across
● Segments
● Stacks
● Company Life-Cycle
● Product Maturity
08/08/09 26 jbullock@rare-bird-ent.com
The Other Kind - Change by Telling
Inspections "Mature" KPAs "Structured" CM
"Agile" "Lean"
08/08/09 27 jbullock@rare-bird-ent.com
"If Only"
Sages of Software Engineering, invited to contribute to a book
"No, I'm retired and intend on staying that way."
"Thanks but no thanks, I tried to make a difference ... but look at things now."
"Sorry, not interested since it seems that software engineering has evolved
Into a science of excuse methodologies ..."
Me (private email):
"This sounds like 'Everything would be fine if only those people would do
what we big brains told them to.'"
Weinberg:
"If only."
http://secretsofconsulting.blogspot.com/2007/04/why-would-old-consultant-retire.html
08/08/09 28 jbullock@rare-bird-ent.com
"Resistance to Agile"
Lean / Agile SPI-er:
'I like <redacted>'s approach ...
"Do what I say. See that it works. Learn
why it works. In about 6 months I'll let you try
to improve on it for your context ..."'
So "Agile"
Different (but not really) payload.(a)
Same "Big Brain" delivery.
08/08/09 29 jbullock@rare-bird-ent.com
Claim Check – Big Brain SPI
Claim BB Victims
How To BB Victims
Data Driven Y(a) N(b) Support the Fact of Change N N
Your Process is What You Do Y(a) N(e)
Experiential Y(a) N(b) Knowledge Formation ½(d) N(e)
(a) Big Brains: used data, learned experientially, in their context, together
(b) Other people should respond like computers: "Just do as I say."
(c) Sometimes the victims throw them out as a team (start of gelling)
08/08/09 30 jbullock@rare-bird-ent.com
Claim Check II – Big Brain SPI
Claim BB Victims
How To BB Victims
Data Driven Y(a) N(b) Support the Fact of Change N N
Experiential Y(a) N(b)
(in Context) Y(a) N(b)
Your Process is What You Do Y(a) N(a)(c)
(as Team) Y(a) ½(c)
Knowledge Formation ½(b) N(c)
Mission & Data N(b) N(c)
Seems to Work? 1(d)
Otherwise fails? 1(d) (or N for Victims)
Implies a Strategy? 2(d)
(a) Big Brains, addressed their own situations, not someone else's
(b) Often introspection based and untested
(c) Don't start from their process, build (team) knowledge, or align w mission
(d) Worked for big-brains. Not for others. Hmmmmmm.
08/08/09 31 jbullock@rare-bird-ent.com
As a Team
08/08/09 32 jbullock@rare-bird-ent.com
"Team"
08/08/09 34 jbullock@rare-bird-ent.com
Experiential Learning
Wikipedia - "Experiential learning is learning through reflection on doing,
which is often contrasted with rote or didactic learning ..."
"Experiential learning focuses on the learning process for the individual ..."
Lewin - " ...learning is best facilitated in an environment where there is
dialectic tension and conflict between immediate, concrete experience and
analytic detachment."
Also Kolb, Dewey, Piaget
Peter Schon
Book - "The Reflective Practitioner: How Professionals Think in Action"
V. Basili – Empirical software engineering lab
“Experience in Implementing a Learning Software Organization" - K.
Schneider, J. von Hunnius and V. Basili
08/08/09 35 jbullock@rare-bird-ent.com
"Conscious Development"
Doing Learning
Science a Skill
08/08/09 36 jbullock@rare-bird-ent.com
Knowing This, Design SPI
Design an SPI process that is based on:
● SPI as Doing Science & Adopting a Skill
● Data-Driven, In Context
● Experiential & As A Team
● Supports the Fact of Change
● It's Knowledge Formation
● Your Process is What You Do
● Drive from Mission and Data
"... you can't live your life in the baby seat. You've got to stand on your own
..."
It's all been done ...
● TQM, Deming, Toyota, Lean, Critical Chain
● Satir, Schoen, Schein, Argyris, Senge
08/08/09 37 jbullock@rare-bird-ent.com
How - Use A Change Kanban
3 Round Trips w/
Product Backlog Iteration Iteration Iteration a New Thing
? ? ? How'd we do?
(Mission & Data)
"I have a theory ..."
08/08/09 39 jbullock@rare-bird-ent.com