Java Posse #428

The Java Posse

Roundup ‘13 - Devops and Culture Change

Fully formatted shownotes can always be found at http://javaposse.com

Extra thanks to D.J. Hagberg for the excellent and complete show notes.

- Netflix "Freedom & Responsibility" - Every team can organize as they choose

- "Firemen" who get paged when system goes down. Sets best practices

- "Site Reliability Team" - sets best practices: deployment, monitoring, ...

- "Cloud Performance Team" - sets best practices for cloud performance

- Groovy

- Python

- Standarization: set up practices/tools such that choosing the "Standard" makes life easier

- Regulatory constraints (Sarbanes-Oxley, PCI DSS)

- Communication with Devops: Build, Deploy, Release

- Documentation: in many cases, auditors are happy with scripts

- tl;dr

- Adrian Cockcroft - InfoQ, YouTube, @adrianco

- Opportunities for Ops to deploy systems, devs to assist & learn from Ops

- #monitoringsucks

- Uptime and Availabitility - metrics

- Wake up the developer at 3am when they break things. Devs need to understand Operational Risk of changes

- PCI Compliance mitigation ideas

. Separate out the SOX/PCI regulated parts of application, pared down to most minimum possible

. Put those parts under separate accounts, machines, audit trails, etc.

- Metrics: what do you use to measure? Customer Happiness, Money, Loyalty? (Netflix: "movie start metric")

- Adobe Cloud Services

- Quicken cloud

- Twitter - measure sentiment (for consumer-facing/high-profile services)

- Automation: bite-size, recoverable pieces

- Eiffel design-by-contract concept: preconditions/postconditions

- Redefining the job of operations

- Executive sponsorship

- QA does the deployment through DevOps-provided tools

- Post-Mortem, military AAR, Incident Review, Agile Retrospective: have thresholds for Big Meetings vs small outages/rollbacks

- Metric: time to get new feature to the customer

- Self-directed improvement: Deming/Lean improvement cycles - Kaizen

- Visible Ops Handbook - Gene Kim

- The Phoenix Project (book) - Gene Kim

- business Case for improvements? What was cost of the last outage?

- Chase Manhattan - 3 day outage in 2010

- Every business in the future will be a software business

- Move developers into Ops teams - Google engineers get extra "hazard" pay/leather jacks - DevOps Culture Hacks

- Practices brought back from Ops to Dev

. JMX: Measure/Instrument *all* the things (esp. consumer facing)

. Vagrant/Opscode Chef/Puppet stack into Dev environment - exercise deployment on every Dev change.

- "Foreign Exchange" (hostage exchange?) program

- "Don't make customers modify XML Files"

- Static Config to Files. Stays the same for release.

- Dynamic Config - looked up from service - Archaius - Netflix

- Feature Switches/Feature Flags - Etsy

. Enable/disable features & settings at runtime

. Hierarchical/tagged/scoped configurations

. Use config changes for rollback

Thanks

• Libsyn.com - http://www.libsyn.com - for hosting and bandwidth

• Feedburner.com - http://www.feedburner.com - for feed redirect

• Kirsty Doherty, Amy Ehmann for Java Posse artwork

• Theme Music:

• Opening - "Java" the parody song Copyright 1997 Broken Records and Marjorie Music Publ. (BMI),

• written and performed by Loose Bruce Kerr of the Dr. Demento Show and Sun Microsystems attorney.

• Based on the WWI popular song, "Ja-da." Ukelele style on the recording taught to Bruce by his dad.

• Re-produced with kind permission from "Loose" Bruce Kerr - http://loosebrucekerr.libsyn.com http://www.youtube.com/watch?v=TAX0gJt-aZg

• Closing - Juan Carlos Jimenez - In the House (Intro No. 1)

• To contact us:

• Visit our homepage - http://javaposse.com

• Post on our Google Group - http://groups.google.com/group/javaposse

• Pose a question on our Google Moderator group - http://tinyurl.com/q4javaposse

• Call us with questions and feedback - [phone]

• Or send us email - [email protected]

The Java Posse consists of Tor Norbye, Carl Quinn, Chet Haase and Dick Wall