PortfolioCloudera / Case study
Try the prototype
Case study · Cloudera Data Platform · Shipped 2023

Cloudera Data Platform Upgrade Design

Upgrading a data lake in Cloudera Data Platform means downtime, prerequisites and a risk of losing data. The old flow showed admins none of that up front, and many put upgrades off. I redesigned it to let them plan the upgrade, back up their data and check they were ready before starting.

Try the prototype WireframesCloudera docs
LiveA new version is availableTry it yourself
Role
Product DesignerResearch to handoff
Team
1 PM, engineering podWider design critique
Timeline
2 months, 2023Discovery, then design
Methods
~20 interviews11-platform benchmark
Deliverables
Shipped UIFlows, wireframes, a pattern
ShippedLive in production, and still running

The flow is described in Cloudera's public docs, linked at the end.

AdoptedA pattern that spread beyond the feature

My vertical tabs are now part of the Cloudera Design System.

01 · Context

A routine task that kept getting postponed

A data lake in Cloudera Data Platform holds a company's data and the security and compute around it. Upgrading its runtime is routine maintenance, but it takes planning.

Disruption
It needs downtime

Workloads stop, and every Data Hub cluster has to be stopped first.

Failure
It can fail partway

Prerequisites showed up too late.

Loss
It can lose data

A failed upgrade without backup loses metadata.

Admins had to work all of this out on their own, and many chose to wait. Each delay left the environment further behind on security fixes and made the next upgrade bigger.

02 · Research

Talking to the people who sit with customers

Admins are hard to reach directly, so I spent the first month with the people who hear their problems every day: Cloudera's sales engineers and customer stakeholders.

~20
Interviews

With sales engineers.

11
Platforms benchmarked

How peers handle upgrade risk.

4
Themes

Clustered from the interviews.

BenchmarkedSnowflakeDatabricksTeradataOracle CloudIBMAWSGoogle CloudMicrosoft AzurePalantirSAP HANASAP Datasphere
Read the interview synthesis: quotes and affinity map

What I kept hearing

"Customers don't know what's going to happen when they click upgrade. So they don't click it."Sales engineer
"The upgrade failed because a Data Hub was still running. Nothing told them to stop it first."Sales engineer
"We need to plan a window and explain it upward. We can't do that if we don't know how long it takes."Customer stakeholder

Affinity map: four themes

Can't see ahead readiness

No idea what happens after clicking upgrade
Prerequisites not surfaced
No readiness check before committing

Can't plan for it planning

Unknown duration, disrupts the workday
Can't compare current vs target version
Hard to justify the window to leadership

Might lose data recovery

Backup on a separate tab, easy to forget
A failed upgrade without backup loses metadata

Fall behind drift

No signal that a newer version exists
Upgrades slip for months at a time

Four insights that shaped the design

Readiness

The upgrade worked. Information didn't.

Planning

Admins must justify the window first.

Recovery

A step on another tab gets skipped.

Drift

Nothing said a new version existed.

03 · Persona, journey and jobs to be done

Who I designed for

A composite of the Environment Admins described across the interviews.

Marcus

Environment Admin · owns his company's CDP environments

Composite · from ~20 interviews

"Before I click upgrade, I want to know it'll work and how long it'll take. Otherwise I'd rather wait."

Goals

  • Keep environments secure and current
  • Plan a window he can defend to the business

Frustrations

  • Upgrades failing on prerequisites nobody mentioned
  • No idea how long it will take
See the stakeholders

Who else it had to work for

Environment AdminConfidence to run it, and a way back.
Business / leadershipSupported versions, planned downtime.
EngineeringReal checks that fit the platform.
Sales engineersFewer stuck customers.

Current-state journey: where confidence drops

StageThinkingDoingConfidence
01NoticeCan this wait?Hears about new versions late, or never
02DecideHow long will it take?Guesses what will change
03PrepareDid I miss a step?Checks prerequisites from memory
04RunIs it working?Clicks upgrade and waits
05RecoverWhere's my data?Reads logs, opens a ticket

Main jobs to be done

J1

When a new runtime is out, I want to know I'm ready before I commit.

J2

Before I upgrade, I want my data backed up without leaving the flow.

J3

When I plan an upgrade, I want to see what changes and how long it takes.

04 · Problem and approach

From "I'd rather not" to "I'm ready"

The problem

Admins started upgrades without knowing if they were ready, with backup on another tab and no signal that a newer version existed.

Design challenge

How might we turn an upgrade admins postpone into a planned operation they're confident to run?

Challenges

Real constraints

Checks had to be real

Agreed with engineering what to validate.

Existing system

It had to fit the console

Built to Cloudera Design System rules.

Time

Two months, one designer

Focused on what built confidence most.

Opportunities and solutions

OpportunityCheck readiness first
SolutionValidate and Prepare

Runs first, with no downtime.

OpportunityMake failures actionable
SolutionNamed, linked blockers

Each cluster to stop, with a link.

OpportunityHelp admins plan
SolutionClear information up front

Current vs target, support window and estimated time.

OpportunityPut backup where the risk is
SolutionBackup with Upgrade

Recommended inside the flow.

OpportunityKeep upgrades visible
SolutionThe Available badge

Stays until the lake is current.

The flow always

  • Checks readiness with no downtime
  • Names what's blocking, with a link
  • Recommends a backup first

The flow never

  • Starts before readiness is known
  • Fails with "something went wrong"
  • Nags with a popup to dismiss
05 · User flows

No path ends at a dead end

The system checks what it can before the admin commits.

Start / endScreenReadiness checkSystem actionDecision
06 · Wireframes

Low-fidelity wireframes

Greyscale, to test structure before style. Blue marks what I redesigned. Tap a pin to see its note.

    07 · Final design and prototype

    Built on the Cloudera Design System

    Built from Cloudera's existing components, with each safety step in its own card so the page reads top to bottom.

    Navy, blue and info tint from the Cloudera Design System. Red only for the badge.

    1 / 7

    Click the screen to enlarge

    Click through the real screens

    Pick a journey and click where an admin would.

    Management Console / Data Lake / Upgrade
      On screen

      08 · Design system contribution

      The pattern that outlived the project

      Horizontal tabs couldn't hold the features on the roadmap. I made the case for vertical tabs.

      Before · horizontal
      Event HistoryEndpointsTagsNodesNetworkUpgradeMore ▾

      Each new feature pushes the rest behind "More".

      After · vertical
      Event HistoryEndpointsTagsNodesNetworkUpgradeBackup+ room to grow

      All sections stay visible as new ones arrive.

      How it got into the system

      01 · SpotSaw the limitMore sections were coming.
      02 · DesignDesigned optionsSeveral versions, one agreed.
      03 · SystemiseAdded to the systemNow used across Cloudera products.
      09 · Design decisions

      The trade-offs I made on purpose

      DecisionTrade-offHow I handled it
      01A readiness check firstSpeed wasn't the problem. Confidence was.
      Trade-offOne more step before Upgrade.
      How I handled itNo downtime, and it prepares the upgrade.
      02Failures name the fixThe system already knows which clusters are blocking.
      Trade-offMore engineering work.
      How I handled itStarted with the most common blocker.
      03Backup in two placesAdmins also back up outside upgrades.
      Trade-offThe same feature in two places.
      How I handled itThe same card in both places.
      04Detail one click awayBuild numbers matter to some, not all.
      Trade-offFull detail is hidden by default.
      How I handled itA plain summary, with details inline.
      05Alerts in the navigationPopups get dismissed. Navigation doesn't.
      Trade-offA badge can fade into the background.
      How I handled itIt shows only when an upgrade exists.
      10 · Outcome and metrics

      Shipped, documented, and adopted

      I left before launch metrics came in. Here's the shipped work, and how I'd measure it.

      Cloudera docsUpgrading a Data LakeValidate and Prepare, backup, target version Cloudera docsBacking up a Data LakeSchema, lineage and audit backup Cloudera docsData Lake informationThe Data Lake details page
      How I'd measure success

      How I'd measure success

      AreaIntended outcomeIndicatorHow I'd measure
      ReadinessFewer failed upgradesUpgrades that fail on a prerequisiteFailure rate for upgrades with and without Validate and Prepare first
      RecoveryMore upgrades with a restore pointBackup attach rateShare of upgrades started with Backup with Upgrade on
      Self-serviceAdmins fix blockers themselvesClicks from failure links to clustersValidation failures resolved without a support ticket
      CurrencyEnvironments stay currentTime behind the latest runtimeMedian days between release and upgrade, before vs after
      SupportFewer escalationsUpgrade-related ticketsTicket volume and sales engineer escalations per quarter