Grokking the Solutions Architect Interview
Vote

0% completed

​

How a Customer Engagement Reaches Production

The Stages

Discovery and Qualification

Proposal

Proof of Concept

Decision

Migration in Waves

Launch and Review

Where Engagements Fail

Using This in an Interview

Key Takeaways

Practice Questions

Title image: the retailer's engagement as a timeline from the first call in week 1 to the last server switched off in month 15.
Title image: the retailer's engagement as a timeline from the first call in week 1 to the last server switched off in month 15.

Week 1: one call with the CTO. Week 4: a proposal. Week 8: a load test at 2,400 orders an hour. Month 3: a signed agreement. Month 9: a holiday sale with no outage. Month 15: the last server in the old data center is switched off.

These six points are the retailer's engagement from What a Solutions Architect Does. Between them are the stages through which most SA engagements pass, and each stage has its own output and its own common failure. This lesson follows the retailer through those stages in order.

The Stages

An engagement moves through seven stages: discovery, qualification, proposal, proof of concept, decision, migration, and review. The stages often overlap, and a small engagement may combine several of them, but the order rarely changes.

The seven stages of the retailer's engagement on one timeline, with the output of each stage written below it.
The seven stages of the retailer's engagement on one timeline, with the output of each stage written below it.

Discovery and Qualification

Discovery took weeks 1 to 3: one call with the CTO, then two workshops with the retailer's engineers. The workshops produced a list of about 60 applications running on the 220 servers, each with its owner, its dependencies, and how much downtime the business could accept during a move.

Qualification is the decision, made by the vendor and the customer, that the platform fits the problem and that the engagement is worth the effort of both sides. The retailer qualified easily: the platform could meet the sale peak, and the lease set a fixed date. A customer who needs a feature that the platform does not have should hear that early, because a late "no" wastes months for both sides.

Proposal

In week 4, the SA presented the architecture and the migration plan to the CTO and the retailer's architect. The proposal also recorded each major decision with the options rejected and the reason, so that the retailer's engineers could understand the design without the SA in the room.

One decision shows the format. The online store could stay on fixed servers sized for 2,400 orders an hour, or it could add servers with traffic. Fixed servers would be mostly idle for 51 weeks of the year, so the proposal chose servers that the platform adds automatically, and the proof of concept had to show that they arrive fast enough.

Proof of Concept

A proof of concept tests the riskiest assumption against success criteria that the customer agrees to before the test starts. Without agreed criteria, any result can be argued either way.

The SA and the retailer agreed on three criteria in writing:

CriterionTargetResult
Throughput2,400 orders an hour for 2 hoursMet
Page response time, p95Under 1 second0.7 seconds
Failed ordersUnder 0.1 percent0.02 percent

The p95 response time is the time that 95 percent of requests finish within. The test ran in weeks 5 to 8 on a copy of the online store with test data. The first run failed at about 1,800 orders an hour, because the database allowed too few connections. The team raised the limit, and the second run met every criterion.

The failed run was useful. It found a limit in a test that would otherwise have appeared during the sale.

Decision

The retailer signed in month 3. The proof of concept results, the cost estimate, and the migration plan were the technical case. The commercial terms were handled by the vendor's account manager, the salesperson responsible for the customer, and not by the SA.

Migration in Waves

The plan placed each application in a wave by three rules: the deadline that mattered most, the dependencies between applications, and the risk of the move.

WaveMonthsApplicationsReason for the order
13 to 6Online store and the order databaseMust be ready before the sale in month 9
27 to 10Store and warehouse systemsDepend on the order database, already moved
311 to 14Finance, HR, and the ERP systemLeast urgent, and the hardest to change

An ERP system (enterprise resource planning) runs core business records such as accounts, purchasing, and stock. Most applications were rehosted, moved to cloud servers without changes. Only the online store was changed so that it adds servers automatically, because it was the only application with the sale problem. The order database also moved to a managed database, in which the provider runs a standby copy.

Launch and Review

The holiday sale in month 9 reached about 2,600 orders in its busiest hour, which was above the tested 2,400, and the store stayed available. The last server was switched off in month 15, 3 months before the lease ended.

After each wave, the SA reviewed the running systems with the retailer's engineers against a structured checklist. AWS publishes one such checklist as the Well-Architected Framework, which groups its questions into six areas: operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. The review found, for example, that two applications in the second wave had no tested backup.

Where Engagements Fail

Each stage has a typical failure:

  • Discovery: the design answers the request but misses a constraint, such as a deadline or a team that cannot operate it.
  • Proof of concept: no agreed criteria, so the test proves nothing either side accepts.
  • Migration: a wave moves an application without the database it depends on, and every request now crosses between two environments.
  • Review: the SA leaves after the signature, and the customer's first incident in production is also its first contact with the design.

Using This in an Interview

A common question is how you would run a proof of concept. Answer with the criteria first, because they decide whether the test means anything.

"First I would find the assumption that would make the project fail if it were wrong. For a retailer with a sale peak, that is whether the store scales to 2,400 orders an hour. I would agree on the targets in writing before the test: throughput, p95 response time, and failed orders. Then I would run the test, report every failure, and fix it before the customer decides."

Expect two follow-up questions:

  • "What if the customer keeps adding requirements to the proof of concept?" Keep it to the riskiest assumption and a fixed time, and record the other requests for the full project.
  • "What if the test fails?" Report it the same day, find the cause, and test again. A failure found here is the reason to run the test.

Key Takeaways

  • Engagements move through discovery, qualification, proposal, proof of concept, decision, migration, and review.
  • A proof of concept tests the riskiest assumption against criteria agreed in advance.
  • Migration waves are ordered by deadline, dependencies, and risk.
  • The SA's work continues after the signature, through reviews of the running systems.

The next lesson, How Solutions Architect Interview Loops Test the Job, shows how companies test these stages in interviews.

Practice Questions

1. Why did the SA agree on proof of concept criteria before the test started?

<details><summary>Show answer</summary>

Because criteria set after the test can be argued either way. Written targets for throughput, response time, and failures let both sides accept the result. They also make a failure useful, because it is measured against a known goal.

</details>

2. Why was the store and warehouse wave placed after the online store wave?

<details><summary>Show answer</summary>

Because those systems depend on the order database, which moved in the first wave. Moving them first would have left them calling a database in the old data center. Waves follow dependencies so that related systems run in the same environment.

</details>

3. The first load test failed at about 1,800 orders an hour. Was the proof of concept a failure?

<details><summary>Show answer</summary>

No, because it found a real limit before the sale. The database connection limit would have caused an outage during the sale. The team raised it and passed the second run, which is exactly what a proof of concept is for.

</details>

Reading Progress

0%


Vote for new content

On This Page

The Stages

Discovery and Qualification

Proposal

Proof of Concept

Decision

Migration in Waves

Launch and Review

Where Engagements Fail

Using This in an Interview

Key Takeaways

Practice Questions