0% completed
Mapping the Customer's Data
On This Page
What a Data Map Is
The Five Sources
The Empty Injury Field
The Separate Policy System
The Queue History and the Known Correct Answer
Questions for Each Source
Freshness and Volume
Using This in an Interview
Key Takeaways
Practice Questions
This is the first record the FDE read from the insurer's claims API, trimmed to the fields that matter:
{ "claim_id": "C-48213", "policy_number": "P-207731", "accident_day": 412, "description": "Rear bumper dented in parking lot, mirror cracked", "anyone_injured": null, "attachments": ["photo-1.jpg", "photo-2.jpg", "photo-3.jpg"], "current_queue": "minor_damage" }
The photos are names of files, not images. The injury field is empty, which could mean "no" or "not answered". There is a policy number, but no policy start date, which the fraud rule needs.
What a Data Map Is
A data map is a short list of every source of data the system needs, with where each source is stored, who controls access, what format it uses, and how reliable it is. It is written in the first week, to find missing data, slow access, and poor quality while the plan can still change cheaply.
The map starts from the decision: what does an adjuster look at to choose a queue? The answer was the claim form, the photos, any scanned police report, and the policy start date. Testing needed one more source, the queue in which each past claim was finally handled.
The Five Sources
| Source | Where it is stored | Access | Format | Quality found in week 1 |
|---|---|---|---|---|
| Claim form | Claims system | Claims API, approved by security | Structured fields and free text | Injury field empty on about 1 claim in 10 |
| Photos | Claims system's document store | Second API call per file | Image files, about 4 per claim | Some photos taken at night, hard to read |
| Police report | Claims system's document store | Same as photos | Scanned image, no text | Present on about 1 claim in 5 |
| Policy start date | Policy system, a separate system | Read-only policy API, 4 days to approve | Structured fields | Reliable |
| Queue history | Claims system | Claims API | A list of queue changes per claim | About 8 percent of past claims rerouted |
Three rows changed the design.
The Empty Injury Field
In 300 claims read with the adjusters, the field asking whether anyone was injured was empty on about 30, or 1 claim in 10. The online form did not require an answer. An empty field is not the same as "no". The adjusters treated it as unknown and read the description, and several of those descriptions mentioned a hospital visit.
Plan change: an empty field is stored as unknown, never as false, and the extraction step reads the description for signs of injury. The FDE also asked the claims manager to have the field made required.
The Separate Policy System
The policy start date is stored in the policy system, which a different team owns. That team offered a read-only API, and approval took 4 days. The FDE requested access on the day the gap appeared, the second day of the engagement, and approval arrived on the sixth.
Plan change: the rules step calls the policy API once per claim. When the API does not answer, the claim goes to an adjuster rather than skipping the fraud rule.
The Queue History and the Known Correct Answer
A collection of past cases with known correct answers is called a test set. The claims system records every queue change, so the last queue is where the claim was finally handled. About 8 percent of past claims had been rerouted at least once. For those, the first queue was wrong and the last was right.
Plan change: the test set of 2,000 past claims uses the final queue as the correct answer. The rerouted 8 percent are kept as a separate group, because they show which claims are hard even for people.
Questions for Each Source
For every row of the map, the FDE asked the same six questions.
- Where is it stored? Name the system and the team that owns it.
- How is it reached? An API, a file export, or a query, and who approves access.
- How fresh is it? Within seconds, or only in a nightly export.
- What format is it in? Fields, free text, images, or scanned documents.
- How complete is it? Count missing and unclear values on a real sample.
- Who may see it? Medical details in injury claims have stricter rules.
Count quality problems on a real sample instead of asking whether the data is good. The claims manager believed the data was complete. The count from 300 real claims showed otherwise.
Freshness and Volume
The claims API returns a new claim within about a minute, fast enough for routing in under an hour. A nightly export would have made that target impossible.
Each day brings 1,200 claims with about 4 photos each, or about 4,800 photos. About 1 claim in 5 has a police report, so roughly 240 scanned reports a day need conversion to text. These numbers set the load on the extraction step and its cost.
Using This in an Interview
In a design problem, list the data sources before you draw the system, with one check for each. Interviewers look for candidates who expect missing fields, slow access, and data owned by other teams.
"Routing needs the form, the photos, any police report, and the policy start date, plus the queue history to test on past claims. For each source I would ask who owns it and how long access takes. I would count missing values on a few hundred real claims before trusting any field."
Expect two follow-up questions:
- "What if the data you need is in a system you cannot access?" Request access on the first day, send those claims to a person meanwhile, and report the delay to the sponsor.
- "How would you know the past labels are correct?" Use the final queue, and keep rerouted claims as a separate group.
Key Takeaways
- A data map lists each source with where it is stored, how it is reached, its format, and its quality, starting from what a person looks at to decide.
- An empty field is unknown, not false. Count quality problems on a real sample, because the customer's belief about the data is often wrong.
- Data owned by another team often needs separate approval, so request access as soon as the need appears.
The next lesson, Defining Success Before Building, turns the baseline and the queue history into agreed measures.
Practice Questions
1. Why did the FDE treat an empty injury field as unknown instead of "no"?
<details><summary>Show answer</summary>Because the form did not require an answer, so an empty field did not mean nobody was injured. Treating it as "no" would send some injury claims to the wrong queue, one of the costliest mistakes.
</details>2. Why was the final queue in the history used as the correct answer instead of the first queue?
<details><summary>Show answer</summary>Because about 8 percent of past claims were rerouted, so the first queue was sometimes wrong. Testing against it would count some correct decisions as errors.
</details>3. The policy API sometimes does not answer. What should the system do with a claim it cannot check?
<details><summary>Show answer</summary>Send the claim to an adjuster. Skipping the fraud rule could route a real fraud case automatically, while a person costs about 5 minutes.
</details>Reading Progress
0%
On This Page
What a Data Map Is
The Five Sources
The Empty Injury Field
The Separate Policy System
The Queue History and the Known Correct Answer
Questions for Each Source
Freshness and Volume
Using This in an Interview
Key Takeaways
Practice Questions