1.12 – Transparency and Design - Know Your Stack (Who Built It? How Does It Work?) Lesson

Lesson Overview

Transparency is the starting point for AI governance. If you cannot see and understand an AI system, you cannot manage its risks. Modern stacks combine multiple models, data sources, and integrations, which makes visibility essential. This lesson shows how to get clarity, document what matters, and keep it current as your systems evolve.

You will learn how to build a System Bill of Materials for AI. It captures the models in use with version numbers and providers, the prompts that guide behavior, the datasets used for training or fine tuning with their sources and known biases, and any tools, APIs, or MCPs your system connects to. Each connection adds potential failure points, so getting this list right is key.

You will also see how to evaluate vendors and components. Ask who created a model or dataset, who maintains it if it is open source, and how training data was collected and filtered. Expect testing documentation that covers robustness and fairness. If a vendor or internal team cannot provide these artifacts, that is a governance failure.

Finally, you will frame explainability as operational transparency. Leaders do not need to decode model math. They need boundaries, failure modes, and confidence signals. Practical steps include a one page transparency summary during procurement and a living architecture diagram. With visibility in hand, you can set contract controls, build code guardrails, and track usage and incident metrics on an executive dashboard.

Who This Is For

This lesson suits teams that build, buy, or operate AI systems and need clear accountability for how those systems work and perform.

• Product and engineering leaders who must own system behavior and reliability
• Compliance and risk managers who need evidence of testing and controls
• Procurement teams evaluating AI vendors and services
• Data and ML practitioners documenting models, prompts, and datasets
• IT and platform teams integrating tools, APIs, and external services
• Executives who approve AI investments and want clear confidence signals

Where This Fits in a Workflow

Use this lesson’s approach at three moments. First, before approving any AI tool or service, request a one page transparency summary that outlines models, data, known limits, and testing. This simple gate filters weak offerings quickly. Second, during design and build, maintain a living architecture diagram that maps models, data sources, integrations, and dependencies. This helps engineers place guardrails where they matter. Third, after deployment, track usage and incident metrics, and run a regular review cadence so visibility stays current when models or data change.

Example applications include a procurement checklist for new AI vendors and an engineering practice that updates the architecture map whenever a new integration is added. Both give you a clear picture of what is in play and what needs testing.

Technical & Workflow Benefits

Old way: teams adopt AI tools with unclear provenance, undocumented prompts and datasets, and unknown dependencies. Testing is ad hoc. Problems appear as surprises in production. Incident response is reactive and slow because owners and failure points are unclear.

This approach replaces guesswork with visibility. A System Bill of Materials and one page transparency summary set shared expectations for components, data sources, and known limits. Robustness and fairness evaluations expose weaknesses early. An updated architecture diagram shows where inputs need validation and where outputs need filtering. Contract clauses require vendors to notify you of material changes and commit to performance standards.

Two places you will feel the difference: vendor selection and ongoing operations. Vendors that cannot produce testing artifacts or limits self select out. In production, usage and incident metrics surface what is being used, what is breaking, and how fast issues get resolved. That shortens time to fix and supports smarter investments in guardrails.

Practice Exercise

Pick one AI system your organization already uses. Complete three steps.

1. Draft a one page transparency summary. Include: model names, versions, and providers; prompts that guide behavior; datasets used for training or fine tuning with sources and known biases; connected tools, APIs, or MCPs; known limitations; and the status of robustness and fairness testing.
2. Start a System Bill of Materials. Make a clear list of every model, data source, prompt, integration, and dependency. For each item, add owner, update cadence, and where documentation lives.
3. Sketch your current architecture diagram. Mark inputs, outputs, and integration points. Identify two likely failure modes and add one input validation and one output filtering guardrail. Select two dashboard metrics, one usage and one incident, to track weekly.

Reflection: Where are your blind spots today, and which vendor or internal component lacks testing documentation or clear limits?

Course Context Recap

This lesson sits inside the Transparency and Design portion of the course. It gives you a practical way to see your AI stack, assess vendors and components, and define operational limits. The practices shown here feed directly into the next steps of your risk program: contract controls with change notifications and performance guarantees, code guardrails that catch common failures, and dashboards with usage and incident metrics supported by a monthly and quarterly review cadence. Continue with the course to put these controls in place and turn transparency into steady, measurable risk reduction.