Memory hook: Catalog describes; policy controls; lineage explains.
Must remember
An enterprise data platform needs ownership, discoverability, classification, lineage and quality rules, not just a storage bucket. Dataplex Universal Catalog helps organize metadata and governance across supported assets. Metadata access does not automatically grant data access. Assign accountable stewards and measure quality dimensions such as completeness, freshness, validity and uniqueness.
Separate development, test and production identities and datasets. Apply least privilege at organization/project/dataset/table scope as appropriate. Sensitive Data Protection discovers/classifies and can de-identify supported content; masking is not equivalent to irreversible anonymization. Keep residency constraints, retention, deletion and access evidence together.
BigQuery sharing, associated with the Analytics Hub name, distributes governed shared datasets without ordinary full-copy workflows. Publisher and subscriber permissions, refresh behavior and commercialization rules remain separate design decisions. An exported file may escape later centralized revocation, so sharing mechanisms affect control.
Prepare AI features with point-in-time correctness: a model must not learn information unavailable at prediction time. Fit preprocessing on training data and apply it consistently. Data leakage can produce impressive offline scores and poor production results. BigQuery ML enables supported model workflows using SQL; embeddings encode similarity for retrieval, not guaranteed factual truth.
For retrieval-augmented generation, preserve document provenance and permissions through chunking, embedding, indexing and retrieval. A vector search result must still be authorized for the requesting user. Evaluate retrieval quality and final answer quality separately; stale indexes, missing context and prompt injection require explicit defenses.
Choose under exam pressure
| Requirement | Choice and reason |
|---|---|
| Discover and trace enterprise data | Catalog, owners, classification and lineage. |
| SQL-oriented supported ML | BigQuery ML. |
| Share governed analytics datasets | BigQuery sharing with publisher/subscriber access design. |
Traps
- A catalog entry is not a grant to read the underlying data.
- Removing names alone does not prove a dataset is anonymous.
Active recall
1. What does lineage explain?
Where data came from and which transformations produced it.
2. What is point-in-time correctness?
Features contain only information available at the prediction timestamp.
3. What is training leakage?
Evaluation/training uses information that would not legitimately be available at prediction time.
4. Why filter retrieval by identity?
To prevent an AI answer exposing documents the user cannot access.
5. Does metadata governance replace enforcement?
No. Permissions and technical controls must enforce policy.