USA
Canada
UK
Europe
Australia
Singapore
Japan
IndiaQ1. What is the lakehouse architecture?
Short answer: Combines data-lake storage with warehouse reliability via Delta Lake.
Strong answer: The lakehouse unifies data lakes and warehouses: cheap object storage plus a transactional layer (Delta Lake) that adds ACID, schema enforcement, time travel and performance — so engineering, BI and ML run on one copy of data. Databricks pioneered this pattern; it removes lake-vs-warehouse silos.
Likely follow-up: What does Delta Lake add over plain Parquet?
What the interviewer is checking: Platform concept.
Common mistake: Treating a lakehouse as just a data lake.
Q2. What does Delta Lake provide?
Short answer: ACID transactions, schema enforcement, time travel and performance on the lake.
Strong answer: Delta Lake adds a transaction log over Parquet giving ACID writes, concurrent read/write safety, schema enforcement/evolution, time travel (versioned data), and optimizations (compaction, Z-ordering, data skipping). It turns an unreliable lake into a reliable, queryable store.
Likely follow-up: How does the Delta transaction log enable ACID?
What the interviewer is checking: Delta fundamentals.
Common mistake: Assuming plain Parquet gives transactional guarantees.
Q3. How do you optimize a Spark job on Databricks?
Short answer: Fix skew and shuffles, tune partitions, cache wisely, and use Delta optimizations.
Strong answer: Profile the Spark UI for skew, shuffles and spills; mitigate skew (broadcast joins for small tables, salting), right-size partitions, prune columns/partitions, cache reused data, and use Delta features (OPTIMIZE/Z-ORDER, data skipping). Avoid collecting large data to the driver. Photon can accelerate SQL workloads.
Likely follow-up: When do you use a broadcast join?
What the interviewer is checking: Spark performance skill.
Common mistake: Blaming cluster size instead of fixing skew/shuffles.
Q4. What is Unity Catalog and why does it matter?
Short answer: Centralized governance for data and AI assets across workspaces.
Strong answer: Unity Catalog provides centralized governance — fine-grained access control, lineage, auditing and discovery — for tables, files, models and more across Databricks workspaces. It replaces fragmented per-workspace permissions with a consistent, governed catalog, which is essential for enterprise data/AI security and compliance.
Likely follow-up: Why is centralized lineage valuable?
What the interviewer is checking: Governance awareness.
Common mistake: Ignoring governance in a multi-team platform.
Q5. Explain medallion (bronze/silver/gold) architecture.
Short answer: Layered Delta tables: raw → cleaned/conformed → business-ready aggregates.
Strong answer: The medallion pattern organizes data in layers: bronze (raw ingested), silver (cleaned, conformed, joined), gold (business-level aggregates/marts). Each layer is a Delta table, improving quality and reusability progressively. It gives a clear, incremental, auditable pipeline structure.
Likely follow-up: What belongs in the gold layer?
What the interviewer is checking: Lakehouse design pattern.
Common mistake: Dumping everything into one layer with no structure.
Q6. How do batch and streaming work together on Databricks?
Short answer: Structured Streaming + Delta enables unified incremental batch and streaming.
Strong answer: Spark Structured Streaming treats streams as incremental tables and writes to Delta with exactly-once guarantees; the same Delta tables serve batch queries. Auto Loader handles incremental file ingestion. This unifies batch and streaming code paths, simplifying pipelines that need both.
Likely follow-up: What does Auto Loader do?
What the interviewer is checking: Streaming knowledge.
Common mistake: Maintaining separate, divergent batch and streaming codebases.