← All roles
Interview practice

Data Engineer

5–8 years · Senior — SQL · Python · Spark · Airflow · Warehousing · ETL / ELT

0–2 years 2–5 years 5–8 years 8+ years
scenario Performance Tuning 1 of 6
A user reports that the dashboard refresh is unacceptably slow, taking over an hour to load. How do you approach this problem?
What a strong answer covers

Analyzes query performance metrics using the query plan; identifies slow-running queries and their bottlenecks; checks for unnecessary data being queried; evaluates the need for indexing or materialized views; consults with users to understand their requirements.

The follow-up you would get

If optimising the queries doesn’t resolve the issue, what alternative strategies do you consider?

Could you have answered that?
scenario Pipeline Ownership 2 of 6
You receive a complaint that a data pipeline you own has been delivering incorrect data for the last week. How do you assess and respond to this situation?
What a strong answer covers

Immediately reviews logs for error messages and data discrepancies; traces data flow to identify where the error originated; communicates with stakeholders about the issue and estimated resolution time; implements temporary data fixes if necessary; conducts a root cause analysis post-resolution.

The follow-up you would get

How do you ensure that data integrity is maintained during this investigation?

Could you have answered that?
scenario Scalability 3 of 6
You are facing an unexpected surge in data volume that your current ETL process cannot handle without significant delays. What steps do you take to address this?
What a strong answer covers

Evaluates the current architecture for bottlenecks; considers adding parallel processing or scaling up resources; reviews the data ingestion frequency and adjusts if necessary; assesses if data can be dropped or summarised; implements immediate fixes while planning for long-term scalability.

The follow-up you would get

Once the immediate issue is resolved, what long-term strategies do you implement to handle future surges?

Could you have answered that?
ownership A decision that improved performance 4 of 6
Can you discuss a design decision you made that resulted in improved performance for a data system?
What a strong answer covers

Describes a specific performance issue with relevant metrics before and after; details the design decision made, including technologies or architecture changes; mentions the testing and validation process; explains the impact on system performance and user experience; avoids vague references to 'optimisation'.

The follow-up you would get

What alternatives did you consider at the time?

Could you have answered that?
technical Data Pipeline Reliability 5 of 6
A nightly ETL job failed at 3am due to a data quality issue, and the alert system notified you at 8am. How do you investigate the failure and ensure it doesn’t happen again?
What a strong answer covers

Checks logs for error messages and timestamps; identifies the root cause of the data quality issue; adds validation checks in the pipeline; implements alerts for data quality thresholds; schedules a review of the data source.

The follow-up you would get

What specific validation checks would you add to prevent future failures?

Could you have answered that?
technical Data Transformation 6 of 6
You have a Spark job that processes terabytes of data but is running out of memory. What approaches do you take to troubleshoot and resolve the issue?
What a strong answer covers

Analyses Spark UI for job stages and task durations; checks the configuration of executor memory settings; optimizes data partitioning to reduce data shuffling; evaluates the use of broadcast variables for smaller datasets; considers using DataFrames over RDDs.

The follow-up you would get

How do you determine the optimal number of partitions for your Spark job?

Could you have answered that?
That is this set.

Load six more — you will get questions you have not seen first.

Give me six more →

Say your answer out loud before revealing. Reading the answer first feels like learning and is not.