Databricks Certified Data Engineer Professional Sample Questions:

1. Which statement describes the correct use of pyspark.sql.functions.broadcast?

A) It marks a column as having low enough cardinality to properly map distinct values to available partitions, allowing a broadcast join.
B) It marks a column as small enough to store in memory on all executors, allowing a broadcast join.
C) It caches a copy of the indicated table on attached storage volumes for all active clusters within a Databricks workspace.
D) It caches a copy of the indicated table on all nodes in the cluster for use in all future queries during the cluster lifetime.
E) It marks a DataFrame as small enough to store in memory on all executors, allowing a broadcast join.

2. The view updates represents an incremental batch of all newly ingested data to be inserted or updated in the customers table.
The following logic is used to process these records.

Which statement describes this implementation?

A) The customers table is implemented as a Type 3 table; old values are maintained as a new column alongside the current value.
B) The customers table is implemented as a Type 0 table; all writes are append only with no changes to existing values.
C) The customers table is implemented as a Type 2 table; old values are overwritten and new customers are appended.
D) The customers table is implemented as a Type 2 table; old values are maintained but marked as no longer current and new values are inserted.
E) The customers table is implemented as a Type 1 table; old values are overwritten by new values and no history is maintained.

3. An upstream system has been configured to pass the date for a given batch of data to the Databricks Jobs API as a parameter. The notebook to be scheduled will use this parameter to load data with the following code:
df = spark.read.format("parquet").load(f"/mnt/source/(date)")
Which code block should be used to create the date Python variable used in the above code block?

A) date = spark.conf.get("date")
B) import sys
date = sys.argv[1]
C) input_dict = input()
date= input_dict["date"]
D) date = dbutils.notebooks.getParam("date")
E) dbutils.widgets.text("date", "null")
date = dbutils.widgets.get("date")

4. A data engineer is designing a pipeline in Databricks that processes records from a Kafka stream where late-arriving data is common. Which approach should the data engineer use?

A) Use batch processing and overwrite the entire output table each time to ensure late data is incorporated correctly.
B) Implement a custom solution using Databricks Jobs to periodically reprocess all historical data.
C) Use a watermark to specify the allowed lateness to accommodate records that arrive after their expected window, ensuring correct aggregation and state management.
D) Use an Auto CDC pipeline with batch tables to simplify late data handling.

5. A data engineer has configured their Databricks Asset Bundle with multiple targets in databricks.yml and deployed it to the production workspace. Now, to validate the deployment, they need to invoke a job named my_project_job specifically within the prod target context.
Assuming the job is already deployed, they need to trigger its execution while ensuring the target- specific configuration is respected. Which command will trigger the job execution?

A) databricks execute my_project_job -e prod
B) databricks bundle run my_project_job -t prod
C) databricks run my_project_job -t prod
D) databricks job run my_project_job --env prod

Solutions: