Notebook Development Flashcards
(10 cards)
… (existing content unchanged) …
How can you develop and debug Lakeflow pipelines using notebooks?
You can connect a notebook to a pipeline to start and validate updates, view cluster status, access the dataflow graph and event log, and debug directly from the notebook environment.
What prerequisites are required to develop Lakeflow pipelines in notebooks?
You need an existing pipeline with a Python or SQL notebook as source code and must have the CAN_MANAGE privilege or be the pipeline owner.
What limitations exist when using notebooks with Lakeflow pipelines?
Notebook development is only supported in Databricks notebooks, not Workspace files. The web terminal is also unavailable when attached to a pipeline.
How do you connect a Databricks notebook to a Lakeflow pipeline?
Use the compute drop-down menu in the notebook to select a pipeline that lists the notebook as its source code.
How do you validate Lakeflow pipeline code from a notebook?
Click ‘Validate’ in the notebook, press Shift+Enter in any cell, or use the cell’s menu to initiate validation without processing data.
What does the event log tab in a Lakeflow-connected notebook show?
It displays a log of pipeline events, which can be used to track the status and behavior of pipeline updates.
What does the dataflow graph tab show in a Lakeflow notebook?
It visualizes the pipeline structure, allowing you to select nodes and view their schemas.
How can you access the Lakeflow UI from a notebook?
Use the menu in the top-right corner of the notebook to jump directly to the Lakeflow Declarative Pipelines UI.
How can you access Spark UI and driver logs from a Lakeflow notebook?
Use the View menu in the notebook to access driver logs and the Spark UI associated with the running pipeline.
