Multi-file Editor Flashcards
(16 cards)
… (existing content unchanged) …
What is the purpose of the multi-file editor in Lakeflow Declarative Pipelines?
The multi-file editor enables users to develop and debug ETL pipelines with multiple source and non-source code files organized in a structured folder layout.
What is the pipeline asset browser in the multi-file editor?
It’s a dedicated sidebar showing all files related to a pipeline, allowing creation, deletion, renaming, and organization into folders.
What types of files can you manage in a pipeline?
Pipelines support source code files (evaluated during execution) and non-source code files (like notebooks or utility scripts not executed).
What is the recommended folder structure for new pipelines?
A default structure includes ‘transformations’ for source code, ‘explorations’ for notebooks, and ‘utilities’ for importable Python modules.
How do you create a new ETL pipeline using the multi-file editor?
Choose from options like sample SQL/Python, single transformation, or existing assets. You specify catalog/schema and get a ready-to-edit folder structure.
What are the steps to enable the multi-file editor?
You can enable it via pipeline creation settings, from an existing notebook, or through user settings under Developer options.
How do you move a pipeline to a Git folder?
Use the asset browser’s move option on the root folder. This updates pipeline settings to track the Git location.
What does the DAG in the multi-file editor represent?
It visually shows dependencies between pipeline tables. You can click nodes to view schemas and preview data.
How can you run code in the multi-file editor?
Options include running the full pipeline, a single file, or individual tables using toolbar and inline controls.
What are execution insights in the multi-file editor?
They provide table-level and pipeline-level performance data, query history, and profiling information.
What does the issues panel provide?
It summarizes errors across files and allows direct navigation to problem areas, enhancing debugging.
What does the event log in the multi-file editor show?
It logs events from the last pipeline run. You can enable persistent logs via settings.
How do you manage pipeline dependencies in the multi-file editor?
Edit the Environment settings to add package dependencies, similar to a requirements.txt file.
Can you still use notebooks with the multi-file editor?
Yes, but some features like table-level run actions are not available. Files are the preferred source code format.
How can you add existing files to a new pipeline?
Include them via Settings > Configure paths or from the All files tab in the asset browser.
