Microsoft Fabric

Includes Synapse and Power BI. Cards will be gradually added :-)

35 flashcards · Computer Science · by claudehsu

1. Compare below data architecture Data Mesh Data Fabric Data Hub What is the use case/business scenario for each architecture? Does Microsoft Fabric applies the Data Fabric architecture?

   Data Mesh: a decentralized, domain-driven data architecture that empowers domain teams to own and manage their data. Use case: diverse, distributed teams requiring domain-specific data management. Data Fabric: an integrated(in between centralized and decentralized), end-to-end data management architecture that connects and orchestrates data across diverse sources and systems. Use case: complex data environments needing integration across on-premises and cloud systems Data Hub: a centralized dat…

2. How to understand below terms? Provide some example Experience Item Workspace

   Experience: A collection of capabilities(note 'capabilities', so don't get confused with the word 'capacity') targeted to a specific functionality. Example: Fabric Experiences include Synapse Data Warehouse Experience, Synapse Data Engineering Experience and etc. Item: A set of capabilities within an experience. Example: Data Engineering Experience includes the lakehouse, notebook, and Spark job definition items; Power BI Experience has report item, dashboard item and etc. Workspace: A containe…

3. Recall some of the components of Fabric Synapse Data Engineering: provides a Spark platform with great authoring experiences. Data Activator: a no-code experience in Fabric that allows you to specify actions, such as email notifications and Power Automate Workflows, to launch when it detects specific patterns or conditions in the changing data. Data Factory: provides a modern data integration experience to ingest, prepare and transform data from a rich set of data sources. It incorporates the s…

   Note: NOT all components of Fabric are listed here.

4. Is there any limitations on workspaces?

   Some limitation Workspaces can contain a maximum of 1,000 Fabric and Power BI items. A user can be a member of up to 1,000 workspaces. Certain special characters aren't supported in workspace names when using an XMLA endpoint. As a workaround, use URL encoding of special characters, for example, for a forward slash /, use `%2F`.

5. Below items are Fabric data items that can be shown in OneLake data hub. lakehouses warehouses semantic models

   Note: use plural form

6. What's the service architecture does Fabric and its predecessor Azure Synapse Analytics respectively? Fabric: Serverless Azure Synapse Analytics: PaaS

7. Suppose we're gonna build a report/dashboard for analytics on Bing News. Explain the project architecture design according to below diagram Especially describe how the data is transferred and stored

   From start to end We'll get data from Bing API using Data Factory, and the raw JSON data will be stored as JSON file in Lake DB. Then transform the raw JSON data to structured data and store it in a Table in Lake DB using Spark via Synapse Data Engineering. Then apply ML to do sentiment analysis using Synapse Data Science and the result will also be stored in Lake DB. Then build the report/visualization with the sentiment analysis result data using Power BI. Finally set up the alert using Data…

8. Microsoft Fabric is an end-to-end data and analytics solution that brings together new and existing components from below products into a single integrated environment Power BI Azure Synapse Azure Data Factory

9. From coding perspective, which one is more robust to build the BI solution, PBI Desktop or PBI Service?

   Power BI Desktop and Power BI Service have become more similar over time, and now share many of the same tools, features, and functionality for connection, creating, building and sharing with data. From coding perspective, it's quite common to build the BI solution using PBI Desktop and then publish to PBI Service.

10. To enable Fabric, we have two options Power BI Premium P SKUs Fabric Capacity F SKUs Which one is the recommended way to enable Fabric?

    The recommended way to enable Fabric is F SKUs option. Power BI Premium will be retired soon and users can no longer purchase P SKUs after July 1st, 2024.

11. Is F SKUs option a Pay-As-You-Go option or subscription mode option?

    In fact, F SKUs option supports both Fabric Reservation: licensing for your org to access Fabric in a unified product, plus Microsoft Azure Consumption Commitment(MACC) eligible. It's the cost-saving way compared with PAGO. Fabric Pay-As-You-Go: licensing based on dedicated capacity and we can pause the capacity when not it's not used, so that we'll not be charged.

12. Regarding experiences/features, what's the difference between two groups of F SKUs? F32 and below F64 and above

    There is difference on Power BI Experience F32 and below To even consume PBI contents, a Pro or PPU per user license is needed. F64 and above A Free license is sufficient to consume PBI contents. A Pro or PPU license is needed for editing/creating PBI items.

13. What is exactly a 'Task Flow'?

    The concept 'Task Flow' is more of a productivity tool instead of being a technical tool. To put it simply, there may exist many Fabric items like Dataflow Gen2, Data pipeline, Datamart, Lakehouse, Dashboard, Scorecard, Report, KQL queryset and etc. under a workspace, and task flows provides us visualization on which items are actually connected/related so we have a big picture of the data processes in the workspace.

14. There are two types of workspaces What's exactly workspace? What's the difference between those two types of workspaces?

    Workspaces are containers for Fabric items, such as dataflows, semantic models, paginated reports, reports, dashboards and etc., where you can collaborate and share content with colleagues. We have two types of workspaces My Workspace mainly for personal use content can be shared with individual Pro/PPU users (shared with me) Workspace mainly working as shared workspace for many users content can be created and shared across the organization

15. A Data Gateway acts a bridge connecting on-prem data source to Power BI Service without needing to move data to the cloud.

16. Currently, is Data Gateway supported on Windows and Linux?

    No, by the date Oct, 1st, 2024, only Windows is supported

17. If one Excel file is not supported with scheduled refresh, most likely what's the reason?

    To use scheduled refresh with Excel, the Excel file must contain a data model, so it's most likely the Excel file doesn't contain one data model.

18. What are the two types of Data Gateway? Which one is recommended to use?

    Two types of Data Gateway Personal Mode: compatible with other cloud services like Power Apps, Power Automate, Azure Logic Apps, Azure Analysis Service Multi-user support Import data & set up scheduled refresh Direct Query support Live connection to Analysis Services Standard Mode: mainly for personal use Runs as an app for non admin Runs as a single user Import data & set up scheduled refresh The pro tip is to use the Standard Mode.

19. What is Dataflow? What are some reasons that we'd like to use Dataflows?

    Dataflow is essentially cloud-based data pipeline, like Power Query in the cloud, which is used to extract, transform, store and integrate data from a wide variety of sources. Some reasons to use Dataflows Reuse queries across semantic models and other dataflows Run ETL processes on specific schedules(scheduled refresh) Create a centralized data warehouse to serve as a single source of truth Create, manage, and share reusable transformation logic

20. Describe the typical process/flow of using Dataflows

    Create the Dataflow in PBI Service: connect to data source, apply transformations with Power Query, and save as a table Access the Dataflow in PBI Desktop: from the Get data menu, select Dataflows as the data source, navigate to the workspace, and select the table to load Publish as a Semantic Model: once done with PBI Desktop, publish to Service and it will appear as a new Semantic Model

21. Fabric is deployed to Entra tenant, which is tied to a specific Domain Name System(DNS)

    This one is used to called Azure ID.

22. What is the relationship among tenant, capacity and workspace?

    A tenant can have multiple capacities associated with it. A capacity is associated with a tenant. A workspace is assigned within a capacity, and a capacity can be split into multiple workspaces.

23. How is capacity usage measured?

    Capacity usage is measured as CU(Capacity Units) per second, and some meters aggregate to CU per minute, per hour or higher. Non-duration meters will translate their compute to CU/sec. For example, Copilot meters based on the number of tokens sent back and forth, and there is corresponding translation. 1 CU has 2,628,000 CU/sec each month. The logic behind the number is 365days / 12months * 24hours * 3600secs

24. Explain Burstable Capacity and Capacity Smoothing.

    Burstable Capacity: Capacity within Fabric is not static, and you may temporarily get more resources beyond the baseline of your capacity. This behavior is managed by the system, instead of by you. Under the hood, burst feature utilizes a shared pool of CUs across tenants that are in the same region with the same service tier. Capacity Smoothing: When we are in 'spike' time, burst feature kicks to provide resources more than baseline to ensure performance. After that 'spike' time, we need to pa…

25. Explain Capacity Carryover. Especially explain the difference between Capacity Smoothing and Capacity Carryover

    Capacity Carryover There'll be the cases that even with Capacity Smoothing, the desired capacity is still over baseline, such as we continuously add interactive jobs in a short period of time. Capacity Carryover kicks in to allow further 'spread'(even beyond 5-min or 24-hr period) when there is available capacity Difference between Capacity Smoothing and Capacity Carryover Capacity Smoothing: we can utilize near future compute power to get consistent performance and then Capacity Smoothing pays…

26. What if the workload is so heavy that 'overload' capacities continue to be carried forward and forward due to Capacity Carryover?

    In that case, usually throttling happens and it's and indication that we over-utilize the deployed capacity.

27. Explain different throttle triggers and stages.

    Three throttle stages: Initial: Trigger: used up all available CU for next 10 minutes Behavior: a delay of 20 seconds for all new interactive operations Extended Trigger: carry forward reaches a full hour Behavior: new interactive requests are outright rejected, though background schedule operations can still run Maximum Trigger: accumulate a full 24 hours of carry forward Behavior: entire capacity is frozen until the deficit is cleared So think it as different severity of carry over.

28. Regarding throttling, there are cases that are not suitable to gradually reject requests. List those special cases.

    Event Stream can run for extended periods once event streams are started Fabric reduces CU allocation instead of halting operations Ensures streams remain open, albeit with potentially reduced capacity Real-Time Analytics Exempt from the 20-sec delay at initial throttling to keep 'real-time' Permitted to continue without delay until reach rejection phase(extended throttle stage)

29. We can pause and resume capacities. Is it a good idea to do that?

    No, usually it's NOT a good idea, so we need to weigh operational needs and performance impacts before pausing. Pausing rejects new requests and cancels active processes(kind of rollback) Resumption involves a reinitialization phase, like warming caches, so we may encounter temporary slowdowns

30. Typically, how do we use Dataflow and Data Gateway together?

    Dataflow transforms data from source to desired target form. Data Gateway provides the bridge between local computer to Fabric cloud storage, so it transfers data. One good example is data in a CSV file gets transferred from local to cloud and in the same time transformed to the desired format. When setting up a Dataflow, we need to specify the Data Gateway if we'd like to use a local file via Data Gateway instead of using the one on OneDrive

31. Once we set up a Dataflow, what's the recommended action that we'd better perform immediately?

    Refresh operation

32. Where can we review the refresh status of Workspace items?

    On the list of Workspace items, we can manually refresh items like dataflows, and semantic models.

33. Where is the entrance of Dataflow Gen2? Why does Dataflow Gen1 still exist given the fact we have Dataflow Gen2?

    Dataflow Gen2 can be found in Data Factory Dataflow Gen2 is the newer generation and some new features are introduced by Dataflow Gen2, but in the meantime some features are now not available in Dataflow Gen2, such as Direct Query Incremental refresh AI insight support

34. Scheduled refresh can be set on Dataflow Semantic Mode Is it fine to set 8:00AM as refresh time for both Dataflow and the corresponding Semantic Mode?

    No, that's not a good idea. If both refreshing time are set as 8:00AM, the Semantic Model will not include the latest data since Dataflow is still running to retrieve the latest data. As a result, usually we stagger the scheduled time for refreshing Dataflow and refreshing Semantic Model

35. Below diagram is part of a sample lineage of one workspace Analyze the similarity between it and Power Pivot or Power BI Desktop solution.

    They all have three common parts ETL: mainly implemented by Power Query Data Model: shows the relationship among tables Report: provides info based on different types of visuals