Posts

Load data into a lakehouse

Fabric lakehouses are a central element for your analytics solution. You can follow the ETL (Extract, Transform, Load) process to ingest and transform data before loading to the lakehouse. You can ingest data in many common formats from various sources, including local files, databases, or APIs. You can also create Fabric  shortcuts  to data in external sources, such as Azure Data Lake Store Gen2 or OneLake. Use the Lakehouse explorer to browse files, folders, shortcuts, and tables and view their contents within the Fabric platform. Ingested data can be transformed and then loaded using either Apache Spark with notebooks or Dataflows Gen2. Use Data Factory pipelines to orchestrate your different ETL activities and land the prepared data into your lakehouse. You can use your lakehouse for many reasons, including: Analyze using SQL. Train machine learning models. Perform analytics on real-time data. Develop reports in Power BI. Secure a lakehouse Lakehouse access is managed eith...

Explore the Microsoft Fabric lakehouse

A  lakehouse  presents as a database and is built on top of a data lake using Delta format tables. Lakehouses combine the SQL-based analytical capabilities of a relational data warehouse and the flexibility and scalability of a data lake. Lakehouses store all data formats and can be used with various analytics tools and programming languages. As cloud-based solutions, lakehouses can scale automatically and provide high availability and disaster recovery. Some benefits of a lakehouse include: Lakehouses use Spark and SQL engines to process large-scale data and support machine learning or predictive modeling analytics. Lakehouse data is organized in a  schema-on-read format , which means you define the schema as needed rather than having a predefined schema. Lakehouses support ACID (Atomicity, Consistency, Isolation, Durability) transactions through Delta Lake formatted tables for data consistency and integrity. Lakehouses are a single location for data engineers, data scie...

Copilot plugins and role requirements

Your role controls what activities you have access to, such as configuring settings, assigning permissions, or performing tasks. Copilot doesn't go beyond the access you have. Additionally, individual Microsoft plugins may have their own role requirements for accessing the service and data it represents. As an example, an analyst that has been assigned a security operator role or a Copilot workspace contributor role is able to access the Copilot portal and create sessions, but to utilize the Microsoft Sentinel plugin would need an appropriate role like Microsoft Sentinel Reader to access incidents in the workspace. To access the devices, privileges, and policies available through the Microsoft Intune plugin, that same analyst would need another service-specific role like the Intune Endpoint Security Manager role. Generally speaking, Microsoft plugins in Copilot use the OBO (on behalf of) model – meaning that Copilot knows that a customer has licenses to specific products and is aut...

Role permissions

  To ensure that the users can access the features of Copilot, they need to have the appropriate role permissions. Role permissions are configured per workspace. Permissions can be assigned using Microsoft Entra ID roles or Security Copilot roles. As a best practice, provide the least privileged role applicable for each user. The Microsoft Entra ID roles are: Global administrator Security administrator Security operator Security reader Although these Microsoft Entra ID roles grant users varying levels of access to Copilot, the scope of these roles extends beyond Copilot. For this reason, Security Copilot introduces two roles that function like access groups but aren't Microsoft Entra ID roles. Instead, they only control access to the capabilities of the Security Copilot platform. The Microsoft Security Copilot roles are: Copilot owner Copilot contributor The Security Administrator and Global Administrator roles in Microsoft Entra automatically inherit Copilot owner access. Only use...

Set up the default environment

  To set up the default environment, you need to have, at least, a Security Administrator role. During the setup of Security Copilot, you're prompted to configure settings. These include: SCU capacity - Select the capacity of SCUs previously provisioned. Each workspace must have its own capacity. Data storage - When an organization onboards to Copilot, one of the available settings determines where your customer data will be stored. Configuration of the data storage location applies at a workspace level. Microsoft Security Copilot operates in the Microsoft Azure data centers in the European Union (EUDB), the United Kingdom, the United States, Australia and New Zealand, Japan, Canada, and South America. Decide where your prompts are evaluated - You can restrict the evaluation within your geo or allow evaluation anywhere in the world. Logging audit data in Microsoft Purview - As part of the initial setup and listed under Owner settings in the standalone experience, you can choose to ...

Describe how to enable Microsoft Security Copilot

To start using Microsoft Security Copilot, organizations need to take steps to onboard the service and users. These include: Provision Copilot capacity Set up the default environment Assign role permissions Provision capacity Security Copilot operates on a provisioned capacity and an overage model. Provisioned capacity is billed by the hour while the overage capacity is billed on usage. You can flexibly provision Security Compute Units (SCUs) to accommodate regular workloads and adjust them anytime without long-term commitments. An SCU is the unit of measure of computing power used to run Copilot in both the standalone and embedded experiences. To manage unexpected demand spikes, you can allocate an overage amount to ensure that additional SCUs are available when initially provisioned units are depleted during unexpected workload spikes. Overage units are billed on-demand and can be set as unlimited or a maximum amount. This approach enables predictable billing while providing the flex...

Other prompting tips

  Some things to remember when coming up with your own prompts: Be specific, clear, and concise as much as you can about what you want to achieve. You can always start simply with your first prompt, but as you get more familiar with Copilot, include more details following the elements of an effective prompt. Basic prompt: Pearl Sleet actor Better prompt: Can you give me information about Pearl Sleet activity, including a list of known indicators of compromise and tools, tactics, and procedures (TTPs)? Iterate. Subsequent prompts are typically needed to further clarify what you need or to try other versions of a prompt to get closer to what you're looking for. Like all LLM-based systems, Copilot can respond to the same prompt in slightly different ways. Provide necessary context to narrow down where Copilot looks for data. Basic prompt: Summarize incident 15134. Better prompt: Summarize incident 15134 in Microsoft Defender XDR into a paragraph that I can submit to my manager and cre...