Skip to main content

 Creating the pipleine in azure 


inorder to login to azure portal we need microsoft account 

--> using that crEATE AZURE ACCOUN

AFTER NAVIGATE INTO THE AZURE PORTAL 

GO AND CREATE THE RESOURSE GROUP UNDER THAT CREATE THE RESOURES NED TO CREATE THE PIPELINE 

1. AZURE BLOG STORAGE ( ENABLE HIRTENIAL NAMESPACE TO CHANGE INTO GEN2)

INSIDE CREATE THE CONTAINERS (BLOG CONTAINER ) INDIDE THAT SUBFLODER CAN ABE CREATED USING ADD DICTIONATIES 

CREATED A STORAGE ACCOUNT sdmm 

UNDER THAT SDMM . GOLD , SILVER , BRONZE INSIDE THAT WE CREATE SD , MM FOLDERS IN THREE EXPECT GOLD WE CREATE PROCEMENT, SALES 


2. DATA FACTORY 

INSIDE DATA FACTORY --> NAVIGATE TO MANAGED INDENTIES 

UNDER THAT ENABLE SYSTEM ASSIGNED THEN SAVE

THEN NAVIGATE BACK TO STORAGE AND NAVIAGE TO IAM ACCESS CONTROL IN THAT ADD ROLES AS STORAGE BLOG CONTRIBUTER AND THEN ASSIGNED MEMBER AS DATA FACTORY .





THEN LAUNCH STUDIO IN DATA FACTORY THEN GO TO MANAGE THEN LINKED SERVICE TO LINK ADLS AND DATA FACTORY 





THEN CLICK TEST CONNECTION IF THE CONNECTION IS SUCCESSFUL CLICK CREATE 

CREATE SUCCESFULLY .

Comments

Post a Comment

Popular posts from this blog

session 19 Git Repository

  🔁 Steps to Create a Branch in Databricks, Pull from Git, and Merge into a Collaborative Branch Create a New Branch in Databricks: Go to the Repos tab in your workspace. Navigate to the Git-linked repo. Click the Git icon (or three dots ⋮) and choose "Create Branch." Give your branch a name (e.g., feature-xyz ) and confirm. Pull the Latest Changes from Git: With your new branch selected, click the Git icon again. Select “Pull” to bring the latest updates from the remote repository into your local Databricks environment. Make Changes & Commit: Edit notebooks or files as needed in your branch. Use the "Commit & Push" option to push changes to the remote repo. Merge into the Collaborative Branch: Switch to the collaborative branch (e.g., dev or main ) in Git or from the Databricks UI. Click "Pull & Merge" . Choose the branch you want to merge into the collaborative branch. Review the c...
Post a Comment
Read more

Session 18 monitering and logging - Azure Monitor , Log analytics , and job notification

 After developing the code, we deploy it into the production environment. To monitor and logging the jobs run in the real time systems in azure  we have scheduled the jobs under the workflow , we haven't created any monitoring or any matrics . After a few times, the job failed, but we don't know because we haven't set up any monitoring, and every time we can't navigate to workspace-> workflows, under runs to see to check whether the job has been successfully running or not and in real time there will be nearly 100 jobs or more jobs to run  In real time, the production support team will monitor the process. Under the workflow, there is an option called Job notification. After setting the job notification, we can set a notification to email . if we click the date and time its takes us to the notebook which is scheduled there we can able to see the error where it happens . order to see more details, we need to under Spark tab, where we have the option to view logs ( tha...
Post a Comment
Read more

Entity Relationship (ER) Diagram Model with DBMS Example

Reference :   Entity Relationship (ER) Diagram Model with DBMS Example What is ER Diagram? ER Diagram  stands for Entity Relationship Diagram, also known as ERD is a diagram that displays the relationship of entity sets stored in a database. In other words, ER diagrams help to explain the logical structure of databases. ER diagrams are created based on three basic concepts: entities, attributes and relationships. ER Diagrams contain different symbols that use rectangles to represent entities, ovals to define attributes and diamond shapes to represent relationships. At first look, an ER diagram looks very similar to the flowchart. However, ER Diagram includes many specialized symbols, and its meanings make this model unique. The purpose of ER Diagram is to represent the entity framework infrastructure. Entity Relationship Diagram Example Table of Content: What is ER Diagram? What is ER Model? History of ER models Why use ER Diagrams? Facts about ER Diagram Model ER Diagram...
Post a Comment
Read more