Skip to main content

Terraform (overview )

 If  we want to create anything in the cloud, we can just navigate and go to click, and then we can create its ok in the dev environment and prod environment, but what about UAT and PROD env that where we need to create any reasoures by using code 

A form automation tool for Iaac is Terraform


And then why do we need a service account in GCP instead of an individual account? because if we leave the team, the work which I left or incase of one person absence other person and able to do that.

The service account is designed in the Principle of Least Privilege, so it means its automatically restricts the user by their role.

The resources:

For example, if we want to create the GCP bucket just need to create one resource for it, likewise to any biquery , composer, etc.. we can access through terraform using resource, likewise in airflow DAGS .

resource "google_storage_bucket" "my_bucket" {

    name     = "my-unique-bucket-name"

    location = "US"

  }

Always refer Terraform official documentation to know the syntax for creating any resources for anything.

To install Terraform, just use CMD in Mac, use brew(package installer) like wise Choco in Windows 

if we want to connect from our local machine to GCP using gcloud commands instead of logging into the cloud and giving these commands in google Cloud Shell. 

examples:

gcloud compute instances create my-instance --zone=us-central1-a --machine-type=e2-medium --image-family=debian-11 --image-project=debian-cloud

  • my-instance: Name of the instance.
  • --zone: Specifies the zone.
  • --machine-type: Sets the machine type.

  • --image-family and --image-project: Specify the OS image.
  • remaining refer terraform


    • Comments

    Post a Comment

    Popular posts from this blog

    Entity Relationship (ER) Diagram Model with DBMS Example

    Reference :   Entity Relationship (ER) Diagram Model with DBMS Example What is ER Diagram? ER Diagram  stands for Entity Relationship Diagram, also known as ERD is a diagram that displays the relationship of entity sets stored in a database. In other words, ER diagrams help to explain the logical structure of databases. ER diagrams are created based on three basic concepts: entities, attributes and relationships. ER Diagrams contain different symbols that use rectangles to represent entities, ovals to define attributes and diamond shapes to represent relationships. At first look, an ER diagram looks very similar to the flowchart. However, ER Diagram includes many specialized symbols, and its meanings make this model unique. The purpose of ER Diagram is to represent the entity framework infrastructure. Entity Relationship Diagram Example Table of Content: What is ER Diagram? What is ER Model? History of ER models Why use ER Diagrams? Facts about ER Diagram Model ER Diagram...
    Post a Comment
    Read more

    GCP Buckets

      CREATING BUCKET Method 1: Using GCP Console (UI) Go to the  Google Cloud Console . Navigate to  Cloud Storage . Click  Create Bucket . Enter a  unique name  for your bucket. Select a  storage class  (Standard, Nearline, Coldline, or Archive)... Choose a  location  (region/multi-region). Set  access control  (Uniform or Fine-grained). Click  Create . how do you optimise the class storage buckets - we can talk about cloud storage choosing wisely standard vs nearline vs coldline vs archive... how you managed your buckets - how did you manage? life cycle rule - regions multi regions etc
    Post a Comment
    Read more

    Pyspark

    pyspark link the above link for a sample code for Pyspark  Transformations create new datasets, actions return values, or write data. Transformations: Definition: Transformations are operations that create a new RDD or data frame from an existing one. They are "lazy," meaning they don't execute immediately. Instead, Spark builds a lineage graph (DAG - Directed Acyclic Graph) of transformations. Examples: map() : Applies a function to each element. filter() : Select elements based on a condition. flatMap() : Similar to a map, but flattens the results. groupByKey() : Groups elements by key. reduceByKey() : Reduces elements by key. join() : Joins two datasets. distinct() : Removes duplicate elements. union() : Combines two datasets. coalesce() : Reduces the number of partitions. repartition() : Increases or decreases the number of partitions. Actions: Definition: Actions trigger the execution of the lineage graph, which returns a result to the driver progra...
    Post a Comment
    Read more