Skip to main content

2. Things data engineer should know

 

data enginner should know how to write and read python code to extract the data from the internal and external apps .

1. Need understand what is API call means .


if we search in the front end (GUI) the results will be fetched back  database through API call .

a. We need to know to write the python code to fetch the details through API calls .

its possible by using Request library in python.

2. Need to understand the use case of the company.

example : telecom company (news channel).

the company like news channels , weather report like shown in the diagram .

it can be acheive through  by exposing certain API by the company X to teh other company . another company gets the details by sending the API calls .

The structure of an API call is generally the same across industries. In telecom, APIs are commonly used to manage subscribers, retrieve usage, send SMS, provision services, or check network status.

General Structure of an API Call

HTTP Method + URL + Headers + Parameters/Body

1. HTTP Method

Defines what action you want to perform.

HTTP MethodPurposeTelecom ExampleIdempotent?
GETRetrieve dataGet subscriber details or account balance✅ Yes
POSTCreate a new resource or perform an actionActivate a new SIM or recharge an account❌ No
PUTUpdate or replace an existing resourceChange a customer's tariff plan✅ Yes
DELETERemove a resourceDeactivate a service or cancel a subscription✅ Yes
PATCHPartially update an existing resourceUpdate only the customer's email or roaming status❌ No (typically)

2. Endpoint (URL)

The endpoint specifies the resource.

Example:

https://api.telecom.com/v1/subscribers/9876543210

Here:

  • https:// → Protocol

  • api.telecom.com → Server

  • /v1 → API version

  • /subscribers → Resource

  • 9876543210 → Subscriber ID (or MSISDN)

3. Headers

Headers provide metadata about the request.

Example:

Authorization: Bearer eyJhbGciOi...
Content-Type: application/json
Accept: application/json

Common telecom headers include:

  • Authorization Token

  • API Key

  • Transaction ID

  • Correlation ID

  • Content Type

4. Parameters

Parameters pass additional information.

Query Parameters

Example:

GET /usage?month=June&year=2026

Full URL:

https://api.telecom.com/v1/usage?month=June&year=2026

Path Parameters

GET /subscribers/9876543210

Here:

9876543210

is the subscriber identifier.

5. Request Body

Used mainly with POST or PUT requests.

Example: Activate a SIM

{
  "customerId": "C12345",
  "msisdn": "9876543210",
  "plan": "Premium_5G",
  "simSerial": "8991101200003204512"
}

Telecom Example 1: Get Subscriber Details

Request

GET /v1/subscribers/9876543210

Headers

Authorization: Bearer <token>
Accept: application/json

Response

{
  "subscriberId": "9876543210",
  "status": "Active",
  "plan": "Unlimited 5G",
  "balance": 120.50
}

Telecom Example 2: Activate a SIM

Request

POST /v1/sim/activate

Headers

Authorization: Bearer <token>
Content-Type: application/json

Body

{
  "customerId": "C12345",
  "simSerial": "8991101200003204512",
  "plan": "Basic"
}

Response

{
  "status": "Success",
  "activationId": "ACT123456",
  "message": "SIM activated successfully."
}

Telecom Example 3: Recharge Account

Request

POST /v1/recharge

Body

{
  "msisdn": "9876543210",
  "amount": 50
}

Response

{
  "transactionId": "TX987654",
  "status": "SUCCESS",
  "newBalance": 170.50
}

Telecom Example 4: Check Data Usage

Request

GET /v1/usage/9876543210?type=data

Response

{
  "subscriber": "9876543210",
  "usedGB": 12.4,
  "remainingGB": 17.6,
  "billingCycle": "June-2026"
}

End-to-End API Call Flow in Telecom

Client (Mobile App / CRM)
          │
          │ HTTP Request
          ▼
+----------------------------+
| API Gateway                |
| Authentication             |
| Rate Limiting              |
+----------------------------+
          │
          ▼
+----------------------------+
| Telecom Service API        |
| Subscriber Service         |
| Billing Service            |
| Network Service            |
+----------------------------+
          │
          ▼
+----------------------------+
| Telecom Backend Systems    |
| HLR/HSS                    |
| CRM                        |
| Billing                    |
| Provisioning               |
| Usage Database             |
+----------------------------+
          │
          │ Response
          ▼
Client receives JSON response

Complete Example

HTTP Request

POST https://api.telecom.com/v1/recharge
Authorization: Bearer abc123xyz
Content-Type: application/json

{
    "msisdn":"9876543210",
    "amount":100
}

Processing

  1. The client sends a POST request to the recharge endpoint.

  2. The API Gateway validates the authentication token.

  3. The recharge service processes the request.

  4. The billing system updates the subscriber's balance.

  5. The service returns a response.

HTTP Response

HTTP/1.1 200 OK
Content-Type: application/json

{
    "transactionId":"TX1002456",
    "status":"SUCCESS",
    "newBalance":220.50
}

This example shows all the key components of an API call:

  • Method: POST

  • Endpoint: /v1/recharge

  • Headers: Authorization, Content-Type

  • Request Body: Recharge details (MSISDN and amount)

  • Response: HTTP status code and a JSON payload containing the transaction result and updated balance.

Hands on :

free api : # https://restcountries.com/v3.1/all

import requests

# https://restcountries.com/v3.1/rc_live_b4d27ddee40740e5aca33f36f24c2358 -- api url


import requests

response = requests.get(

  'https://api.restcountries.com/countries/v5?q=canada',

  headers={'Authorization': 'Bearer rc_live_b4d27ddee40740e5aca33f36f24c2358'}

)

data = response.json()

print (data)



Here output is in JSON foramt --> dictionary --> key , value pairs.

its in semi structured format , means we can perdict what is inside partically not fully 

What is Semi-Structured Data?

Semi-structured data does not follow a fixed table structure like a relational database. However, it does have some organization using keys, tags, or attributes, so we can partially predict its structure.

For example, consider addresses from different countries.

India

{
    "country": "India",
    "address": {
        "street_name": "MG Road",
        "city": "Bengaluru",
        "pincode": "560001"
    }
}

United Kingdom

{
    "country": "United Kingdom",
    "address": {
        "street_name": "Baker Street",
        "city": "London",
        "postcode": "NW1 6XE"
    }
}

Why is this semi-structured?

We can predict that:

  • There will be an address object.
  • It will probably contain a street_name and city.

But we cannot always predict every field:

  • India uses pincode.
  • The UK uses postcode.
  • Another country might use zip_code.

So the structure is similar but not identical.

Comparison

Structured DataSemi-Structured Data
Fixed schemaFlexible schema
Every record has the same columnsRecords can have different fields
Example: SQL tablesExample: JSON, XML
Easy to queryMore flexible but may require extra handling

A simple definition for interviews

Semi-structured data is data that has an organized format using keys or tags, but it does not follow a fixed schema. Different records may contain different fields, although they share a similar overall structure. JSON and XML are common examples of semi-structured data.


 

🔐 1. JWT (JSON Web Token)

A JWT is a self-contained token. It carries information inside it.

Structure:

It has 3 parts:

  • Header
  • Payload (data/claims)
  • Signature

Example payload might include:

  • user id
  • role (admin/user)
  • expiry time

How it works:

  • Server creates and signs the token
  • Client sends it with each request
  • Server does NOT need to store it
  • Server just verifies the signature

Pros:

  • Stateless (no database lookup needed)
  • Fast and scalable
  • Good for distributed systems (microservices)

Cons:

  • Cannot easily revoke before expiry
  • If stolen, usable until it expires
  • Token size is larger

🔑 2. “Normal” API Tokens (Opaque Tokens / Session Tokens)

These are random identifiers that don’t contain user data.

Example:

a8f3k2m9xq1...

How it works:

  • Server generates a random token
  • Stores it in a database (or cache like Redis)
  • On each request, server:
    • checks token exists
    • looks up user session

Pros:

  • Easy to revoke instantly (delete from DB)
  • More secure in some cases (no data inside token)
  • Good control over sessions

Cons:

  • Requires server-side storage
  • Slightly slower (DB lookup every request)
  • Harder to scale without shared storage

⚖️ Key Difference Summary

FeatureJWTNormal/Opaque Token
Data inside tokenYesNo
Server storage neededNoYes
RevocationHardEasy
SpeedFasterSlightly slower
Security modelStateless trustStateful control



Comments

Post a Comment

Popular posts from this blog

Entity Relationship (ER) Diagram Model with DBMS Example

Reference :   Entity Relationship (ER) Diagram Model with DBMS Example What is ER Diagram? ER Diagram  stands for Entity Relationship Diagram, also known as ERD is a diagram that displays the relationship of entity sets stored in a database. In other words, ER diagrams help to explain the logical structure of databases. ER diagrams are created based on three basic concepts: entities, attributes and relationships. ER Diagrams contain different symbols that use rectangles to represent entities, ovals to define attributes and diamond shapes to represent relationships. At first look, an ER diagram looks very similar to the flowchart. However, ER Diagram includes many specialized symbols, and its meanings make this model unique. The purpose of ER Diagram is to represent the entity framework infrastructure. Entity Relationship Diagram Example Table of Content: What is ER Diagram? What is ER Model? History of ER models Why use ER Diagrams? Facts about ER Diagram Model ER Diagram...
Post a Comment
Read more

SQL Joins and advanced joins and Subqueries

  Refernce :  Expert Guide to Advanced SQL Joins: What You Need to Know It's helpful to visualize how these different SQL joins work. Here's a breakdown in a table-like format, along with explanations: SQL Join Types Overview Join Type Description Key Characteristics Use Cases INNER JOIN Returns rows where there is a match in both tables. - Shows only matching records. - Excludes unmatched rows from both tables. - Retrieving related data that exists in both tables. - Finding records with corresponding entries in another table. LEFT OUTER JOIN (LEFT JOIN) Returns all rows from the left table, and the matched rows from the right table. - Includes all records from the left table. - Fills in NULL values for columns from the right table where there's no match. - Retrieving all records from one table and their related data from another, even if some records don't have matches. - Finding records in one table that don't have corresponding entries in another. RIGHT OUTER JO...
Post a Comment
Read more

GIT BASH

  Bash Shell: Git Bash uses the Bash (Bourne Again SHell) command-line interpreter. This means you can use many of the same commands you'd find in a Linux or macOS terminal. Git Integration: Git Bash is tightly integrated with Git, making it easy to execute Git commands Essential Commands: Navigation: pwd : Prints the current working directory. ls : Lists files and directories in the current directory. cd <directory> : Changes the current directory. cd .. : Moves to the parent directory. File Management: mkdir <directory> : Creates a new directory. touch <file> : Creates a new file. rm <file> : Removes a file. rmdir <directory> : Removes an empty directory. Git Commands: git init : Initializes a new Git repository. git clone <repository URL> : Clones an existing Git repository. git status : Displays the status of your working directory. git add <file> : Adds a file to the staging area. git commit -m "commit message" : Commits chan...
Post a Comment
Read more