History of Digital Authentication

The history of digital authentication dates back to several decades when the scenario of the computer and the overall information technology was completely different from today’s age.

1. 1960s → Birth of Passwords

   Passwords first came into picture during 1960s when a computer was having the size of a modern day refrigerator. The numbers of computers were also very less and it was only used by the government offices and some other sectors, unlike today where every commoner has access to it. Each system was shared by 10-15 users and thus every user had their own passwords. These passwords were often written on papers as there was no threat of stealing. Internet was a theoretical concept at that time far from the practical world, hence hackers were also not present.

   Thus, passwords worked simply because threat didn’t exist.

2. 1990 to 2005 → The Era of Internet

   This was the era when internet was fairly popularizing. People started using internet and it penetrated into every society. Simple and common passwords were used at that time like 12345, iloveyou, birthdays, names were used. This was the time when there were no patterns fixed for passwords. This was due to the overall low awareness which perhaps caused low risk perception.

   Thus, users choose convenience over security.

3. 2005 to 2015 → The Complexity Era

   It was the time where the footprint of the internet rose to a global level. Internet came into individual smart phones. Dynamic, responsive and interactive applications like Instagram, Facebook came into the picture. Thus the companies demanded complexities in the passwords. Applications demanded
   uppercase, lowercase, special characters, numbers and a fixed length (generally 8) for the passwords, for example Johndoe#1677@. These complexity rules improved the policy but not the human behavior. Users responded to these complex password policies by repeating patterns thus, not increasing entropy.

4. 2015 to 2020 → OTP & MFA Era

   It was the time when increased security became imperative as hackers/attackers were most active. Thousands of data breaches occurred between 2010 to 2020 with millions of records exposed annually.

   💡

   Data breaches impacted major companies like Yahoo (3 billion accounts, 2013/14), Equifax (143M, 2017), and Marriott/Starwood (500M+, 2018), involving hacking, ransomware, and phishing, highlighting trends in massive data theft during that decade.

   It was the time when companies pivoted to OTP (One Time Passwords) based authentication that is usually sent via verified email address or phone number and expires after a certain time. Though the security improved, human errors remained. Victims of social engineering shared their OTPs to scammers and attackers. This led to the creating MFA (multi-factor authentication) as a second layer of authentication. This includes either verifying OTPs through email, messages and push notifications or using a dedicated Authenticator application to synchronize OTPs with the user and the platform.

5. 2020 to Present → Biometric & Hardware

   It was the time when all the major companies shifted their core authentication principle from “Something you know” to “Something you are” or “Something you own”. This led to the formation of biometrics based authentication that included fingerprints, face IDs and retina scans etc.

   

Why are Passwords failing?

1. Human Behavior

   If we really want to understand why passwords are failing we have to understand this quote:

   The real problem is not whether machines think but that men don't.

   — B.F. Skinner

   The real problem was never with password systems or the authentication mechanism, it was always the users using it. Multiple cybersecurity reports consistently find that 90% to 95% of data breaches involve a human element. In the present digital era, where nearly every major company, government department and private sectors work online, users have 100+ online accounts. Password use is very common, for example, user having same password in Netflix, Instagram, Email and even Bank Application. Thus, if someday, Netflix’s data breaches everything else also gets compromised. Users cannot be blamed because creating and remembering unique passwords for every platform is not only unrealistic but also impractical.

   Thus, the reason behind failing of passwords is simply the fact that humans become victims of social engineering confirming the fact is Humans are the weakest link, not the system.

2. Password Reuse & Single Point of Failure

   As discussed in the previous point, users tend to use same passwords for multiple platforms, therefore, breach or compromise of one platform leads to the compromise of other platforms too. This entire scenario creates a single point of failure. To solve this issue, password managers were created, but they too come with some complexity. A single master password is required to authenticate the Password manager itself thus, keeping the single point of failure still prevalent, just shifting from one to another.

   Therefore, the centralization of password manager increases the risk of compromise.

3. Weak Password Choices

   Users have a tendency to choose convenient passwords which turns to be very weak and predictable. Some common patterns of weak passwords are: Birthdays, Phone numbers, some global general passwords like “admin123”, “password123” and the predictable variations of all these.

   Thus, password cracking becomes very easy as hackers already know these patterns and their variations.

4. AI-Based Password Cracking

   The advent of AI has impacted almost every sector in the IT industry. Hackers and attackers also use significant amount of AI for their job, like brute-forcing passwords. The strength of AI based password cracking tools is that it does not brute-force blindly. It usually does pattern based brute-force where it learns patterns, behaviors and habits and make targeted and calculative guesses according to it. Passwords of 8 to 10 characters that was considered safe length can be cracked in seconds as powerful GPUs removes the technical limits.

   Thus, with the current boom of AI and GPUs, password length alone no longer guarantees safety.

5. Phishing: The Biggest Killer

   Phishing, is and always will be the biggest killer in the game of authentication. Hackers have achieved enough expertise that fake websites look more real and load faster than the original ones. Users fall prey to this and voluntarily shares passwords. Even 20 to 25 words paraphrases fail if gets phished.

   Thus, the strongest password becomes useless if handed over.

6. Data Breaches

   The industry has faced several data breaches over the last few years. Big shot companies like Domino’s, Zomato, BigBasket, Air India.

   💡

   Domino’s (2021) saw 180 million order records leaked, Air India (2021) suffered a breach of 4.5 million passenger records via its service provider SITA, Zomato (2017) had 17 million user emails and hashed passwords stolen, and BigBasket (2020) experienced a leak of over 20 million user profiles including phone numbers and addresses.

   This data proves the fact that even large companies fail to protect the passwords of their users.

7. Quantum Computing Threat

   Quantum Computing is not merely a theory anymore, it is practically present and it poses an inevitable threat to cybersecurity. The threat may not be visible to us but it is real and urgent primarily due to the "Harvest Now, Decrypt Later" strategy where adversaries collect encrypted data today to decrypt it once quantum power matures.

   💡

   In 2025, the landscape of quantum computing has reached a practical turning point: Google’s 105-qubit Willow chip performed a benchmark calculation in just 5 minutes that would take a classical supercomputer 10 septillion years while IBM is currently operating its 1,121-qubit Condor processor while executing a roadmap to deliver the first large-scale, fault-tolerant system by 2029 and Microsoft, in collaboration with Atom Computing, has successfully demonstrated and offered commercial access to 28 logical qubits, achieving the highest number of entangled logical qubits on record.

   This data proves the fact that the current encryption that stands on the shoulder of RSA (Rivest-Shamir-Adleman) and ECC (Elliptic Curve Cryptography) will ultimately break and therefore, text based passwords will become a cake-walk to break by hackers.

   Hence, passwords will or maybe already become technologically obsolete, not just weak.

8. Industry Shift

   In 2025, industry giants like Google, Apple and Microsoft have officially designated passwords as "vulnerable transitional technology" transitioning to a "password-less by default" model to combat escalating cyber threats.

   💡

   The vulnerability of traditional security was exposed by a historic data dump known as the "Mother of All Breaches" that leaked over 16 billion unique credentials from major companies like Google, Apple, and Microsoft, highlighting why Microsoft now blocks roughly 7,000 password attacks every second and why the persistence of "123456" as the most popular password across 7.6 million leaked accounts makes passwords a legacy risk.

   Thus, it can be concluded that industry leaders already know passwords won’t survive.

Authentication without Passwords

1. OTP & MFA

   OTP (One time passwords) and MFAs (Multi factor authentication) are already discussed in the previous part of the article. OTPs are verified after verifying passwords hence acts as a second factor of authentication. OTPs can be shared to the user via verified phone numbers, emails or by using special type of applications called Authenticator applications.

   Pros :

   • It adds an extra layer of protection.

Limitations:

• Users become victim of Social Engineering which results in sharing their OTPs.

• SIM swapping is another common problem due to which users loose their OTPs.

• Email Compromise is another limitation to OTPs.

Thus, OTPs and MFA maybe better than passwords alone, but the security is entirely dependent on the channel through which the OTP is shared (like email, phone etc.), hence not future proof.

2. Passkeys

   Passkeys are a secure, password-less authentication method based on industry standards from the FIDO Alliance. This replace traditional passwords with cryptographic key pairs that are split between our device and the service we are accessing. It consist of two keys that are cryptographically split as:

   • User Device (Private Key): When you create a passkey, your device generates a unique private key that is stored securely on your hardware (such as a phone, laptop, or physical security key). This key never leaves your device and is never shared with anyone.

   • Server (Public Key): A corresponding public key is sent to the website or app and stored on their server. Unlike a password, this public key is not secret even if the server is hacked, an attacker cannot use the public key to log in to your account without your private key.

Pros:

• No typing

• No phishing

• Brute-Force not possible

• Fast & cross-platform

Limitations:

• Device loss may create a huge problem

• Sync issues between private and public keys

• Poor usability for non-tech users

• Limited adoption

Thus, this mechanism is strong but it is also heavily ecosystem dependent.

3. Biometrics

   This simply means authentication using Fingerprint, Face, Retina etc. It is an effective authentication mechanism fingerprints, facial structures, retina are unique for each individuals.

   Pros:

   • Unique per user

   • Easy and simple authentication

   • No memory required

Limitations/Risks:

• With the rise of Deepfakes, creating false biometrics is easy

• 3D-printed fingerprints

• Immutable that is cannot be changed if leaked/compromised (unlike passwords that can be changed easily).

Thus, biometrics based authentication is secured but irreversible if compromised.

4. Hardware Security Keys

   Hardware security keys are FIDO2-compliant physical devices that connect via USB or NFC to provide phishing-resistant authentication through a cryptographic challenge-response process. When a login is initiated, the server sends a challenge that the key signs with its internal private key often requiring a physical touch or PIN to confirm user presence before returning a verifiable signature to grant access.

   Pros:

   • Cannot be remotely hacked

   • No phishing

   • No brute-force

Cons:

• The device can be lost

• Expensive (₹1500–₹5000)

• Poor usability

• Backup complexity

Thus, hardware security keys are most secure today, but not mass-adoptable.

Future of Authentication

The future of authentication will be completely different from the present scenario. Today’s authentication mechanisms are based on a session-based approach, i.e., the user gets authenticated and gets a session for a certain amount of amount of time. In future, it would be risky to follow this session based approach and the following mechanisms may be seen:

1. Behavioral Biometrics

   Behavioral biometrics is a type of authentication principle that records various signals like your unique, consistent digital body language such as typing speed, mouse movements, gestures, speech patterns and facial expressions and creates an authentication system that continuously verifies your identity in the background.

   Thus, in this mechanism you become your own password.

2. Continuous Authentication

   Continuous authentication is a mechanism where authentication doesn’t stop after login. System constantly verifies the behavior of the user (as mentioned in the above point), location of the device and integrity of the device.

   Therefore, according to this principle, Trust becomes dynamic, not static.

3. Post-Quantum Cryptography

   As we have already discussed in this article that Quantum Computers are not mere a theory anymore. RSA and ECC will eventually and ultimately break. Therefore, new quantum-resistant algorithms like Kyber, Dilithum, Spincs etc. are required to be used in the encryption systems. Future authentication must be quantum-safe at its foundation.

4. Ambient Authentication

   It can be considered as invisible-login systems. These systems uses GPS, Wi-Fi, Motion sensors, Bluetooth and Device proximity to authenticate the user. For example, Payments verified via physical closeness to POS device. As a result, Authentication becomes frictionless and passive.

5. Localization of Identity

   Localization of Identity is an utmost requirement in the authentication systems. A core shift in the mechanism is essential where the identity stays with the User, Device and the Hardware and not stored on remote servers. This is essential for security reasons. If we consider the India specific insights alone we can observe that India has a massive digital population, which is probably worlds largest. India has systems like UPI, Aadhar and a smartphone-first user base. Therefore, India needs password-less systems more than anyone.

Conclusion

Passwords are not merely weak anymore they are fundamentally outdated. They were designed for a simpler digital era, not for a world driven by automation, AI-powered attacks, massive data breaches, and emerging quantum capabilities. Over time, increasing password complexity has only added friction for users without meaningfully improving security. Longer passwords, special characters, and frequent rotations fail to address the core issue: authentication systems that rely heavily on human memory and behavior are bound to break.

Password-less authentication offers a clear improvement by reducing the very factors that make passwords fail. It minimizes human error, significantly lowers the impact of phishing attacks, and eliminates large-scale password reuse across platforms. By moving away from static secrets, these systems reduce the blast radius of breaches and shift security away from guessable information.

The future of authentication lies in mechanisms that align better with modern threat models. Behavioral signals, hardware-backed identity, continuous verification, and cryptographic systems designed to withstand post-quantum attacks will define the next generation of digital trust. Authentication will no longer be a one-time checkpoint but an ongoing process that adapts to context, behavior, and risk in real time.

At its core, authentication is undergoing a fundamental transition — from what you know to who you are and what you own. In a post-AI, post-quantum world, reliably verifying identity without sacrificing privacy or usability may become the hardest challenge in cybersecurity. Passwords solved yesterday’s problems. Tomorrow’s world demands something far stronger.

UPI stands for Unified Payments Interface is a digital payment infrastructure which used by more than 66% of the Indian population. It's known to serve 491 million individuals as of July 2025 and accounts for 85% of all digital transactions in India. It is an epitome of technological advancement in the finance sector which allows us to transfer money from entity to another within a few seconds. NPCI (National Payments Corporation of India) is the umbrella organisation which is responsible for all the retail payment settlements in India. NPCI was established in the year 2008 under the initiative of the RBI (Reserve Bank of India). NPCI develops and maintains the UPI infrastructure.

How digital payments were done before UPI?

Digital payments were also relevant in the pre-UPI era in the form of NEFT (National Electronic Funds Transfer) , RTGS (Real-Time Gross Settlement) and IMPS (Immediate Payment Service). These services are different from each other as they vary in different parameters like payment speed, service availability, minimum and maximum payment limits. For instance, IMPS payments were done instantly, RTGS payments took minutes to complete while NEFT took hours to complete. The entire system was centrally controlled and monitored by the RBI.

For these type of payments we need the following details of both the sender and the receiver :

• Account Number

• Bank Name

• IFSC code

For example, a user (sender) who is having an account in ICICI Bank will use its payment portal to send money to the user (receiver) who is having an account in HDFC bank. In this process both parties will use their Account Number, Bank Name and IFCS code for identifying each other and helping the system to authenticate them.

How UPI entered the game and changed it completely :

After the launch of UPI on April 2016, the online money transfer scenario was changed completely.

NPCI acts as the central controlling entity of the entire UPI infrastructure. It can be regarded as the “brain” of the UPI’s infrastructure. It interacts with the secured private APIs whose access are limited only to the trusted banks (e.g. SBI, ICICI, HDFC etc). Those APIs are cannot be accessed by the public systems. This internal NPCI orchestrates the backbone of the digital payment, it defines the flows how the amount will be deducted from the sender’s account and added to the receiver’s account ensuring a transparent transaction system in real time.

Some entities which are involved in this entire process of UPI are :

1. CPSPs (Customer Payment Service Provider) : Products like Google Pay and PhonePe which are used by the end user (sender as well as receiver) to interact and utilise the UPI system. These PSPs do not have direct access to the NPCI’s internal system through NPCI’s APIs. PSPs have their interaction with the banking systems which in turn communicates with the internal system of the NPCI.

2. Banking systems : These are the individual technological entities handled by the banks independently. For example banks like SBI, ICICI, and Axis have their individual systems. These systems acts as a middleware between the CPSPs (like Google Pay and PhonePe) and the internal system of the NPCI. These banking systems are the only entity in the entire UPI pipeline which can access the highly secure private APIs of the NPCI’s internal system which handles the core backbone functions of the UPI pipeline.

3. VPA (Virtual Payment Address) : VPAs servers as the unique identifier of each UPI user. It can be considered as a sole substitute of the informations like Bank name, branch code, IFSC code etc which was used in the previous system to identify the sender and the receiver. The VPA is generally in the form username@bank_code for example johndoe@okasbi.

Because of this simpler function and usage of UPI it has been widely adapted and used even among the rural population of India.

Process in which the UPI pipeline works :

1. User request for payment through the CPSPs (like Google Pay, PhonePe etc) by choosing the VPA of the receiver. This VPA selection either can be done directly or by scanning receiver’s QR Code or entering receiver’s number (which ultimately translates to the VPA of the receiver).

2. The CPSPs communicates to the Bank system with the the sender’s payment request. The Bank System verifies whether there is sufficient balance in user’s account to transfer the requested amount and passes the request to the NPCI internal system.

3. NPCI’s internal system verifies with bank server about sender’s balance . It also verifies whether the VPA of the receiver is valid or not.

4. After successful verification of all the mentioned information, finally the NPCI requests for the user approval/authentication to proceed forward. This is the segment where the end user (sender) enters his UPI PIN into the PCSP.

5. After the completion of the authentication process the NPCI’s internal system does all the internal backend stuff i.e., deducting money from the sender’s account and adding money to the receiver’s account.

If any of the checks and validations in the above steps fails, the transaction automatically fails and rolls back. If every check and validation passes, the transaction also completes successfully.

I have tried to explain the process of sending money, similar pipeline follows in the receive payment function also.

Acknowledgement :

Thanks to Piyush Garg sir to explain the concept behind the pipeline of UPI, this article is possible only because of this efforts. I tried my best to replicate and document what I understood and grasped from this video.

When we think of a browser, lets say Chrome, Brave, Safari, we consider it just as a piece of software. This undermines the browser’s real capabilities and diminishes the amount of attention that it deserves. It won’t be wrong to call the browser as a near-OS as it has so much capabilities like controlling network, enabling interaction, displaying and storing data, managing its own timer etc. Thus , browser has lot more to discuss than we think, which we will discuss in this article.

The functional architecture of a browser can be divided in the following parts :

1. Data : Data is stored in the browser in the forms of local storage, cookies, session storage etc. Nowadays even the entire scripts (Javascript codes) can be stored an executed from the browser data. Memory that is responsible for the execution of programs in the browser is also a part of this section.

2. User Interface : It is the part of the browser which is directly exposed to the end-user for display of web pages and interaction with those pages.

3. Browser Engine : This is the most interesting and the critical part of the browser. Browser Engine is responsible for all its working. It can be further divided in the following :

   1. Rendering Engine : It is part responsible for rendering the HTML, CSS, develops the Document Object Model (DOM), paints the canvas.

   2. Javascript Engine : Javascript engine is the part which deals with the javascript execution, nowadays typically v8 engine is used. It works exactly same as your Node, Dino, Bun and other runtime environments.

   3. Network Engine : It deals with all the network related things of the web application like sending requests, receiving responses, understanding status code, dealing with network protocols like HTTPS, Web Sockets connections.

   4. Timer Engine : This is the part which deals with the timing of the web applications. The setTimeout, setInterval functions that we use in our javascript are not directly executed through the javascript engine, instead they are implemented using web apis from the browser’s timer engine. In other runtime environments like Node and Dino, similar concept is used to maintain the timing.

In this article, we will focus on the Rendering Engine of the browser as we are understanding in context of web pages.

What is HTML actually?

Most of us think HTML (Hypertext Markup language) is simply writing <h1> tags or <p> tags but it is not that simple. Ever wondered how a simple programming language (it is still a debate whether HTML is a programming language or not) which has no access to the memory, no access to the LEDs of our screens, can render the entire complex webpages containing complex animations, gradients, images, videos etc on our screen.

HTML which perhaps creates the skeleton of the web pages, is actually a parser that uses C++ under the hood to render all the webpages in our web browsers. We are writing HTML but it is not as it is going to browser in the HTML format, rather it gets converted into C++ and goes into browser in the C++ format. This can be simply understand by writing a simple code for getting all the <h2> tags in the console of any wikipedia page:

We can clearly see, we are getting a NodeList [] in return. But we never write any NodeList right, we write HTML, which gets converted to C++.

It would not be wrong to say that whatever tags we write in our .html files is actually not the reason for the rendering of web pages in our browsers, there is a lot more to understand about it , which we will discuss in this article .

How the Rendering Engine actually works?

The rendering engine, as discussed earlier is responsible for the painting of the web pages on our browser screens. It is Rendering Engine’s job to understand the HTML and generate the Document Object Model.

The two ultimate jobs of the Rendering Engine are - Displaying the content and Interaction with the content.

This process occurs in the following steps:

• We write our .html files, which is considered to be DOCUMENT that will be rendered by the browser.

• The initial step is loading the file into the browser. This file could be anywhere in the internet , in some server, in some database or even in the local machine. Irrespective of where this file is located, the browser always treats as a resource from “another machine” , even if it is present on the local machine. That is the reason why I have mentioned as a near-OS.

• The file which is loaded is in the form of raw bytes, i.e., 0s and 1s in which our computer stores and understands it.

• The next step is character encoding , i.e. converting this raw data into characters of languages like english, hindi or japanese. In order to do character encoding, standards like UTF-8 (which is backward compatible with ascii) are used .

• Next step is tokenisation, i.e. taking language specific tokens from the characters. In case of programming languages these tokens are generally keywords like if, else, for, while . In the case of HTML these tokens are generally keywords like h1, p, html, body, head, style etc.

• Now, these tokens are converted into OBJECTS in these formats as mentioned below :

    {
        tag : h1,
        title : something, 
        value : something
    },
    {
        tag : head,
        title : something, 
        value : something
    }
    // similarly for all tags like body, html, styles etc.
  

• Just these objects in the gibberish format will not work. In the next step, they are arranged with respected to their relations. Here relations can be in the format like “Siblings”, “Children” and so on. This process can be called as MODELLING the OBJECTS that are created from the DOCUMENTS, thus bringing up the term Document Object Model or DOM.

• Exact same steps are followed for the CSS files also from loading raw bytes , encoding to characters, tokenisation and arranging these tokens with respect to relations and ultimately the CSS object model or the CSSOM is created.

Now, lets take a pause and understand where we are. We have created the DOM and the CSSOM. Note that until this time, DOM and the CSSOM have not aware of each other. Here comes the concept of Render Tree comes.

When the DOM and the CSSOM is created, the browser engine starts its work with its mathematical capabilities. As the CSS deals a with a lot of screen sizes in pixels, percentages, rems and ems, mathematics to display this is also involved. Then the Browser starts rendering the elements of the DOM on the screen. This is process is called painting.

What is the role of Javascript engine in the process of rendering?

The Javascript engine plays an importantly role in the Painting process, as the <script> tag has the power to manipulate the entire DOM (as well as the CSSOM).

• While doing the rendering process, the Browser Engine works to create the DOM. Whenever the Browser Engine encounters a <script> tag, it haults its process of creating DOM and first executes the Javascript code. DOM creating is halted to execute the Javascript code because the Javascript may entirely change the structure of the DOM.

• Sometimes the javascript code is asynchronous written with the async keyword. In some frameworks like React.js and Next.js, this process is called hydration where the client-side JavaScript takes over the server-rendered HTML to make it interactive. Thus, it has the capability to modify the DOM.

Conclusion

Understanding the architecture and the working of Browser is utterly essential for a developer. It helps us to develop better applications. In this article, we have discussed the architecture of the browser, the different engines working internally in the browser, how the entire “painting” process of the webpage works and why the browser can be considered as a near-OS.

Watch Hitesh Choudhary sir’s video to understand this concept in a better

Aj ki class ke baad, aap ChatGPT ko kabhi uss nazariye se dekh hi nahi paoge ~ Piyush Garg Sir

This is what our mentor told us in the beginning of our first class of Gen AI cohort , and this is absolutely true. This lecture was my first face off with generative AI and it was really over my expectation. Behind writing a simple “Hi” to ChatGPT and getting response “Hey, Whats Up!” in return , there are so many underlying mechanisms that works. Here are some of the key takeaways that I have personally got to know that I am diving into sips.

Nowadays, AI is everywhere, from personal assistant to all our software application. So being an engineer it becomes very essential to know about the jargons which common in the domain of AI. So lets start with some sips of Chai :

Sip 1 :

• NLP (Natural Language Processing) → Its a way by which AI can understand human language and responds back in the same human language.

• Tokenisation → Its the process by which the AI breaks our input into chunks and further process it as tokens. Each AI model has its own way of tokenising inputs. For example, OpenAI (ChatGPT) uses a tokenising library called TikToken. Similarly other models like Gemini, Claude has its own algorithms of tokenisation.

import tiktoken

encoder = tiktoken.encoding_for_model('gpt-4o')

print("Vocab Size", encoder.n_vocab) 

text = "The cat sat on the mat"
tokens = encoder.encode(text)

print("Tokens", tokens) 

my_tokens = [976, 9059, 10139, 402, 290, 2450]
decoded = encoder.decode([976, 9059, 10139, 402, 290, 2450])
print("Decoded", decoded)

• Vector Embeddings → The tokens are then converted into Vector Embeddings. Vector embeddings are basically some sort of numbers that helps the model to understand the input. In real life , it can be considered as a 3D graph, where each point is plotted according to their semantic meaning and these points are related to each other based on their semantic meaning. For example, if we check the vector point of the word “tea” in the Vector Embedding Projector, we can see the near by points are “coffee”, “beverage”, “cola”, “drink” etc.

from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()

client = OpenAI()

text = "Eiffel Tower is in Paris and is a famous landmark, it is 324 meters tall"

response = client.embeddings.create(
    input=text,
    model="text-embedding-3-small"
)

print("Vector Embeddings", response.data[0].embedding)

Sip 2 :

• Attention & Multi head Attention → Its a mechanism thorough which the tokens communicate with each other. In other words, though attention mechanism, the model can focus on the important (or effective) part of the input. Now, multi head attention means, parallel running of these attention mechanisms that helps the model to focus main concept of the input which helps the model to understand the context of the input.

Sip 3 :

• FeedForward → It refers to the Neural network which is the brain of the transformer. It helps the model to understand complex patterns, behaviour and context of the input based on which it gives the output.

• Softmax → It is a mathematical function which creates probability on prediction of next word. Based on this, we can decide the creativity of the model.

• Temperature → It is the factor in an AI model which decides the creativity (i.e. randomness) in of the output. For instance, in case of high temperature, the model would respond with “the sky is blue” and in case of low temperatures the model would respond with the “The sky, a canvas of swirling nebulae, whispered ancient secrets”.

  ---

  These were some my personal take aways from the lecture. It was an eye opening session, and yes I won’t see ChatGPT as I used to see it before.

Proof of Work (PoW) is a decentralized consensus mechanism used in blockchain networks to verify and add new transactions to the ledger. It works by requiring miners to solve complex computational puzzles to add a block to the blockchain, earning cryptocurrency as a reward for their efforts. The purpose of PoW is to make it difficult and expensive to tamper with the blockchain, ensuring its integrity and security.

How PoW Works

Here's a simplified explanation of how PoW works:

1. A miner receives a new transaction.

2. The miner creates a block that includes the transaction and other information.

3. The miner hashes the block header to create a unique identifier.

4. The miner attempts to find a nonce (a random number) that, when combined with the block header hash, produces a result less than or equal to the current target difficulty.

5. If the miner finds a valid nonce, they broadcast the block to the network.

6. Other nodes on the network verify that the block is valid and that the transaction is not already included in the blockchain.

7. If the block is valid, it is added to the blockchain and the miner is rewarded with cryptocurrency.

Key Components

• Nodes: Devices that actively participate in the network and maintain a copy of the network's data, validating transactions, relaying messages, and maintaining the integrity of the network.

• Nonce: A number used only once, which miners guess to solve the puzzle and add the block to the blockchain.

• Difficulty Target: The range within which the miners' guesses must fall to solve the puzzle and add the block to the blockchain.

• Proof of Work: The process of solving complex computational puzzles to demonstrate the expenditure of computational effort and the right to add a block to the blockchain.

Advantages and Disadvantages

Advantages of PoW include:

• Security: PoW ensures the integrity and security of the blockchain by making it difficult for bad actors to overtake the network.

• Decentralization: The network is not reliant on a central authority, making it more decentralized and resilient to censorship or single points of failure.

Disadvantages of PoW include:

• Energy Consumption: PoW requires vast amounts of energy, especially as more miners join the network, leading to a growing carbon footprint.

• Inefficiency: The process of mining and solving puzzles can be resource-intensive and time-consuming.

In summary, PoW is a consensus mechanism used in blockchain networks to maintain the security and integrity of the blockchain. It involves miners solving complex computational puzzles to add blocks to the blockchain and earn cryptocurrency rewards. However, PoW has some drawbacks, such as high energy consumption and inefficiency.

Citations: [1] http://blockchain.com [2] http://blockchaindemo.com [3] https://blockworks.co/news/what-is-proof-of-work [4] https://en.wikipedia.org/wiki/Proof_of_work [5] https://www.investopedia.com/terms/p/proof-work.asp

• In this concept, we generally consider that aggregation pipelines consists of one or more stages that processes the documents.

• Each stage is used to perform a function on the input documents.

• The documents that are output from a stage are passed to the next stage.

• An aggregation pipeline can return results for groups of documents.

// how to write aggregation pipelines.
db.collection.aggregate([
    {}, // first pipeline
    {}  // second pipeline 
])

Some stages used in Aggregation Pipelines :

1. $match : In MongoDB, the $match operator is used within the aggregation framework to filter documents based on specified criteria. It is similar to the WHERE clause in SQL. Here's how you can use it:

    db.collection.aggregate([
      { $match: { field: value } }
    ])
   

   In this example:

   • db.collection.aggregate() is used to perform aggregation operations on the collection.

   • { $match: { field: value } } is the stage where you specify the filtering criteria. Replace field with the field you want to filter on and value with the value you want to match.

For instance, if you have a collection of documents representing books and you want to find books with a specific genre:

    db.books.aggregate([
      { $match: { genre: "Science Fiction" } }
    ])

This query will return all documents from the books collection where the genre field is equal to "Science Fiction".

2. $lookup : In MongoDB, the $lookup stage is used within the aggregation framework to perform a left outer join between documents from two collections. This allows you to combine data from multiple collections in a single query. Here's how you can use it:

    db.collection1.aggregate([
      {
        $lookup: {
          from: "collection2",
          localField: "field1",
          foreignField: "field2",
          as: "outputField"
        }
      }
    ])
   

   In this example:

   • db.collection1.aggregate() is used to perform aggregation operations on collection1.

   • $lookup is the stage where you specify the details of the join.

   • from specifies the name of the collection to join with (collection2).

   • localField specifies the field from the input documents (collection1) to join on (field1).

   • foreignField specifies the field from the documents of the "from" collection (collection2) to join on (field2).

   • as specifies the name of the output field that will contain the joined array.

For instance, if you have two collections, orders and products, and you want to retrieve all orders with details of the corresponding products:

    db.orders.aggregate([
      {
        $lookup: {
          from: "products",
          localField: "productId",
          foreignField: "_id",
          as: "productDetails"
        }
      }
    ])

In this example, orders and products are the collections, productId is the field in the orders collection that matches with the _id field in the products collection, and productDetails is the name of the output field that will contain the joined array with product details.

3. $addFields : In MongoDB, the $addFields stage is used within the aggregation framework to add new fields to documents in the pipeline. This stage is particularly useful when you want to include computed fields or transform existing fields. Here's how you can use it:

    db.collection.aggregate([
      {
        $addFields: {
          newField: expression
        }
      }
    ])
   

   In this example:

   • db.collection.aggregate() is used to perform aggregation operations on the collection.

   • $addFields is the stage where you specify the fields to be added.

   • newField is the name of the new field you want to add.

   • expression is the expression used to compute the value of the new field.

For instance, if you have a collection of documents representing employees and you want to add a new field totalSalary that combines salary and bonus:

    db.employees.aggregate([
      {
        $addFields: {
          totalSalary: { $sum: ["$salary", "$bonus"] }
        }
      }
    ])

In this example, $sum is an aggregation operator that calculates the sum of the provided array. $salary and $bonus are the existing fields in the documents, and totalSalary is the new field that will contain the sum of salary and bonus for each document.

You can use any valid expression to compute the value of the new field, including arithmetic operations, functions, or even concatenation of strings.

4. $project : In MongoDB, the $project stage is used within the aggregation framework to shape documents by including, excluding, or renaming fields. It allows you to reshape documents before passing them to the next stage in the aggregation pipeline. Here's how you can use it:

    db.collection.aggregate([
      {
        $project: {
          field1: 1,          // include field1
          field2: 1,          // include field2
          newField: "$field3", // include field3 and rename it as newField
          _id: 0             // exclude _id field
        }
      }
    ])
   

   In this example:

   • db.collection.aggregate() is used to perform aggregation operations on the collection.

   • $project is the stage where you specify the fields to be included, excluded, or renamed.

   • field1: 1 and field2: 1 include the fields field1 and field2 in the output document.

   • newField: "$field3" includes field3 in the output document but renames it as newField.

   • _id: 0 excludes the _id field from the output document.

For instance, if you have a collection of documents representing employees and you only want to include their name and age fields in the output:

    db.employees.aggregate([
      {
        $project: {
          name: 1,
          age: 1,
          _id: 0
        }
      }
    ])

This will output documents containing only the name and age fields, with the _id field excluded.

Additionally, you can use $project to create computed fields, apply expressions, or reshape documents according to your requirements.

Database is always in another continent , therefore always use await.

Content Resources :

• Courtesy : Hitesh Choudhary

• For detailed video explanation follow :

  %[https://youtu.be/SUZKhBvxW5c?si=rR4azTi6cJ4rNHn7]