Skip to main content

Forward and backward pass

 Understanding Forward and Backward Pass in Neural Networks


🔹 Forward Pass: The input data flows through the network, layer by layer, until it produces an output. This step involves applying weights, biases, and activation functions to transform the data.


🔹 Backward Pass: The model evaluates its prediction by comparing it to the actual result. It then calculates how much each weight contributed to the error and adjusts them accordingly to improve future predictions. This process is called backpropagation and is essential for learning.


🚀 Why is this important?

The forward pass makes predictions, while the backward pass fine-tunes the model by learning from mistakes. Together, they enable neural networks to improve over time!


#DeepLearning #AI #MachineLearning #Backpropagation #NeuralNetworks

Comments

Post a Comment

Popular posts from this blog

Python Road map

 
Post a Comment
Read more

Command on Run

 🔰 23 Important Commands in the RUN (Executer) List 🔹 The command dxdiag: Used to check all the specifications of your device.   🔹 The command cleanmgr: Opens the Disk Cleanup tool.   🔹 The command temp: Accesses temporary files, which we delete as they contribute to slowing down the computer.   🔹 The command regedit: Opens the Registry Editor.   🔹 The command calc: Opens the Calculator.   🔹 The command msconfig: A tool to access programs that run with Windows at startup and disable them to speed up the system.   🔹 The command scandisk: Used for disk checking.   🔹 The command cmd: Opens the Command Prompt for Windows.   🔹 The command defrag: Used to stop and defragment the hard drive.   🔹 The command taskman: Allows you to see what is open in the taskbar and manage it.   🔹 The command pbrush: Opens the Paint program in Windows.   🔹 The command debug: Used to ch...
Post a Comment
Read more

Is it the end of lora?

 Is this the end of LoRA as a fine-tuning approach? Singular Value fine-tuning is here! We have a new paper in town called the "Transformers Squared". It promises to adapt any LLM to any task without external intervention.  The core idea of the paper is to use Singular Value Decomposition (SVD) to factorize the weight matrices of transformers. During training, we learn different singular values for different tasks. More specifically, we learn to scale the singular values for different tasks. Tasks can be as diverse as math reasoning or coding. The possibilities are endless. During inference, we do a two-pass inference. In the first pass, the LLM decides which "scale" to use for which task. In the second pass, the LLM adapts itself to be a specialist in the task and responds to our prompt. How cool is that? Paper Title: Transformer2: Self-adaptive LLMs Paper: https://sakana.ai/transformer-squared Blog: https://arxiv.org/abs/2501.06252 Video Explanation: https://youtu...
Post a Comment
Read more