Skip to main content

Ways of pandas making faster

 FireDucks makes Pandas 125x Faster (changing one line of code) 🧠


Pandas has some major limitations:


- Pandas only uses a single CPU core.

- It often creates memory-heavy DataFrames.

- Its eager (immediate) execution prevents global optimization of operation sequences.


FireDucks is a highly optimized, drop-in replacement for Pandas with the same API. 


There are three ways to use it:


1) Load the extension: 

↳ %𝐥𝐨𝐚𝐝_𝐞𝐱𝐭 𝗳𝗶𝗿𝗲𝗱𝘂𝗰𝗸𝘀.𝐩𝐚𝐧𝐝𝐚𝐬; 𝗶𝗺𝗽𝗼𝗿𝘁 𝗽𝗮𝗻𝗱𝗮𝘀 𝗮𝘀 𝗽𝗱


2) Import FireDucks instead of Pandas: 

↳ 𝐢𝐦𝐩𝐨𝐫𝐭 𝗳𝗶𝗿𝗲𝗱𝘂𝗰𝗸𝘀.𝐩𝐚𝐧𝐝𝐚𝐬 𝐚𝐬 𝐩𝐝


3) If you have a Python script, execute is as follows: 

↳ 𝗽𝘆𝘁𝗵𝗼𝗻3 -𝗺 𝗳𝗶𝗿𝗲𝗱𝘂𝗰𝗸𝘀.𝗽𝗮𝗻𝗱𝗮𝘀 𝗰𝗼𝗱𝗲.𝗽𝘆


Done! ✔️


A performance comparison of FireDucks vs. DuckDB, Polars, and Pandas is shown in the video below. Official benchmarks indicate:


↳ Modin: ~1.0x faster than Pandas

↳ Polars: ~57x faster than Pandas

↳ FireDucks: ~125x faster than Pandas


Credit- Ultan 


Follow Data Spoof  to learn Python programming, data Science and big data.


 #datascience #machinelearning #ai #Python #python3 #sql #deeplearning 

#computervision #computerscience #programming #bigdata #architecture #datavisualization #dataanalytics #dataanalysis #dataanalyst #machinelearningalgorithms #machinelearningengineer #fireducks #polars #pandas

Comments

Post a Comment

Popular posts from this blog

Python Road map

 
Post a Comment
Read more

Command on Run

 🔰 23 Important Commands in the RUN (Executer) List 🔹 The command dxdiag: Used to check all the specifications of your device.   🔹 The command cleanmgr: Opens the Disk Cleanup tool.   🔹 The command temp: Accesses temporary files, which we delete as they contribute to slowing down the computer.   🔹 The command regedit: Opens the Registry Editor.   🔹 The command calc: Opens the Calculator.   🔹 The command msconfig: A tool to access programs that run with Windows at startup and disable them to speed up the system.   🔹 The command scandisk: Used for disk checking.   🔹 The command cmd: Opens the Command Prompt for Windows.   🔹 The command defrag: Used to stop and defragment the hard drive.   🔹 The command taskman: Allows you to see what is open in the taskbar and manage it.   🔹 The command pbrush: Opens the Paint program in Windows.   🔹 The command debug: Used to ch...
Post a Comment
Read more

Is it the end of lora?

 Is this the end of LoRA as a fine-tuning approach? Singular Value fine-tuning is here! We have a new paper in town called the "Transformers Squared". It promises to adapt any LLM to any task without external intervention.  The core idea of the paper is to use Singular Value Decomposition (SVD) to factorize the weight matrices of transformers. During training, we learn different singular values for different tasks. More specifically, we learn to scale the singular values for different tasks. Tasks can be as diverse as math reasoning or coding. The possibilities are endless. During inference, we do a two-pass inference. In the first pass, the LLM decides which "scale" to use for which task. In the second pass, the LLM adapts itself to be a specialist in the task and responds to our prompt. How cool is that? Paper Title: Transformer2: Self-adaptive LLMs Paper: https://sakana.ai/transformer-squared Blog: https://arxiv.org/abs/2501.06252 Video Explanation: https://youtu...
Post a Comment
Read more