- Видео 111
- Просмотров 159 015
The Data Corner
Пакистан
Добавлен 4 июл 2021
Helping nerds automate mundane tasks and learn new skills along the way
Simplify PDF Management: Extract First Page with pyPDF2 in Python
In today's video, we are going to learn to manipulate pdf files using pyPDF2 library of python. We are going to focus on how to extract first page from a pdf file and save it to a new file.
Key concepts:
* With context manager (for proper resource cleanup)
* For loop
* path.join from os library
* Converting single use code to a reusable function
If you like the video, please like, share, and subscribe. If you have any comments or video suggestions, make sure to post in comment section.
You can support me by buying a coffee here. ☕︎☕️
www.buymeacoffee.com/thedatacorner
Key concepts:
* With context manager (for proper resource cleanup)
* For loop
* path.join from os library
* Converting single use code to a reusable function
If you like the video, please like, share, and subscribe. If you have any comments or video suggestions, make sure to post in comment section.
You can support me by buying a coffee here. ☕︎☕️
www.buymeacoffee.com/thedatacorner
Просмотров: 44
Видео
VBA - Expand autofilter range in one click macro!
Просмотров 99Месяц назад
In today's video, we will write a VBA that will expand the existing auto filter to include some extra columns at the end, for easier analysis purpose.
Merge multiple dataframes (2 or more) using python pandas and reduce function
Просмотров 75Месяц назад
In today's video, we will look at how we can merge multiple (2 ) dataframes in python pandas using reduce function from functools library. Especially helpful when you have a lot of different tables with information that can be joined on a single column.
Where's Your Money spent? Track and Categorize Expenses with Python!
Просмотров 128Месяц назад
In today's video, we are going to go through our expenses and try to organise them into categories, for further anaysis and visualization using python. We will be using assign function of python pandas. Do you want to convert PDF data of your credit card etc for further analysis in excel or python? follow below videos: ruclips.net/video/ACm9OJWH8WE/видео.html ruclips.net/video/atz8Y6YtAkk/видео...
Fix Excel Dates with This Simple Trick using DATE function
Просмотров 2022 месяца назад
In today's video, we will do a deep dive into DATE function of excel and fix any given date using formulas only.
Convert Email Data to Excel Table in MINUTES with This Hack!
Просмотров 3163 месяца назад
In today's video, we will extract some email data from outlook, clean and convert it into Excel table format using only excel IF formula
Transform Excel Data into Outlook Emails with Python in One Click!
Просмотров 2373 месяца назад
In todays video, we will look at how we can use python to send automated emails through outlook. We will be using pandas and pretty html library to achieve this task. If you like the video, please like, share, and subscribe. If you have any comments or video suggestions, make sure to post in comment section. You can support me by buying a coffee here. www.buymeacoffee.com/thedatacorner For auto...
Power Automate: Automatically Reply to the original email
Просмотров 1,6 тыс.4 месяца назад
In todays video, we will look at how we can reply to original email using power automate. This neat trick is helpful in situations where you want a customised auto reply to be sent when an email is received.
PDF invoices data extraction with pdfplumber in Python
Просмотров 3,2 тыс.4 месяца назад
In todays video, we are going to extract invoice data from PDF file using pdf plumber library of python. Once done, it can be converted to pandas dataframe and extracted to excel, for further analysis.
VBA- From Data Entry to Automated Emails: Build a VBA UserForm for Invoice Data Entry
Просмотров 4884 месяца назад
In this comprehensive VBA tutorial, I walk you through creating a custom UserForm for entering invoice data. Learn how to automate the process of validating invoice amounts against contracted rates and streamline the resolution of discrepancies. By the end of this video, you'll know how to: Design a UserForm for efficient data entry instead of manually entering data, avoids mistakes and easier ...
VBA - Concatenate selected/visible values by custom delimiter & copy to clipboard
Просмотров 465 месяцев назад
In today's video, we will write a VBA that will concatenate selected or visible/filtered values using a custom delimiter and then copy result to clipbard. Concepts used: For each loop Input box Left function Concatenation
VBA - Fix date formats using split and arrays
Просмотров 1186 месяцев назад
In today's video, we will look at how we can fix date formats in excel using VBA. A lot of times we run into situations where dates are not formatted or recognised by excel, today we will tackle those situations. Concepts used: For each loop arrays Split function Concatenation
Power Automate: Download email attachment to onedrive, create new folder, update Excel File
Просмотров 2,5 тыс.6 месяцев назад
In todays video, we will explore how we can download attachments from emails using power Automate and save them to onedrive. We will also learn how to sort and organise attachments to daily created folders, i.e one folder for one day, with all the attachments inside of it. Once we do that, we will create and continuosly update an excel File as well, as a running database of all emails received,...
VBA - Open multiple PDF files from visible/filtered rows in Microsoft Excel directly
Просмотров 1166 месяцев назад
In today's video, we will look at how we can open multiple PDF files directly from excel sheet's visible/filtered rows using any PDF program. Concepts used: For each loop Selection Property Shell If Else construct
Microsoft Power toys: Screenshot to Text / Batch Image Resize / Batch File Rename etc
Просмотров 2027 месяцев назад
Microsoft Power toys: Screenshot to Text / Batch Image Resize / Batch File Rename etc
OCR/Extract text from a screeshot and export to text file using Pytesseract & Python
Просмотров 1767 месяцев назад
OCR/Extract text from a screeshot and export to text file using Pytesseract & Python
Forget Excel, Try Python Xlwings -- Part 3: File Formatting Deep Dive
Просмотров 3177 месяцев назад
Forget Excel, Try Python Xlwings Part 3: File Formatting Deep Dive
VBA-Open Folder Location of an active Excel File
Просмотров 3237 месяцев назад
VBA-Open Folder Location of an active Excel File
Forget Excel, Try Python Pandas -- Part 2: Pivot Tables Basic + Advanced!
Просмотров 1,1 тыс.8 месяцев назад
Forget Excel, Try Python Pandas Part 2: Pivot Tables Basic Advanced!
Forget Excel, Try Python Pandas -- Part 1: Basic Data Manipulations!
Просмотров 4878 месяцев назад
Forget Excel, Try Python Pandas Part 1: Basic Data Manipulations!
Microsoft Power Query - Filter and remove specific rows dynamically - 2 Methods
Просмотров 4,6 тыс.8 месяцев назад
Microsoft Power Query - Filter and remove specific rows dynamically - 2 Methods
VBA: 5 methods for finding the last row
Просмотров 4038 месяцев назад
VBA: 5 methods for finding the last row
Python Pandas- Strip & Clean $ USD and more unwanted symbols from numeric column!
Просмотров 699 месяцев назад
Python Pandas- Strip & Clean $ USD and more unwanted symbols from numeric column!
Claude 3 Opus answers challenging questions on Python Pandas
Просмотров 799 месяцев назад
Claude 3 Opus answers challenging questions on Python Pandas
Python Project for Daily File Backups - Deep Dive - Part 1
Просмотров 2109 месяцев назад
Python Project for Daily File Backups - Deep Dive - Part 1
VBA-Select from activecell to last row of active column
Просмотров 8719 месяцев назад
VBA-Select from activecell to last row of active column
Microsoft Power Query: Remove empty columns dynamically!
Просмотров 3,3 тыс.9 месяцев назад
Microsoft Power Query: Remove empty columns dynamically!
Automate Daily Gas Price with Power Query, xlwings, and Windows Task Scheduler!
Просмотров 19310 месяцев назад
Automate Daily Gas Price with Power Query, xlwings, and Windows Task Scheduler!
VBA: Paste Formulas Instantly till last row and visible rows
Просмотров 2,8 тыс.10 месяцев назад
VBA: Paste Formulas Instantly till last row and visible rows
Microsoft Power Query Deep Dive: Split by new line character & Rename columns dynamically!
Просмотров 11410 месяцев назад
Microsoft Power Query Deep Dive: Split by new line character & Rename columns dynamically!
This is awesome, it removes a lot of manual editing I have to do when people feed me data with blank columns in random spots every month that I then have to remove later. Thanks!
Glad to hear it helps you save time! Similar result can be achieved through python pandas, especially if the dataset is big, which power query is slow to handle.
Nice video!!👌👍
Thank you. Do check out other videos as well.
Hi, thanks for your videos, hope you can help me, to open specific page of pdf pages..according to data records.
Hello. I don't think such a thing is possible with VBA. What exact page you need to open?
thank you
You're welcome
Good video
Glad you enjoyed
Hi, what tool are you using to write and test the code? Thanks.
Hello, I am using VS Code and within VS Code, make sure you have extensions by name of *Jupyter* and by Name *Jupyter Cell Tags* installed, read more on link below, they are basically jupyter code cells. code.visualstudio.com/docs/python/jupyter-support-py Hope it helps.
Thank you for the video. How to implement ActiveWorkbook.FollowHyperlink method, so we can open any file, not just PDF?
Hello, Sorry I missed your comment. I will research and let you know what I find. thanks.
Thanks!
Welcome! Do you check out other videos as well!
Can you explain this code video slow.
Hello. Do you have prior experience with python? If not, I can recommend some beginner lecture videos, as these videos are project based and intermediate level skill is needed to go along.
@@theDataCornerokay and which extension do you installed in vs code to look like this browser view VS code.
hello, that is ipynb support in vs code. Please read below for more information on jupyter notebook. code.visualstudio.com/docs/datascience/jupyter-notebooks
I want to ask question. Should we use VScode or change to cursor or storm editor
Hello Winner. I would suggest to use Vs code, as a lot of people use it and it has good function overall. If you want something very simple for basic editing then use sublime editor or even IDLE that comes with python installation. Cheers!
'auto-editor' is not recognized as an internal or external command, operable program or batch file. It is giving me this error please help
Hello. Do you have python programming language properly installed? and auto-editor library installed as well? Open command prompt and type python, press enter. Does it give you the version of python installed?
@@theDataCorner Yes both are installed and I have worked on different other projects as well with python using vs code editor
Hello. Try below. 1- go to the folder the video is in, in windows explorer page where there is usually address like C:\Web.. type CMD and press enter. Paste below in CMD, change information as needed. auto-editor fileName.mp4 --margin 0.2sec --export resolve 2- If above doesnt work, try below in same command CMD window opened above: python -m auto-editor fileName.mp4 --margin 0.2sec --export resolve Let me know if any of above works.
@@theDataCorner Hey, I tried the last command but its hows no module name auto-editor :(
My attachment in the email is a pdf but it is auto saving as MS-DOS. Why is it saving it in a different format?
Hello, Please check answer at the end of page from below link, let me know if it solves your issue. community.plumsail.com/t/power-automate-not-saving-file-extension/13684/3
Exactly what I want. Thanks
Thank you, do check out other videos as well.
Thank you so much! I was getting frustrated with trying to get all the other columns back. There are other tips online (like merging the grouped table with the same table before it was grouped) that just were not working!
Thanks. I am glad it helped. Check out other videos as well, they could be helpful as well.
When I typed the pdf_fils or pdf_files[1] in the editor I didnt get any results. When I typed the pdf_file[0} in the terminal I got an error as the term is not recognized as the name of the cmdlet,
Sept 28, 2024 7:15:35 AM org.apache.pdfbox.pdmodel.font.PDSimpleFont toUnicode WARNING: No Unicode mapping for .notdef (31) in font YTHFYH+MyriadPro-Regular Sept 28, 2024 7:15:35 AM org.apache.pdfbox.rendering.Type1Glyph2D getPathForCharacterCode WARNING: No glyph for code 31 (.notdef) in font YTHFYH+MyriadPro-Regular Traceback (most recent call last): File "CommandLineApp.java", line 106, in technology.tabula.CommandLineApp.extractTables Exception: Java Exception The above exception was the direct cause of the following exception: Traceback (most recent call last): File "c:\Data Extraction From PDF 24.9.24\main7.py", line 20, in <module> tables = tabula.read_pdf(pdf_files[0], pages="all", multiple_tables = True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Data Extraction From PDF 24.9.24\.venv\Lib\site-packages\tabula\io.py", line 400, in read_pdf output = _run( ^^^^^ File "C:\Data Extraction From PDF 24.9.24\.venv\Lib\site-packages\tabula\io.py", line 86, in _run return _tabula_vm.call_tabula_java(options, path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Data Extraction From PDF 24.9.24\.venv\Lib\site-packages\tabula\backend.py", line 73, in call_tabula_java self.tabula.CommandLineApp(sb, cmd).extractTables(cmd) java.lang.java.lang.StringIndexOutOfBoundsException: java.lang.StringIndexOutOfBoundsException: Index 0 out of bounds for length 0 PS C:\Data Extraction From PDF 24.9.24>
Are you able to send me a similar sample PDF file? so I can test it and let you know. Email it here: TheDataCorner5@outlook.com
Sept 28, 2024 7:11:05 AM org.apache.pdfbox.pdmodel.font.PDSimpleFont to Unicode Warning: No unicode Mapping for .notdef(31) in font YTHFYH+MyriadPro-Regular Traceback (most recent call last) : File "CommandLineApp.java", line 106, in technology.tabula.CommandLineApp.extractTables Execption : Java Exception The above excdeption was the direct cause of the following exception: tables = tabula.read_pdf(pdf_files[0],pages="all", multiple_tables = True) File "C:\Data Extraction From PDF 24.9.24\.venv\Lib\site-packages\tabula\backend.py", line 73, in call_tabula_java self.tabula.CommandLineApp(sb, cmd).extractTables(cmd) java.lang.java.lang.StringIndexOutOfBoundsException: java.lang.StringIndexOutOfBoundsException: Index 0 out of bounds for length 0 PS C:\Data Extraction From PDF 24
01:40 how did you edit this to make the vs editor having each seperate cells. Please someone let m ekno w
Hello. Within VS Code, make sure you have extensions by name of *Jupyter* and *Name: Jupyter Cell Tags* installed, read more on link below, they are basically jupyter code cells. code.visualstudio.com/docs/python/jupyter-support-py
Thank you i will it try tomorrow
All the best
Nice, thanks!
@@shoppersdream thank you and check out other videos as well.
Hi Bro In this Viedo Full Blur I need Code in this program
Hey bro, Code can be accessed from below link, let me know if you have any other questions. codepad.app/edit/yz3801h0
Automating personalized emails with Python and Excel will save so much time. thanks for the clear video❤
You are welcome! Glad it was helpful.
Hello, I have an " org.apache.fontbox.ttf.CmapSubtable processSubtype14 WARNING: Format 14 cmap table is not supported and will be ignored"
Hello Smith, Is your PDF scanned or computer generated? The error seems to be related to some font, I have not seen that error before. You can try using camelot, which works in similar way to tabula library, if that doesnt work, you can try using microsoft power query to do same task. ruclips.net/video/b8VTa3gYOBo/видео.html Camelot Code: import camelot import os import pandas as pd # List all PDF files in the current directory pdf_files = [x for x in os.listdir('.') if x.endswith('.pdf')] print(pdf_files) # Initialize an empty DataFrame to store the combined tables combined_df = pd.DataFrame() # Loop through each PDF file for pdf_file in pdf_files: # Extract tables from the PDF tables = camelot.read_pdf(pdf_file, pages='all') print(f"This pdf has {len(ttables)} pages") # Check the number of tables and select the required ones if len(tables) > 4: required_table = pd.concat([tables[2].df, tables[6].df], ignore_index=True, sort=False) else: required_table = tables[2].df # Add the PDF source column required_table['pdf_source'] = pdf_file # Append the required table to the combined DataFrame combined_df = pd.concat([combined_df, required_table], ignore_index=True, sort=False) # Create a copy of the combined DataFrame df_new = combined_df.copy()
Thanks this short video help me a lot.
Hello Albert, Glad to hear that! make sure to check out other videos on the channel as well.
Excellent solution - this saves me a ton of time!
Thank you, I'm glad you found it helpful. Make sure to check out other videos and leave feedback.
Awesome. Thanks for the detailed walkthrough of the process ❤
Thank you, Azeem, for long time fan of this channel!
hello, i can not can not get the pdf_files[0] there is error saying the term 'pdf_files[0]' its not reconized
Hello Mustaqim, Thats strange, can you share the code you are working on? May be you skipped the line where *pdf_files* was defined? Error code means variable named *pdf_files* doesnt exist which is why it cannot recognise it.
How to get data from pdf that requires password?
Hello Loyd, Please try below answer from *THE ARIN KAMBLE* on this page, seems to be working, I have not tested it myself. learn.microsoft.com/en-us/answers/questions/1725182/how-can-i-export-data-on-excel-from-a-password-pro
@@theDataCorner send the print command on that pdf and save it as a pdf again
@@MrSiaa I havent tested it, but it might work.
"screenshot to image" not in video
Hello This seems to be a typo in description, its actually Screenshot to Text (OCR). Thank you.
thank you for this. good tricks just wonder if you could share the code in the comments. easier to avoid typos
Thank you, Ahmed, Below is the link for the source code. Make sure to check other videos as well. codepad.site/edit/8r4ld901
@@theDataCorner thank you. are you on linkedIn?
Hi man. thanks for this. possible to share the code in the notes?
Hello Ahmed, sure. Code is below, I will add in video description as well. codepad.site/edit/3w48mae6
send source code and btw getting error like java not found , so help me resolve it , appreciate your work.
Hello, if you have java installed already and still getting an error, then please try below steps, the java setup is bit tricky but hopefully a one time setup. from windows start option, search for *Environment Variables* and search for *Edit environment variables*, then follow below steps: **** Under the System Variables click Path and then press the Edit... instead of New. Then in the next screen (Edit environment variable for the Path variable) click New and add the address, e.g. C:\Program Files (x86)\Java\jre1.8.0_201\bin. Press OK and the Path variable will be appended/updated.**** Answer taken from below: stackoverflow.com/questions/54817211/java-command-is-not-found-from-this-python-process-please-ensure-java-is-inst Source code: codepad.site/edit/q9aig7rj
@@theDataCorner thanks for your time bro , keep it up .
Happy to help!
Can I add images into my posts?
Hello, I will check and advise.
Good solution. If you want to remove just blank columns you could also transpose then just apply the Remove Rows>Remove Blank Rows which results in a cleaner formula. I do like the flexibility of your solution though to remove columns with only 1 entry.
Great idea, Tim, Thank you. And make sure to check out my other videos on the channel as well.
why the pdf say no preview
Can you give me an exact time in the video where it says that? And i will look into it for you.
I get a Runtime 429 error on the line with Set OutApp = CreateObject("Outlook.Application") and can not seem to trouble shoot the issue. I have tried early binding and resetting the .DLLs. Any suggestions?
Hello Angel, Please try out below. To use early binding, you first need to set a reference to the Outlook object library, Microsoft Outlook xx.x Object Library (xx.x is version number and depends on outlook you have) Below code lines will need to be changed. Dim olApp as Outlook.Application Set olApp = New Outlook.Application answer taken from: stackoverflow.com/questions/60266233/i-am-getting-a-run-time-error-429-trying-to-get-excel-macro-to-send-a-email Please try it out and let me know.
Good solution, except you lose the column headers. Is there a way to do this without having to rename every column back to what it was?
Thanks Evan, let me look into it and get back to you.
Hi, this is also what I am curious about? Everything else works fantastically, but I do need to keep my headers.
After playing around with it, if you demote the columns (select the Use Headers as First Row option), then do everything in the video, and then promote the columns (Use First Row as Headers) you'll keep your headers!
Thank you Ramisa, thats a great solution. I found a workaround but your solution seems much better and elegant, appreciated!
@@theDataCorner Happy it worked out :)
Is their a way that I can reply that email directly to that to-do list I created?
Hey, John. Do you want to reply to the same/original email? its an interesting question, let me research a bit and get back to you.
Hi, Ye. I try to do make it work like that but no luck. I think it’s much convenience if we can reply directly to that to do list instead of searching it in the inbox to reply.
Hello, i was able to find a way to do this, i will make a small video on it and share link here.
Thanks man!! You’re the best!
@@jayenriquez4550 Thanks. Check out below video, let me know if this is what you are looking for. ruclips.net/video/WTlAaezmlus/видео.html
I was just curious - you can convert all the "dates" of text data type to number/date data type by multiplying by 1, no need to program?
Hey Ananda, Thats a very clever way to fix dates, not many people know it. However, multiplying by 1 can only fix dates which are already in correct format as per US dates mm/dd/yyyy and are recognised at text instead of dates. In such situations where you have dd/mm/yyyy, multiplying by 1 doesnt fix it. Check out this youtube short for another quick way to fix such dates. ruclips.net/user/shorts9T7d_AEGpvg?feature=share
Great helpful content. Keep it up ❤
Thank you 🙌
How to do the same use vba code
Hello I will create a small youtube video explaining how to do this, I will add a link here once I am done, should be soon.
Thank you sir But in my excel there is nothing like this in home tab. Which excel support this?
Hello, per google, please try below: go to File --> Options --> Add-Ins --> Manage Excel Add-ins --> Go, and then check the Analysis ToolPak box and click OK
Good work there, Funny enough, I came across this video because I'm doing something in relation to proforma invoice
Glad it was helpful! I have another VBA video where you can open multiple PDF files in one click, it can also work on visible/filtered rows only, make sure to check it out. Thank you. ruclips.net/video/V33cB3HfgZ8/видео.html
Please provide the link to the excel file
Hello, please access the excel file using below link: docs.google.com/spreadsheets/d/1e0u6WVSPxBNJTFr1bckRMbJw2zdI9Qou/edit?usp=sharing&ouid=110820750791230145171&rtpof=true&sd=true Also adding link to the code: codefile.io/f/rMHi4ctl6X
Very helpful video. Thanks for this ❤
Thank you for watching. I hope you learnt something new.
Great video! Your explanation of PowerToys was super clear and helpful. ❤
Thank you Azeem, hope you enjoyed the content!
0:00
Hi, I am having a crazy task that might be impossible to execute. I already processed the table using PQ, so my table is dynamic and information may in and out depending on my Status. When project is "Done" it's removed from the final table, when project has an "In Progress" status it gets into the final table for my analysis where I have some manual formulas (that is no embeded in PQ) and Text. Hope this step is clear for now! And the most funny issue is that in this final table (excel table that is refreshed) I am adding a column, when I enter a formula there is no issue, because this formula applies to all active cells in the table right? But when I add some "text" - it does not work like a formula and "text" remain in the same cells in spite of my projects may in and out, and "text" does not match the project anymore, does excel have a solution here? Where I would like that my "Text" will be dynamically moved up and down or removed if project goes out from the table :) I can not find the answer unfortunately
Hey, that is an interesting issue. From what I understood, the text corresponds to the project, and I am assuming each project has some unique identifier. One suggested way would be to create a helper table with Project ID and notes, and then merge this table with your final table using Project ID. Let me know if that solution works out for you. Cheers!
very useful. thanks for this🌻
Thank you Azeem, appreciate it.
intresting stuff!
Thank you, Azeem.