i get a base64 error when i want to import pygooglenews because it uninstalls feedparser 6.0 and replaces it with 5.2 during pygooglenews package install how can i solve this?
Had an issue with the install. Had to downgrade setuptools to the older version that has support for 2to3: pip install "setuptools=1.0.0" pip install -U --no-deps "feedparser>=6.0.8"
I'm still having the problem, any suggestion? Collecting pygooglenews Using cached pygooglenews-0.1.2-py3-none-any.whl.metadata (19 kB) Requirement already satisfied: beautifulsoup4=4.9.1 in c:\python312\lib\site-packages (from pygooglenews) (4.12.2) Collecting dateparser=0.7.6 (from pygooglenews) Using cached dateparser-0.7.6-py2.py3-none-any.whl (362 kB) Collecting feedparser=5.2.1 (from pygooglenews) Using cached feedparser-5.2.1.zip (1.2 MB) Preparing metadata (setup.py) ... error error: subprocess-exited-with-error × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [1 lines of output] ERROR: Can not execute `setup.py` since setuptools is not available in the build environment. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details.
@@JohnWatsonRooney Can you please do a video on how to use asynchtmlsessions/arender(), where I have to hit like 50 urls that are javascript, with each request using a different proxy ip from a list, is that possible? i'm trying to hit a few pages and the content is missing and if I did 1 at a time it would take forever. Thank you
@@JohnWatsonRooney awesome, I wasn't sure in that scenario where I need to rotate proxies would it be better to use asynchtmlsession vs normal htmlsession and thread it with concurrent futures but I kept getting errors with concurrent futures and couldn't figure it out. Thanks
Also, it didn't really work, I got an error: Traceback (most recent call last): File "C:\Users\Ryan\PycharmProjects\Stock\Stock Searcher.py", line 1, in from pygooglenews import GoogleNews File "C:\Users\Ryan\PycharmProjects\Stock\venv\lib\site-packages\pygooglenews\__init__.py", line 1, in import feedparser File "C:\Users\Ryan\PycharmProjects\Stock\venv\lib\site-packages\feedparser.py", line 93, in _base64decode = getattr(base64, 'decodebytes', base64.decodestring) AttributeError: module 'base64' has no attribute 'decodestring' Process finished with exit code 1
pygooglenews uninstalls feedparses 6.0 and replaces it with 5,.2 which doesnt work on python 3.9 so thats anoying. i guess its possible to write exceptions to now do it or download a lower version of python that runs feedparser 5.2
@@JohnWatsonRooney I'm actually not using this package, but just using bs4 to scrape google search results. They seem to have some advanced rate limit tracking so maybe a proxy is the way to go but I'd prefer to figure out a better way
Awesome content as usual! How exactly would I go about pairing a list of search terms with the articles it pulls? So for example if I created a list called SearchList = ['Basketball', 'Football', 'Hockey'] and I iterated through this list to search for all articles related to each list item. How would I then associate each item from SearchList to their corresponding articles and printing it out in a neat fashion. So almost having one column showing the search term and the second column showing the corresponding article? Thank you!
I fail already on first line "from pygooglenews import GoogleNews". My error is "AttributeError: module 'base64' has no attribute 'decodestring'". I am using last version of Python. Just me?
with respect, your lecture is much valuable and I get a lot of knowledge from it, but I have a question about how to news file to an excel sheet. I would be thankful
Does anyone knows why do I get this error message? pip install pygooglenews --upgrade Collecting pygooglenews Using cached pygooglenews-0.1.2-py3-none-any.whl (10 kB) Collecting feedparser=5.2.1 Using cached feedparser-5.2.1.zip (1.2 MB) Preparing metadata (setup.py) ... error error: subprocess-exited-with-error × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [1 lines of output] error in feedparser setup command: use_2to3 is invalid. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details.
easy fix... Had to downgrade setuptools to the older version that has support for 2to3: pip install "setuptools=1.0.0" pip install -U --no-deps "feedparser>=6.0.8"
@@informationdominance6434 Thank you for your answer. I did what you did. unfortunatly i still get the error ModuleNotFoundError: No module named 'sgmllib'
@@miguelnuno928 Try this, it just worked for me now: !pip install "setuptools=1.0.0" !pip install -U --no-deps "feedparser>=6.0.8" !pip install pygooglenews==0.1.2
@@JohnWatsonRooney same here. It works but cannot filter by date. Also Another thing I'm missing is a short description of the article, which I need in my dataset. Do you have any advice on the best Google News scraping method right now?
@JohnWatsonRooney Bro Grow your Channel rn put some videos showing poeple how to code all this amazing stuff using Chat GPT you will be 100 K subs in two months.
Creator of the package here. Nice video!
Hey! Thanks for creating it for us to use, I’m glad you liked the video!
You are live saver
i get a base64 error when i want to import pygooglenews because it uninstalls feedparser 6.0 and replaces it with 5.2 during pygooglenews package install how can i solve this?
Hey man, is there any way we can get thumbnail of the headlines?
@@MrSigi1990 pip install feedparser==6.0
This video and the pygooglenews source code are great intro to scraping rss feed. Thanks John!
I honestly love RUclips, you've just done everything I wanted to do for me! Now just to try it out, thanks!
Haha glad to help!
Had an issue with the install. Had to downgrade setuptools to the older version that has support for 2to3:
pip install "setuptools=1.0.0"
pip install -U --no-deps "feedparser>=6.0.8"
I'm still having the problem, any suggestion?
Collecting pygooglenews
Using cached pygooglenews-0.1.2-py3-none-any.whl.metadata (19 kB)
Requirement already satisfied: beautifulsoup4=4.9.1 in c:\python312\lib\site-packages (from pygooglenews) (4.12.2)
Collecting dateparser=0.7.6 (from pygooglenews)
Using cached dateparser-0.7.6-py2.py3-none-any.whl (362 kB)
Collecting feedparser=5.2.1 (from pygooglenews)
Using cached feedparser-5.2.1.zip (1.2 MB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [1 lines of output]
ERROR: Can not execute `setup.py` since setuptools is not available in the build environment.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
Hi John I just wanted to say your videos are awesome thanks
Thanks!
@@JohnWatsonRooney Can you please do a video on how to use asynchtmlsessions/arender(), where I have to hit like 50 urls that are javascript, with each request using a different proxy ip from a list, is that possible? i'm trying to hit a few pages and the content is missing and if I did 1 at a time it would take forever. Thank you
@@kevin-wg5iv its coming up don't worry :)
@@JohnWatsonRooney awesome, I wasn't sure in that scenario where I need to rotate proxies would it be better to use asynchtmlsession vs normal htmlsession and thread it with concurrent futures but I kept getting errors with concurrent futures and couldn't figure it out. Thanks
Very helpful. You are always looking for new ways. 💕
Glad you think so!
AttributeError: module 'base64' has no attribute 'decodestring'
. my py is 3.9.6
did you get it working? I had the same issue and removed some dep.
Great video, you coud have used the built-in 'json' package to display the json data in a more readable way
Thank you very much this tutorial helped me with my current python project. :)
Also, it didn't really work, I got an error:
Traceback (most recent call last):
File "C:\Users\Ryan\PycharmProjects\Stock\Stock Searcher.py", line 1, in
from pygooglenews import GoogleNews
File "C:\Users\Ryan\PycharmProjects\Stock\venv\lib\site-packages\pygooglenews\__init__.py", line 1, in
import feedparser
File "C:\Users\Ryan\PycharmProjects\Stock\venv\lib\site-packages\feedparser.py", line 93, in
_base64decode = getattr(base64, 'decodebytes', base64.decodestring)
AttributeError: module 'base64' has no attribute 'decodestring'
Process finished with exit code 1
Hello, I have this problem 'base64' has no attribute 'decodestring'. I'm trying to solve it but I can't understand how to do it. Can you help me?
Try this, it just worked for me now:
!pip install "setuptools=1.0.0"
!pip install -U --no-deps "feedparser>=6.0.8"
!pip install pygooglenews==0.1.2
How to scrap the content of the news headline using this pygooglenews?
fantastic video. thank you!
Thank you. Can I get the images used by Google News?
Thank you very much. Awesome tutorial as usual.
Interesting! Ty!! Now i have more inspiration on what to look at.(;
"AttributeError: module 'base64' has no attribute 'decodestring'
'
I'm getting this error when I'm using the pygooglenews package can you help me out?
pygooglenews uninstalls feedparses 6.0 and replaces it with 5,.2 which doesnt work on python 3.9 so thats anoying. i guess its possible to write exceptions to now do it or download a lower version of python that runs feedparser 5.2
pip install feedparser==6.0
Is there anyway to get the whole article or at least the first x characters of the article for each story?
You can use the newspaper3k package to do that
Great content! Thank you. When those guitars will be played though?
Sir what if i wanted to search both LOCKDOWN and FOOTBALL together??? please help me im currently facing with this issue.
hello, i want to ask to you. i want to add content of the article. so, what should i declare? please, answer my question and thank you🙏
Please create on a video how to scrape Google SERPs for mobile and desktop and keep track of ranking.
Thank you. But can not collect the news contents??
hi, unfortunately i think this package has stopped working
Thank you John for this helpful video. Is there any way to remove the limit for 100 results?
Thanks! I am not aware of a way to remove the limit, I believe it is google news that controls that
What's a workaround for getting 429 responses? Trying to scrape google search results
I think this package was abandoned and no longer works I’m afraid
@@JohnWatsonRooney I'm actually not using this package, but just using bs4 to scrape google search results. They seem to have some advanced rate limit tracking so maybe a proxy is the way to go but I'd prefer to figure out a better way
Awesome content as usual! How exactly would I go about pairing a list of search terms with the articles it pulls? So for example if I created a list called SearchList = ['Basketball', 'Football', 'Hockey'] and I iterated through this list to search for all articles related to each list item. How would I then associate each item from SearchList to their corresponding articles and printing it out in a neat fashion. So almost having one column showing the search term and the second column showing the corresponding article? Thank you!
you put the results of the search in a pandas dataframe and add a new column with the corresponding search term!
I fail already on first line "from pygooglenews import GoogleNews". My error is "AttributeError: module 'base64' has no attribute 'decodestring'". I am using last version of Python. Just me?
I’m afraid this package does not work anymore, it’s a shame
Can we export to html files the result from scraping? To make static site
with respect, your lecture is much valuable and I get a lot of knowledge from it, but I have a question about how to news file to an excel sheet. I would be thankful
can you make a tutorial how to display scrap data on flask website ?
Yes I’m working on one!
now the package isnt available, :((
How to get thumbanil/image of the article?
Thank you.
Does anyone knows why do I get this error message?
pip install pygooglenews --upgrade
Collecting pygooglenews
Using cached pygooglenews-0.1.2-py3-none-any.whl (10 kB)
Collecting feedparser=5.2.1
Using cached feedparser-5.2.1.zip (1.2 MB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [1 lines of output]
error in feedparser setup command: use_2to3 is invalid.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
I’m afraid I think this package no longer works so this video is out dated now
Hi John, thanks a mill for this video it has helped a lot. Is there a way to turn the result into an Excel file after getting the list?
Sure - I usually use pandas to create a data frame from the list then export to csv but you can use the csv module in Python if you’d rather
I like it!
Thanks for the video, can we return the actual article content?
Unfortunately I don’t think this package works anymore. You can still scrape news but it would be direct using requests rather than this
@@JohnWatsonRooney Sounds good thanks!
how to scrape data from mobile app??? please consider it
Did not work for me. The deps feedparser and others need to be updated.
easy fix...
Had to downgrade setuptools to the older version that has support for 2to3:
pip install "setuptools=1.0.0"
pip install -U --no-deps "feedparser>=6.0.8"
@@informationdominance6434 Thank you for your answer. I did what you did. unfortunatly i still get the error ModuleNotFoundError: No module named 'sgmllib'
@@miguelnuno928 Try this, it just worked for me now:
!pip install "setuptools=1.0.0"
!pip install -U --no-deps "feedparser>=6.0.8"
!pip install pygooglenews==0.1.2
Hey bro can you make a tutorial integrating Scrapy with Django
humongous amount of errors when trying to parse date
Unfortunately I don’t think this works anymore
@@JohnWatsonRooney no problem thanks for the reply. Was trying to use it for my dissertation but it’s due today so couldn’t make it work
@@JohnWatsonRooney same here. It works but cannot filter by date. Also Another thing I'm missing is a short description of the article, which I need in my dataset. Do you have any advice on the best Google News scraping method right now?
IP will be blocked in no time.
Hi john I'm still waiting for your email. So i could send you site for the tutorial
hi! its on my main RUclips page, i'd rather not post it in comments as it get picked up for spam.
@JohnWatsonRooney Bro Grow your Channel rn put some videos showing poeple how to code all this amazing stuff using Chat GPT you will be 100 K subs in two months.