Python Requests Tutorial: Request Web Pages, Download Images, POST Data, Read JSON, and More

Поделиться
HTML-код
  • Опубликовано: 4 фев 2025

Комментарии • 513

  • @coreyms
    @coreyms  6 лет назад +337

    My next video will be a real-world example of a script I wrote to monitor my personal website. We'll use the Request library to monitor the site, and if it is down then we will learn how to send an email and automatically restart the server. I hope everyone finds this useful! Hope you're having a great week!

    • @premjilla8820
      @premjilla8820 6 лет назад +10

      can you please make tutorial o scraping twitter data using tweepy?

    • @coreyms
      @coreyms  6 лет назад +28

      That's on my list of topics to cover, but I have some data science videos I'm trying to put together first. But yes, I definitely will at some point

    • @swadhikarc7858
      @swadhikarc7858 6 лет назад +3

      Awesome idea. Thought that would be a great tutorial to see you teaching how to monitor a website in a candid way

    • @tnhoang088
      @tnhoang088 6 лет назад +2

      @@coreyms Need more data science, computer vision or even Javascript videos. Your lecture is so good but do not upload all the series at the same time like the django, flask before. It's so overwhelming. And if you can, I slightly recommend that you should make a series about Javascript, Nodejs or something related to js. I think it's much more applicable than data science or AI ... Just a suggestion ;)
      Always appreciate your way of teaching

    • @sujildevkota
      @sujildevkota 6 лет назад

      Can you make a video to run python script in the background and display message and progress bar in the web page using Django

  • @NaderTaghinia
    @NaderTaghinia 5 лет назад +107

    09:57 " I never understand how some of these people are so productive and..."
    Since, I started watching your channel I have been thinking the exact same thing about you Mr. Schafer. Honestly, I could never understand how you could author so many impeccable videos on programming so fluently. You are giving so much to people like me. Thank you from the bottom of my heart :)

    • @coreyms
      @coreyms  5 лет назад +23

      Oh wow. Thanks so much, Nader. I really appreciate the kind words :)

    • @karimmoradi2003
      @karimmoradi2003 Год назад +4

      The same. I stopped video to write something like this. Corey! I watched many of your videos but I am still in doubt how you can be so humble, knowledgeable, and productive. You are just incredible. Along side of technology, I am learning morality. Thanks

    • @RamiSobhani
      @RamiSobhani Месяц назад +1

      I was about to write your comment lol

  • @AlexBerkk
    @AlexBerkk 5 лет назад +389

    That's it! I'm disabling my adblock for your channel!

    • @deus5215
      @deus5215 4 года назад +1

      @yeeLix Felt that

    • @paulthedanger
      @paulthedanger 4 года назад +1

      @ً ok hackerr

    • @arielgorlik2664
      @arielgorlik2664 4 года назад +2

      @ً check out privacy badger

    • @wege8409
      @wege8409 4 года назад +3

      What a gentleman

    • @rodricbr
      @rodricbr 4 года назад +6

      I'm disabling my brave shields for this guy

  • @DifficultWorld7
    @DifficultWorld7 3 года назад +9

    I have wasted my money on a python course on Udemy I've regretted to this day, and this guy spreading his knowledge for Free, such an awesome guy you are.. THANK YOU!!

    • @n9583
      @n9583 3 года назад

      I’ve done that before

    • @saurabhchaudhary7342
      @saurabhchaudhary7342 3 года назад

      I also want to start with Python. Given the fact that it's a four year old series, is it still enough ?

  • @ivyunhinged9521
    @ivyunhinged9521 3 года назад +6

    No matter how much I know or who I start learning from, I always end up back to Corey for some reason. Clear and concise!

  • @timtudosa2519
    @timtudosa2519 5 лет назад +36

    You will reach one million subscribers soon Corey. Your videos are so detailed and and easy to follow through. I now understand requests. Thanks for the effort in making these videos!

  • @cyberguygame9096
    @cyberguygame9096 4 года назад +1

    Normally I'm the guy that is getting aggressive because the guy in the tutorial video doesn't tell what I want to know, but THIS -THIS IS DAMN GOOD.

  • @andonglin8900
    @andonglin8900 6 лет назад +8

    WOW, the 100th video in your MARVELOUS Python tutorial series with my 100th thumbs up. Thank u Corey, you are truly amazing!

  • @misskay007
    @misskay007 5 лет назад +16

    Harvard should hire you. I learned so much from watching your videos. Much more through than than CS50 course in web development(although it is a great series). Thank you, your hard work is very much appreciated. I will be donating and sharing.

  • @archstampton5910
    @archstampton5910 6 лет назад +7

    Probably one of the more valuable RUclips video I have ever seen, as far as I am concerned.

  • @rahulreddy6541
    @rahulreddy6541 4 года назад +25

    You have got passion in your voice which is helping people like me to learn programming in a much fun way. Love you for what you are. love you 3000.

  • @pranaybure2651
    @pranaybure2651 6 лет назад +26

    Excellent insight on Requests, Corey!
    Requests library is my goto tool when APIs aren't working as expected.
    I recently used Requests(Sessions) in a script to mimic browser activities for a web portal as APIs had issues with patch fixes.
    Understanding browser components with Chrome Dev Tools in conjunction with Requests was really helpful to understand how requests attribute viz. Headers, Form-Data, Query-string params are assembled.

  • @JoshuaDHarvey
    @JoshuaDHarvey 4 года назад +2

    10:06 dude, welcome to the club. Your videos alone make most of us feel the same way so I can only imagine if these people from your perspective are another step up in productive... then I'm completely puzzled! Anyway, thanks for a great video Corey!

  • @muralimohan807
    @muralimohan807 5 лет назад +24

    Corey you are the best trainer or develper for me as when i see your videos ill get both new things and the things in my mind that i wish to ask you but you answer them in your video itself. so you are trainer and develper for me.keep the good work cheers.

  • @ramonmonda
    @ramonmonda Год назад

    Hello Corey, I'm Brazilian and your video helped me a lot with my project! Thanks for the class! 👊

  • @lamis_18
    @lamis_18 3 года назад

    The best channel, finally ! I have found all basics that I've missed before, and I regret to not find it long time a go, thank u very much.

  • @victoraugusto2224
    @victoraugusto2224 4 года назад +1

    thanks man, im currently working in a project for a job application, your explanation saved my life

  • @RajPatelMansuria
    @RajPatelMansuria 5 лет назад +4

    You are awesome, the way you explain stuff with such calmness and speed is just out of the world. Thanks you again for posting great stuff 👏👏👏

  • @sevenessie
    @sevenessie 6 лет назад +17

    Hey Corey, just wanted to tell you what I realized recently.
    Whenever I'm reading a tutorial or some explanations in python/web dev area, I read it with your voice :D.
    Thanks for the great work and please don't stop :)

    • @alan-j
      @alan-j 5 лет назад +1

      Yes, I would say coreyms is the best place to learn python in the whole of youtube network..
      Please dont stop this good work..
      HATS OFF TO U..

  • @sudhirs003
    @sudhirs003 4 года назад +1

    This is called something left behind when we move ahead...Good work Corey...

  • @sudarsandm
    @sudarsandm 6 лет назад +3

    Your video is a time saver. The information which we need to process from more than one blog or resource is presented elegantly and beautifully to us. Thank you Corey.

  • @dan-3268
    @dan-3268 5 лет назад +2

    Ok, Corey... Corey-Corey-Corey.....*ominous silence"
    You are the Man. Hands down the most well-put and thought through videos on Python I've seen on RUclips.
    Much appreciated! I thank thee for thy work for tis magnificent

  • @narendrar9940
    @narendrar9940 6 лет назад +5

    Hi Corey, Your videos have become my one source of all the information i need on python, your videos are simple and just great!, thank you very much.

  • @nickd892
    @nickd892 4 года назад

    Lol your the first guy I look for when I need a tutorial. Great video as always!

  • @sunilchetri3509
    @sunilchetri3509 3 года назад

    Corey, your name is right up there in the list of people who are so productive and helpful. Your videos are just perfect for someone like me learning the ABCs of Python. Searched and listened to scores of videos but your videos are just perfect! no more no less. Your videos do not overwhelm someone like me. Am loving every bit of them. Just want to let you know that you are awesome!

  • @thengakola6217
    @thengakola6217 4 года назад

    That was one of the few clean videos on the entire internet.........

  • @silverzero9524
    @silverzero9524 6 лет назад +1

    i wanted to get info from site but was stuck so wrote some bad code that doesnt help me much but works
    and now u post this
    time to modify my code :)

  • @pratikshinde568
    @pratikshinde568 4 года назад

    Sir yours video is much more informative and is in more detail compared to others

  • @chaitanyakhambete
    @chaitanyakhambete 5 лет назад +2

    awesome work bro... making simpler stuff more simpler keep it up

  • @ihalil95
    @ihalil95 5 лет назад +2

    I am really looking forward to seeing a video about threading. I hope you will make it as well! Thanks for great tutorials even better than paid courses around in my opinion! I'd rather support your channel instead of paying for less quality courses.

  • @robertocardenosa493
    @robertocardenosa493 11 месяцев назад

    This is a great video. I plan to test its examples within python. The only thing I missed was the Patch command.

  • @robinharris4706
    @robinharris4706 5 лет назад +6

    Great tutorial Corey. Really like your stuff - thank you for covering the material at a brisk pace. Loads of excellent, well presented content without having to listen to slow tedious simplifications. Awesome!

  • @adamgdev
    @adamgdev 4 года назад +1

    You fucking killed it again! So precises and easy to understand AND had some good random nuggets in there. (HTTP request code meanings and httpbin). You're the man! Keep it up!!!

  • @rohitd7834
    @rohitd7834 5 лет назад +2

    I was struggling a lot with requests. but now, I'm clear with it. Thanks Corey!! NO words to thank you. :)

  • @krishnakuraku6853
    @krishnakuraku6853 6 лет назад +1

    looking forward for more videos like this ..pleas post more videos..you are the best tutor in python

  • @mrj1997
    @mrj1997 6 лет назад +6

    Please teach everything ! your videos are fantastic.Thanks Corey

  • @philmontgomery502
    @philmontgomery502 4 года назад +2

    Excellent delivery Corey... the best!!

  • @juanbomfim22
    @juanbomfim22 4 года назад +32

    I'm crying inside cause that's exactly what I was looking for

    • @Seiseary
      @Seiseary 4 года назад +1

      hahahaah

    • @Lakshit.
      @Lakshit. 4 года назад +3

      Damn true bro
      ... I was exactly looking for same

    • @Danksta
      @Danksta 4 года назад

      I’ve never related more to a RUclips comment

  • @henriquepigozzo4707
    @henriquepigozzo4707 5 лет назад +1

    The timeout param is amazing. Thanks!

  • @normoforan1444
    @normoforan1444 Год назад

    If you send a GET request to Merriam Webster online of "BEAST MODE" you will get back a png of Corey!!! 😂 Super job Corey, your videos are always uber understandable even to a neophyte such as myself!!!

  • @ddpwe5269
    @ddpwe5269 2 года назад

    Gratz on your 1mil subs! Well deserved and surprised more haven't sub'd!

  • @iadorepop
    @iadorepop 6 лет назад +12

    believe me !! i was so waiting for this 💓

  • @teklehaimanotaman3150
    @teklehaimanotaman3150 3 года назад +1

    whenever I am challenged while working with python, then my mind recalls Corely.

  • @ahmadabzakh2343
    @ahmadabzakh2343 5 лет назад +1

    Hello Corey,, have been following you for quiet a while now,, really helpful in overall understanding of the subject in hands,, however, this is the first clip where I don't understand what's happening exactly, and obviously it's my problem, so wanted to ask about what prerequisites are needed in order to follow along in the video,
    again man many thanks for the effort put here,, you are one of a kind

  • @drygordspellweaver8761
    @drygordspellweaver8761 2 года назад

    Thank God Corey has a requests tutorial

  • @Uxrandom996
    @Uxrandom996 2 года назад +1

    Thanks, for such an awesome high quality tutorial 😀 Very clear and quick to the point!

  • @jag831
    @jag831 4 года назад +1

    10:04 Yeah, I think that same thing every time I open any library. Respect to those guys

  • @Bergamoism
    @Bergamoism 6 лет назад +4

    Great job on explaining this! Your tutorials are to the point, clear and cover everything we need to know. Every video takes me to the next level.

  • @arete_
    @arete_ 3 года назад

    This was a good video. I am still struggling with using session objects though. Would love an in-depth on that. Happy new year! Stay safe

  • @joeking5211
    @joeking5211 Год назад

    Absolutely super, well done, very well paced. Thank you for this, learned a lot.

  • @ravindersanjay
    @ravindersanjay 4 года назад +1

    this video is worth watching. full of knowledge

  • @asifahussain899
    @asifahussain899 2 года назад

    Corey fam gonna be 1 million.. 💚
    Cheers 🥂

  • @elghark
    @elghark 2 года назад

    Do u remember Miller's planet in the Interstellar movie?
    There, time used to move much slower than on Earth. Well, Corey's videos are the opposite: 20 minutes of contents here are the equivalent of 1 month spent studying hard on a python library ☺️☺️

  • @manishadwani386
    @manishadwani386 6 лет назад

    Awesome video tutorial as always Corey. Believe me I'm writing this comment after watching the entire video and it's great, even though I knew requests I still learned something new, So Thank you. One more thing, I noticed there are not a lot of asynchronous python video tutorials out there. I would love it if you could make a video on explaining different tactics and techniques used, to make your code asynchronous and an asyncio tutorial elaborating async/awaits functions.
    Anyways think about my request and thanks again for this great video.

    • @coreyms
      @coreyms  6 лет назад

      I definitely want to do that. I've been trying to plan some videos like that, but it is a very hard topic to teach well. Hopefully I'll get something together that can explain it.

    • @manishadwani386
      @manishadwani386 6 лет назад

      @@coreyms awesome man, it's great that you're trying to put something together. Thanks

  • @j.d.3890
    @j.d.3890 6 лет назад

    Hey Corey, thanks for this guide.
    In work sometimes `allow_redirects=False` as request param is helpful as well.
    Also if your request resulted in several redirects (which is also quite usual), request.history will return all request objects which have happened.

  • @bgable7707
    @bgable7707 4 года назад

    Concise, clear, explanation and examples. Always hitting the key points with any fluff, thanks!!!

  • @axeditingzone513
    @axeditingzone513 3 года назад

    thanks alot you saved my day .... i have only mistake in post request thanks alot

  • @huy3986
    @huy3986 4 года назад

    I really love your videos. Thank you so much and wish you're always happy.

  • @howiewhite4773
    @howiewhite4773 3 года назад

    You explain things the simplist and the best, i still cant get any of this stuff to work.

  • @yolberpabon4973
    @yolberpabon4973 4 года назад

    Genial Corey, me gusto el video y era lo que buscaba, una explicación rápida y clara de que es requests y ademas lo haces con profesionalidad (+sub)

  • @willumdgork
    @willumdgork 4 года назад

    Outstanding tutorial! Thanks for the coaching and for pointing out the httpbin tool!!!

  • @exinty
    @exinty 3 года назад

    Thanks so much for your help.

  • @MusicGameFinatic999
    @MusicGameFinatic999 5 лет назад +1

    You are the man! You very much. Very well explained.

  • @thatlinuxguy9902
    @thatlinuxguy9902 3 года назад

    The best tutorial I've ever seen

  • @ananyamahapatra6597
    @ananyamahapatra6597 2 года назад

    😍😍😍 hey Corey… u are just the best 🥳

  • @falconX5000
    @falconX5000 6 лет назад +1

    Great work pal!!!!!! Looking forward to your next video.

  • @robinhood3841
    @robinhood3841 5 лет назад +2

    Thanks for the video we can now perform Dos attack
    XD am just kidding its a good method to use keep it up buddy

  • @potatobot2802
    @potatobot2802 Год назад

    Thank you so much Corey for your time!

  • @myselfandpesit
    @myselfandpesit 2 года назад

    Thanks for the video. I am a total beginner to APIs. Would be good if you could suggest videos.

  • @jeffgalef121
    @jeffgalef121 5 лет назад +1

    Fantastic video. Thank you for your hard work.

  • @kautilyab
    @kautilyab 4 года назад +4

    For anyone who needs the most of the code written in this session :
    #pip install requests
    import requests
    r=requests.get("xkcd.com/353/")
    print(r)
    print(dir(r))
    print(r.text)
    #downloading images
    r=requests.get("imgs.xkcd.com/comics/python.png")
    print(r.content)
    #writing a downloaded image to system
    with open ('comic.png', 'wb') as f:
    f.write(r.content)
    print(r.status_code)
    #200 is success
    #300 is redirects
    #400 are client errors .ie. if you don't have access or permission for you
    #500 are server errors
    print(r.ok)
    #Return true for anything less than 400
    payload= {'page':2,'count':25}
    r=requests.get('httpbin.org/get', params=payload)
    print(r.text)
    print(r.url)
    payload= {'username':'corey','password':'testing'}
    r=requests.post('httpbin.org/post', data=payload)
    #data is used for payload to be more likely in a form
    print(r.text)
    #no args as it is a url parameter
    #form is uploaded
    #to know what values the 'form' url expects, we need to look at the source code
    #of the url
    #most of the times the output we get will be in json, so we have method we could use
    print(r.json())
    #created a python dictionary from json response
    #to capture that in a variable
    r_dict=r.json()
    print(r_dict['form'])
    #the authentication done above is form based authentication
    #there are other types of authentication like basic authentication
    #'httpbin.org/basic-auth/corey/testing' this a url which basic authentication
    #and accepts only username= corey and password=testing
    #in auth, we provide a tuple for input parameters
    r=requests.get('httpbin.org/basic-auth/corey/testing', auth=("corey","testing"))
    print(r.text)
    print(r)
    r=requests.get('httpbin.org/basic-auth/corey/testing', auth=("coreyms","testing"))
    print(r.text)
    print(r)
    # , so unauthorized response
    #when checking if website is working or not it is good practice to keep "timeout"
    #or else it might hand indefinitely, exceptions if api's in website take too much time to load
    #'httpbin.org/delay/{delay} is used to delay the site by certain time
    r=requests.get('httpbin.org/delay/6', timeout=3); print(r)
    r=requests.get('httpbin.org/delay/1', timeout=3) ; print(r)

    • @acpucio
      @acpucio 4 года назад +1

      Thanks!!!

  • @serious6037
    @serious6037 2 года назад

    jfc, i finally found this... THANK YOU!!!!

  • @someonemaking5626
    @someonemaking5626 2 года назад

    this guy has videos on everything

  • @olha_
    @olha_ 6 лет назад +2

    Great video as usual Corey! Please do more videos on Django and Rest Framework, thanks!

  • @wanderer-1986
    @wanderer-1986 5 лет назад

    Awesome video! Thanks for posting this. Easy to follow your examples. Thank you

  • @chaks2432
    @chaks2432 4 года назад

    Awesome, now i don't have to save each page 1 by 1 when downloading comics, just add the link and presto!

  • @chaks2432
    @chaks2432 4 года назад

    Took me a couple hours, but finally managed to modify the code so I can download entire comics by iterating through the pages., and storing the downloaded comics in individual folders

    • @berkaybilir3051
      @berkaybilir3051 4 года назад

      I'm trying to do the same to download new mangas whenever they're available. How did you do it? I think its a bit tricky just using the request module.

    • @berkaybilir3051
      @berkaybilir3051 4 года назад

      I'm trying to do the same to download new mangas whenever they're available. How did you do it? I think its a bit tricky just using the request module.

    • @chaks2432
      @chaks2432 4 года назад

      @@berkaybilir3051 Here's the link to my Github where I uploaded the script. I used the split() method to separate the url at the end, where the page number and image type is, then used a for loop to iterate through the pages, saving each page as a png. github.com/EnricoCecchini/comicDownloader Hope it helps

    • @chaks2432
      @chaks2432 4 года назад

      @@berkaybilir3051 Just finished a new version of the script using Selenium, if you wanna check it out. It should work with any website with little modification, just add the XPath of the butto to go to the next page and the image Tag from the HTML code. github.com/EnricoCecchini/ComicDownloader2

  • @call-me-poor9313
    @call-me-poor9313 4 года назад

    This Video is so helpful. I can not thank you enough

  • @parthparadkar1021
    @parthparadkar1021 6 лет назад +1

    Thank you so much for this video!! Happy to have learnt something new today. Keep 'em coming! :D

  • @karthikl4692
    @karthikl4692 6 лет назад +2

    Thanks Corey right video at right time

  • @jenmanzella1948
    @jenmanzella1948 5 лет назад

    Your tutorials are FANTASTIC! Thank you

  • @mouradbarakat123
    @mouradbarakat123 6 лет назад +1

    This video is just in time for me! Thank You!

  • @lindaduong289
    @lindaduong289 2 года назад

    Best tutorial for requests

  • @daegudude1048
    @daegudude1048 6 лет назад +2

    Like and thumbs up before watching it!!👍👍

  • @doug_silva
    @doug_silva 4 года назад

    Can I double like this video? It's so awesome!

  • @mariamhasan3733
    @mariamhasan3733 Год назад

    Excellent explanation!

  • @j.n.y790
    @j.n.y790 4 года назад

    FANTASTIC TUTORTIAL! well done

  • @mr.RAND5584
    @mr.RAND5584 4 года назад

    Amazing i can request in my webhost-php. So easy to learn comparing to java httpurlconnection.

  • @tolulopeayemobola1446
    @tolulopeayemobola1446 3 года назад

    Thank you very much for this video. You gave me some confidence

  • @Александр-р3э3м
    @Александр-р3э3м 5 лет назад +1

    Thank you so much, Corey! I like your videos a lot!

  • @stonecoldcold2941
    @stonecoldcold2941 4 года назад +1

    Awesome Tutorial!

  • @collin-lj7ho
    @collin-lj7ho 9 месяцев назад

    I’m totally doing legal things with these !

  • @vijaysheri7390
    @vijaysheri7390 Год назад

    Awesome stuff, thanks. Can you please do a video on logging for just requests ?

  • @talhaamir9023
    @talhaamir9023 5 лет назад +1

    great buddy!! keep it up!!
    this was highly knowledgeable.:)

  • @tymothylim6550
    @tymothylim6550 3 года назад

    Thank you very much for this video! It was very helpful and I learnt a lot :)

  • @waiitwhaat
    @waiitwhaat 2 года назад

    Amazing tutorial. Kudos for making it. It was really awesome.

  • @eduardovillasmil832
    @eduardovillasmil832 4 года назад

    Excellent video go directly to the point

  • @amitbajpai6265
    @amitbajpai6265 6 лет назад +10

    I been waiting for this long long time ago thank you Corey

  • @alan-j
    @alan-j 5 лет назад +2

    @corey This is an awesome video on requests.. If anybody asks me what is the best place to learn python, i will directly point them to your channel.
    Can you also cover some indepth topics of this "requests" module?

  • @kianushghahari3038
    @kianushghahari3038 Год назад

    Hi, thank you for the thorough tutorials. As a beginner, I really appreciate your work very much. Did you ever post the the videos you talked about here as your real world example of a script for your personal website? because I wasnt able to find it...