Learn Python Through Public Data Hacking
HTML-код
- Опубликовано: 2 апр 2013
- David Beazley
What's more fun than learning Python? Learning Python by hacking on public data! In this tutorial, you'll learn Python basics by reading files, scraping the web, building data structures, and analyzing real world data. By the end, you w
I've never programmed anything in Python, but half an hour into this video, after trying to find the solution on my own, and coming up with a Google Maps image showing me all the northbound 22 busses... i'm very impressed.
I used to hate Python because of the indentation and dependency management It's growing on me lately though. My next small project might be in Python.
Thanks much for putting this informative seminar on the web. One night, I randomly came across this video, followed along & it excited me so much, I'm now studying as a web developer. These type of resources are valuable to beginners.
They should be doing a similar tutorial for python 3. Still the best I've seen on youtube so far.
0:30 I'm laughing my ass off because of that hand that just shot up in the air.
Same here
+Cammar Glass If I was there that hand would def me mine.
+Cammar Glass SPIRIT FINGERS hahah
This is excellent. I always learn so much more by actually being asked to do what the trainer is doing and I learn so much along the way. I installed python 2 days ago (never using it before) and now am tracking the Chicago transit system with it, exploring my own ideas along the way...!
This is one of the best overview of Python I've ever seen. You will get an idea about data structures in Python, Ipython (notebook), pandas, collections and many more. Thanks...
Have never programmed in python very much but I'm just learning Julia. Going through this video and trying to figure out how to do it all in Julia is very interesting. Julia is very similar to Python so it's not that hard to figure out what's going on. Very good tutorial. A lot more interesting than most of the programming tutorials out there. Thanks.
I really like these type of videos. Hats off to those who shared this!
"Don't be afraid of trying things out!"
"Sure! What if i put this there? :D "
*Entire city network goes offline*
"Oops.... :( "
this will work pastebin / YJg7Bzut
What a great intro to Python! Thank you for presenting such a great class!
Interesting way of teaching. Better than going through the grind of spending lot of time on basics. example based approach is much more fun
There is no space between print and parentheses
print('Hello, World')
@@mopeyman420 also no need for the parentheses
@@ryanz8775 python 3x needs the parentheses. 2x doesnt.
Ryan z depends what python
Wow, straight to the point tutorial. You have increased me Python confidence a lot.
Superb! I've learned so much with this. More please.
Ok, great intro. I'm ready to enrol on your next course!
For the people stuck at the Suitcase excercise, note that it is: Buses with 1 'S', besides that: check if you spelled .JSP and not JPS. The people that say that the code is not working anymore are not right, as of 11-24-2014 it is still working.
+Steven Sijperda thx
url lib fix for python 3+
# import urllib
# from urllib.request import urlopen
# u = urlopen (' .........')
** 3.5.1
#from urllib.request import urlopen
#u = urlopen('........)
from urllib.request import urlopen
u = urlopen('ctabustracker.com/bustime/map/getBusesForRoute.jsp?route=22')
data=u.read()
f=open('rt22.xml','wb')
f.write(data)
f.close
#my code looks likes this but when i run it shows ==== No Subprocess ====
oh mb i actually get xml file didnt relised
9 mins, and am loving this already! ;)
He is possibly the BEST Teacher in Python or Computer Science in General..
I like that old hand at 0:30 .... Never old to learn a new language ❤️
This was an Excellent presentation. Thank You!
is there any more of these? OMG this are awesome!
Perfect! Thanks for sharing... Feels like I am too in classroom... :)
that's reaaaaly nice video buddy, very usefull, fun to watch and interactiv, thanks a bunch !
I'm new to Python, about 3/4 the way through Coursera's "Intro to Programming" (using Python) and this...though I don't fully understand the syntax construction - this was very understandable AND awesome. I was able to find a different route for the bus route and then the pothole thing was very familiar in the IDLE.
I feel I'd need a few more intermediate classes and then perhaps...well, how to do I become employable with Python? I love the syntax, as I'm also a Data Analyst - SQL Server/T-SQL, but need to know how exactly to put Python to use versus a 'electronic pen and paper' with more data points type of scenario. Or something 'more' than what SQL Server is, as I could have easily downloaded the 2nd data set, imported into SQL Server and voila. Figured out a bunch of stuff. Perhaps needing some stuff for mileage info. But you get the point. I want to use Python more...and I want to learn more...where this type of thinking/problem solving scenario is common place. But where one can actually get a job doing it.
Any help is greatly appreciated.
Hello Sir, I did it with maps as well. Loved it frm bangalore India. I feel like urging our Gov to provide APIs for Trains & City Buses on similar concept. Its very useful. Thank you
thanks for share, I love it and watch over. learned more~
Most useful Python tutorial Ever
Thank God I've found this video again🙏
This is so cool. I have spent 2 days looking at videos on Python im a total noob. But im starting to understand how it works so soon i will try and mess around my self. I find it very interesting just to listen on what can be done
yes also reading up on some python books id pdf which yiu can finf plenty for free, it gives a great starting base for the language
My school gave me a book about basic stuff with python. Just so I get a basic understanding before I start my GCSE
Keep at it! I'm hoping to hack Facebook once I'm comfortable with Python.
I'm reading a python book by Richard L. Halterman is anybody's interested. So far I really like it, it gives the basic of a lot of topics. Free pdf
Spoder Man What is the link for the pdf?
Between single quotes and double quotes there is a difference. The different is of style. For example, type single quotes when using a single 'word' while double quotes for a sentence "Phrase"
03/19/2018 Still working. Awesome
Thanks.
github.com/enisozgen/PublicDataHacking
to download potholes.csv
This was a fun tutorial. Finally, something beyond the de facto standard 'Hello,World' program. I would like to know how to finish the first solution? When the bus is .5 miles away from the office, how can you get the application to show the location of the bus a map? Thanks!!
if you = skrub
print ("get shrekt m8")
==
@@scottvelez3154
you = ("skrub")
if you == "skrub":
print ("get shrekt m8")
As someone new to programming, I think the biggest problem with python is the library or module system. The minute I need code that requires a module I'm done. There isn't a single module I can install with PIP without getting an error such as 'no matching distribution found for urllib' or being told I need to upgrade pip, upgrading pip and then being told I need to upgrade pip the next time I start a new session.
Outstanding thanks perfect lecture learned alot!!!
Any possible way you guys could re-do this video in 720HD?
Great tutorial David!
Very Good!
Wonderful!
I love it...
(From Brazil).
Brazil love Python.
Awesome tutorial David thanks so much.
Can you use the google docs code editor with python?
Everybody probably thought the case belonged to the bus driver.
Very informative video tutorial, Thanks a lot
That's quite an interesting intro to Python (and data analytics)
This is fantastically fun.
Do you have the other files? Where can I learn more in depth info on the parsing stuff?
Thank you so much! This was very helpful! One question though: whenever I put "webbrowser.open('www.bing.com') it opens in internet explorer. How do I open it in chrome?
is it possible to show all files + directories on a ftp server without passwort (only viewing not writing). if i write anything wrong i only say that im from Germany
Good video!
Lots of interesting applications for this!
thx a lot for your professionalism!
I did the first bus code exactly as instructed, however, when I finished writing it out, nothing happened. Am I not supposed to be seeing some HTML?
so how did he find the cta link? browsing around on the site it isn't very apparent. this would also be useful to know for other websites how to gather information
Very nice and practical. thanks
Thank you for great lection!
What are you doing with the bash commands at minute 38:16?
It would be great if you would upload the examples converted to Python 3.5
Great tutorial!
I just shifted from LINUX to MAC, man you all are right , LINUX ROCKS and PYTHON ON THE ROCKS
You CAN declare the value type for raw input commands, which is one thing he forgot to mention. 8:39
This guy is great.
"Theres a usb drive going around with the files on it"
... At a conference about hacking :)
lol
Lol when I heard that, the hair on the back up my neck stood up.
Files that come from a Windows XP installation.
It's not 'hacking' in the infosec sense . It is 'hacking' in the
gnu/linux , richard stallman , linus torvalds etc ie doing knew and
clever things with computers , basically what real programmers consider
'hacking' . See hacker definition 1
searchsecurity.techtarget.com/definition/hacker
that was funny!!!! ^^^ lmfao
mistakes were made
Using 3.8 and cant make 18:50 to work. Is it type error on my behalf, or mismatch with version?
superb video...
lots of fun, thanks!
What should I do if the code dose not open your Browser?
can this be done today using Python 3.6 or do I need an API key ?
I'm following along with this video with the missing suit case but my code is not working
Damn good tutorial!
This guy is refreshingly cool!
You are right. It was kind of depressing to see that all the graphical stuff is almost impossible to accomplish with libraries and bindings which are compatible with Python 3. There is one which should work (do not remember the name yet), but it required lots of manual installations of C and/or C++ libraries. I like Python, however, as long as I can do graphs and graphics more efficiently in Java or R, it's not an option for many applications.
It's true. Mac doesn't lock down any of their bootloaders, and even advertise you can run Windows on them, and Linux can be run on them as well.
I got the following error message when I executed the script for "The Traveling Suitcase".
f = open('rt22.xml', 'wb')
IOerror: [errno 13] Permission denied: 'rt22.xml'
Url is no longer valid. Any idea where the new link is?
Hi the second line starting with U does not execute tried all the tips still does not work I live in the UK
Before this i just made (currently still working on) a custom "KeyLogger" you enter username and password like you would do for minecraft. it saves what is typed there and saved. then it exits out and opens the real minecraft to play i used Java,Python,and batch for it to work so far.
For slide 36; in Python 3.3.2 you'll need to use import urllib.request.urlopen instead of urllib.urlopen. On windows at least :)
I am running this through python shell on windows
and
the shell is not displaying anything after i run the file?
it should download an xml file. check in your execution directory
Very fun exercise.
This video is great, but do any of you have any suggestions for learning python if you have no programming skills what so ever? Also where I'm just starting out should I choose 2.7 or 3.3 to learn on? Thank you for your suggestions.
thanks for the vid..very informative!
i wonder how many options went off the table before we got around to passing a single usb drive around a room.
bravery
Mac's OS is based off of Unix. I remember watching a really cool documentary about the birth of Linux. (proud Linux user)
Very very cool.
Superb
awesome video
15:49 there is no closing ' on urlopen
My First Python! wooo
print "Enter a file name:",
filename = raw_input()
reader = open(filename, 'r')
theLines = reader.readlines()
print(len(theLines), 'Lines in that text document.')
print "Would you like to read each line y/n?",
answer = raw_input()
if answer == "y":
reader.readlines()
print(theLines)
elif answer =="n":
print('Fine!')
RUclips 2020 recommendation, I see you can program now, wanna watch a video of someone giving an overview of a project.
This looks great but is there any chance of an updated version for python 3.2.5? Have you published one already?
which version is this guy using?
applezauc3 2.7.3
I think he meant it to be sarcastic. Learning programming gets more and more important for every human being.
XD The Travelling
❌ salesman
✔ suitcase
Problem
This is good, but I wish it was higher res.
Great tutorial, would like to have the transcript or subtitles in english at least.
this doesnt work in 2016 and in python 3.5?
does anyone know how to execute this on visual studio?
Cool!
Can I get mails from classified ads websites like craigslists with this technique?
Howard seems to be still driving in the same route. lol
Is the bus excersize legal before I use it??????????/////
When I try to parse rt22.xml it brings up a bunch of errors:
Traceback (most recent call last):
File "/Volumes/USB DISK/Python/modules/urllib/bustimesxmlparseandurllibwebbrowser.py", line 9, in
doc = parse('rt22.xml')
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1182, in parse
tree.parse(source, parser)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 657, in parse
self._root = parser.close()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1654, in close
self._raiseerror(v)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
raise err
ParseError: no element found: line 1, column 0
I resolved the error, but only if I exclude f.read()
len 114602 By the way, if anyone else runs into this when trying to follow this tutorial, the error is caused by the rt22.xml file being improper XML... Mine downloaded without the last closing tag and the closing tag. Add those to the end of the rt22.xml file and you should be good.
where do I find xml file?
Which version of python am i supposed to be using to follow along? I currently am using 2.7 because of codeacademy
Even i assume that you already know:
2.7 on this video.