How to Scrape SofaScore for Football Data
HTML-код
- Опубликовано: 8 сен 2024
- Github with code: github.com/mck...
Get Free Sports Data Forever by Building Your Own Web Scraping Pipeline: mckay-s-site.t...
Subscribe to my newsletter talking about data + sports : mckayjohns.sub...
Follow me on Twitter: / mckayjohns
Join the Football Analytics Discord: / discord
Awesome video. Is there a possibility for a part 2 where you show how to recreate the shot map in Python?
Yea would like to see that
I am very happy Johns about your vedeos you are doing for us , to learn more about how yo scrape Football Data from websites , but my question is , how we can organize the Data as Table not like that ??
Hello,, this was pretty helpful. I've been trying to get some data from SS using selenium, but every now and then something changes on the pages,, have to keep changing paths and selectors,, just pure pain,,, this is much pretty neat and easy to maintain,, thanks
Hello McKay Johns, great content. I have a question, is it possible to scrape the Match Momentum of the match as well? Thank you very much!
Thank you McKay, that was quick and easy to follow! Appreciate the great content
Glad it was helpful!
Can you do a detailed video to scrape sofascore's historical score data, i currently have to scroll manually to scrape it. Thank you.
Hi, thank you for your video, I recently saw that sofascore integrated "honeytoken" mechanism into their apis, which means that they randomly send you fake data instead of blocking you. Do you know how to solve this?
Yeah currently working on a updated video. Probably out sometime next week
Really useful video. thanks for sharing!
Thank you so much for this great video!! Would be Very Helpful❤🙏
Hi, is it possible to extract all the Serie A players and the statistics for each player?
Are there too many requests and does it block?
In the video I see that you take the URL from Fetch but can it be done automatically to take the XG of each player for example? Thank you.
I do all the steps but get 403 response. Has sofascore closed access or i am doing something wrong
Hm the only people I know that have gotten a 403 have been using google collab is that your case?
@@McKayJohns Same problem and I use a jupyter notebook. Please can you give an up dated tutorial on how to bypass the 403 on sofascore? Thanks in advance.
Hi! Great video!
However, I have a question: wouldn't it be easier to use the SofaScore API?
thank you very much for this video!
New backdrop is clean
The pikachu lights up
This really helps me.
Oh god where on earth did you get all this from..amazing work. Have one question. Do you work as an analyst anywhere in the industry?
Hahah I work as a data engineer for a startup so no I don’t work in sports just have been doing it as a hobby and to learn new skills for a while now
Just by chance, I saw you in your videos explaining a program between two teams, Fulham against Liverpool, for more than two years... and today, you also explain another program. Can you explain more... Can it help me to have good expectations?I play satellites.... especially football
Hey bro I have an issue when I’m trying to scrape data for statistics it’s getting restricted to one league can you help with me how I can tackle this issue and make it so I can scrape statistics of players from all leagues
this only work for live events? the external api requests
Hey is it possible to scrape the webdata to google sheets? I want to keep track of the player stats thats provided by SofaScore
Yeah you can either save them manually, or if you want to play around with googles sheets api then you could insert it directly
Hello, my friend. I Nedd a Help
With the shotmap endpoint, for each request I make, the api returns different data to me even if the game has already finished. Has this ever happened?
i'm finding a hard time plotting the shots on target locations, i guess it's the 'draw' column that gives the coordinates but don't know what goal dimension they have set. do you have any ideas ?
Look man is it possible to figure something out where I can find out stats for team in the last 5.games I can't find websites that do this
Great video thanks a lot!!!
Would it be possible to scrape a player's shotmap for an entire season
Yes totally possible
Anyway to turn the shot locations to a Data Frame?
you should be able to do pd.DataFrame(shots) and it will load
It works but it is still in a weird format any idea on how to get proper coloumns and rows.@@McKayJohns
@@ryanfeasey3725 you might have to unpack each individual row. I would look into maybe something like this: stackoverflow.com/questions/61847791/how-to-explode-nested-json-in-pandas-as-rows or try and ask chat gpt haha
@@McKayJohnsthanks
Amazing as usual!
I want to ask about the previous program that I explained more than two years ago, which I did, I think between Fulham against Liverpool, how can it help me get good expectations? Is this possible?
Hi, Just wondering, which do you feel is the best website to scrape defensive actions?
probably look at sofascore to see if it has anything or maybe fbref. If not try whoscored
Hey Mckay, is there anyway I can put on together all the data from where a player shot and his mean throughout an entire season? Was thinking as well if there's any way for me to enter in contact with you to see if you can do one for me and how much you would charge for you to do so, thanks a lot in advance!
Hey mckay, thanks for doing these tutorials. Just one question : how do i export it to a CSV?
if you load it into a pandas dataframe, and your dataframe variable is called df
you can do
df.to_csv('location you want to export.csv')
@@McKayJohns Thanks man
Great video !
Did you know how to scrape MLB results with all innings scores from Sofascore ?
You should be able to. I've never done it but process should be roughly the same
I tried to do it without succeded... could you do a quick video please ?
Good tutorial as always, any chance you can do one on Zones of control? Thanks
I can take a look 👍
@@McKayJohns Thanks!
If you see this, is it possible to convert the JSON into CSV and if so how do you do it@@McKayJohns
@@TLFootball123 Ya you can put the shots in a pandas dataframe... pd.DataFrame(shots) like that then export it as a csv if you wish
Hi !!
Thank you for the video! I've been using this for a while now, and I really find it excellent.
Since yesterday, I've encountered an issue: an HTTP 503 Service Unavailable error appears. Have you experienced the same issue? Do you know if this is temporary or if they have implemented a block on the API calls?
Thank you in advance for your help.
Yes I noticed it today. I need to remake this video with the new method I founs
@@McKayJohns That would be great! Thank you in advance.
Do you have to write the headers if you get a 200 code on the response the first time you try it
Technically no, headers just help for authenticity of the request
I have the intuition that if i wanted to scrape every match of the PL I'd have to create a list of the 380 matches, but I'm sure there's an easier way. Same with say, ManUtd. Sofascore doesn't make it very easy to scale I feel
Yeah it’s tough but you should be able to programmatically make a list of match urls
Very nice!
Hey my friend, can you create a football predictions script for me?
After yesterday’s 503 errors, the API queries are working again today. I’ve been making some attempts to consume various APIs as you explain in the video, and I’ve noticed that the returned data is inconsistent, even without changing anything in the headers or the query URL. The data changes, and logically, these are queries for data that isn’t changing. For example, when I check the league statistics, one query returns 567 accumulated goals, and the next one returns 1059 accumulated goals. It’s as if it’s providing fake information. Is this a defense mechanism of Sofascore? Have you had the chance to verify the reliability of the returned data?
Yeah I noticed this and am remaking the video to show how to get around it
I'm having the same issue, using this to get info on a team's next match - for each request (using the API), the parsed data shows a completely different timestamp pertaining to the start time, consistenly inaccurate
hey Mckay, can we code using visual studio as well and not stick to jupiter
Ya I prefer Jupyter for tutorials just cause it’s easy to use but you can use whatever
Hello bro, can you scrape data between date range? like from 2023-02-01 to 2022-08-01
Hello bro
You're the man
@McKayJohns, do you use playerCoordinates or draw coordinates for the shotmap? I followed your tutorial for shotmaps using data scraped by following this video but it's not aligned, goalscorer is somewhere in the middle of the pitch :)
I have this thing
Make a tutorial on how to get trough a chat ban on sofascore please
chat ban? what do you mean?
@@timokorpershoek1893 my messages don't show up in the chat
is there a option for R?
I don't have the code for R, but the concept is the same
Hello, I would like to ask, is it possible, is it possible, is it possible, for example Hello, I would like to ask, can such a program help me, such as the way you choose the results of football matches, especially since I play or any program you talked about I play in the field of sports, note, frankly, can you help me what you play in it and how it is
What’s wrong with you?
response.status_code is 403
While I am using the latest version of Google Chrome
and i followed the instructions exactly :(
Hm what does your code look like
@@McKayJohns hi bro First, I want to thank you for your great content,, I was using DataLore, but when I used jupyter It worked well, but I haven't finished the video yet, and I still lived your exciting experience, But I want to send you an email to inquire about something. Do you have time for that?
Hi Mckay, first of all I've really learnt a lot from your videos thanks for all of your informative videos. I do post Football Data Analytical & tactical videos on my channel really would like to have a chat with you so that I can tell you more about my content, if possible
I'm from India so if anyone is planning to visit my channel there will be language barrier 😅