Scrape Sensor in Home Assistant - Extract Info From Websites
HTML-код
- Опубликовано: 11 сен 2024
- I’ll walk you through extracting information from websites, such as key dates, or facts, and importing them into Home Assistant using a scrape sensor and css selectors.
Code: github.com/wil...
CSS Selector Tester: chrome.google....
CSS Selectors: www.w3schools....
As an Amazon Associate I earn from qualifying purchases. By buying through any of the links below I will earn commission at no extra cost to you.
The smart home tech I have is here: kit.co/WillSur...
▼ My HA setup
Raspberry Pi 3B+: amzn.to/3gF588j
64GB San-Disk: amzn.to/3iDUErN
▼ Amazon Products
Amazon Music Unlimited: www.amazon.co....
Amazon Prime - 30 day free trial: www.amazon.co.u...
Audible Free trial: www.amazon.co....
Kindle Unlimited free trial: www.amazon.co....
Amazon Business: www.amazon.co....
▼ Camera Gear
Canon M50: amzn.to/3iGsGeU
Manfrotto Element Tripod: amzn.to/3e7FYxF
Boya Mic: amzn.to/3fez8Ic
64GB San-Disk: amzn.to/2Z78vPC
▼ DJI Products
Buy a Mavic Air: click.dji.com/...
Buy a Mavic Mini: click.dji.com/...
Buy an Osmo Mobile: click.dji.com/...
▼ Social Media
Remember to subscribe: / @willsurridgetech
Instagram: / willsurridgetech
Facebook: www. Wil...
Will, love what your doing and your videos always entertain me, you are really animated and upbeat whilst explaining things clearly. Your presentations always make me smile as you come across like a really excited puppy, full of excitement, wanting to share info and help us all along. Keep it up I love it, it’s so refreshing!
haha, thank you very much!
For some reason my sensors for RUclips via Socialblade still shows unknown after adding the User-Agent
The log reads "2021-04-25 10:34:26 ERROR (MainThread) [homeassistant.components.scrape.sensor] Unable to extract data from HTML for Forschi RUclips Views"
Wow! Your channel is amazing. Thanks man!
Definetly a follow worth
Awesome, thank you!
Hi Will, now I found out a good few usages for HA. Thanks a lot!
Happy to help!
Annoyingly my council's bin collection site uses the same ID for two types of collections on the same page and no way to differentiate :( ugh!
Great content and very well explained!!! thanks again Will for an awesome job to HA community... I appreciate that!!!
Thank you, glad you like it!
css selector tester seems to be gone now. any other suggestions?
I wish it had two extra functionalities:
- scrape multiple data points from a single pull
- set how often the data is pulled. (e.g.: zero reason to pull info every 30s that's only refreshed every week)
Also, you can just r-click in the inspect "elements" on whatever you want and go copy -> copy selector
you can install multiscrub from hacs. Then you can selec freqency, and yes multiple data points.
Hi will, clear and well explained tutorial as always! Thank you for the great resources you are uploading about home assistant!!
My pleasure! Glad it helps :)
CSS selector tester is no longer available: pluginsaddonsextensions.com/google-chrome/css-selector-tester-chrome-extension_23245
I'm trying to scrape the weather alerts from the NWS alert page. Alerts show up as an #id with an h3. If there are multiple alerts there are multiple h3. Is there a way to scrape multiple ids?
resource: url
select: "#content h3"
The Link for "CSS Selector Tester" is faulty.
How to add attributes to your scrape-sensor?
I have a scrape-sensor which gets the water-temperature from a lido. So I want the coordinates as attributes.
like:
latitude: 47.3519
longitude: 8.5360
If possible I would even like to have an attribute with the website-link where I scraped the information.
Are there some informations about how I can create such attributes or how they have to be named ("link:", "source:", "web:", ...)?
I was just looking that!! You are the best!!!
Glad I could help!
thnaks a lot
Great video. I’m a little stuck. If I was looking for the average review or number of reviews of a product on Amazon, what CSS would you be using? I’m trying but getting nowhere. I’m wondering if it’s because Amazon is blocking (even with the Mozilla 5.0 header) of a, getting the CSS selection wrong....
How do you set the interval between scrapes ?
Nevermind. The answer can be: Use Multiscrape from HACS instead and set scan_interval. Also it lets you scrape multiple information in one go. Maybe a new/updated video?
Will, yet another great video. I really appreciate you covering different subjects to other RUclipsrs. Keep up the good work.
Thank you! More to come!
How do you determine the polling interval in HA?
I have my data in cloud and by doing http get request I can get that sensor values is it possible to create a sensor with http get request if yes please share the tutorial this could help me lot waiting for your video or any response
I tried to use this to make a credit karma credit score sensor, but couldn't figure it out. thanks for the tutorial. I don't understand selectors or maybe it is not possible with that website.
Its probably not possible I'm afraid - scrape sensors are very basic
Thank you
No worries
thanks for this good vid. what is frequency sensor is fetching values?
scan_interval: 3600
www.home-assistant.io/integrations/scrape/#bom-weather
Yeh the scan interval is how often (in seconds) it will scrape the website for an update
Could you make video about Scrape sensor with login field ?
I don't believe that's possible because of the limitations of the sensor
Could have used the RUclips API instead.
This is true, but then I wouldn't have had that example!
The over-the-top speech is completely unnecessary.