Can I Make A Search Engine From Scratch?
HTML-код
- Опубликовано: 27 июн 2020
- I set out to make my own search engine. Yes there are already options like Google, DuckDuckGo, and Bing. But creating my own helps me better understand how they work, and I can make it function however I would like. I don't know if I will ever host this for the public to use. For now, it's just a project I'm working on. I would love to be able to implement complex queries and word associations, and maybe someday include image search as well.
Наука
Interesting video! I learned a lot about how search engines work. Keep it up!
Thanks Seraphim!
Him:
Hey google, how do you sell a child?
Google:
We have a Wikpedia article for that!
4:18
People also ask: How much is a kid worth on the black market?
@@Phiwipuss no just no
@@GamingFemboy I got it from the video. -_-
@@luke-bookbear ?
Yeah that was a weird example for the creator to use!?
Awesome start! I am excited to see where you go with this!
6:05 another fact about your search engine is that it will probably only target english pages and not the whole world's languages
This was amazing. Gotta get to work. Thx for the great content.
I watched the video and thought you had 456k subs because it was so good 😅 you earned a sub with notifications.
Thanks mrbushy!!
in the first 1 minute, i already am enjoying this video
Thanks Pewdiepie
Lol you are so funny i wish i was as funny as you🙄🙄🙄🙄🙄
Ur like really not funny
I'm working on something similar in machine learning. I'll let you know when i am through! Glad to see you workin do hard on it. You can do it!!!
That's probably the only way to compete these days. It would take so much time and manpower to build something like Google, but if you can incorporate machine learning into it, might bypass a lot of the stuff they had to do the slow way
The lad is pretty adorable, I must say; an interesting content.
Just subscribed, great content!
Try word embedding. It basically changes strings to vectors and index said vectors. When you search you have to vectorize the search term and compare it with saved indexes. It should be faster than using strings to compare
Hi i have some questions
Better yet one can use proper database (like MySQL or SQL Server) to increase the search speed.
If there's one table website URL and full text and other 2-3 tables with the most frequent words, bigrams, and trigrams per URL (ordered by frequency), that will be much more efficient than always relying on full-text seach, and it would also improve relevance.
Great video with excellent music choice!
Thanks!
Great project!
Are you aware of nltk package? It is capable of removing stop-words, stemming, word and word collocation frequencies, and so on. I believe that could help greatly with indexing.
Good point! I’ve used NLTK a little, but wasn’t confident enough with it to apply it to this project. Maybe it’s time for me to dig into it
@@_equalowhat happened to your channel?
Amazing video and also very funny lmao
I want to know further about indexing. If possible, make next part of this search engine video.
Thanks for this interesting video, I've learnt about how search engine works after passing my exam
thanks bro, I Learned a lot today by practical way.
00:29 had me crack up, haha :-D
The second I saw python I stopped the video to rethink my life
Subscribed!
Oh hey! I decided to make my own search engine this year back in 2020 too. Now is the time I'll be doing this project. I'll be sharing my progress on my channel.
did you start yet?
How about using GTP-style "tokens" to encode both page contents and search keywords?
Good work bro.try to use hashing of words in a web page and store them in a hash table.I think it might increase it.search hashes instead of word to word
That’s a smart option! I’ll try to do a follow up video trying that and a couple other methods
Very interesting project from Michael Falk!
Good video mate
Equalo you should make a part 2 to this video!!
so, im guessing this project did not get a follow up video?
To create a search I'm a coumputer science student beginner .there are lot of fields in computer science major, in all of these which field do I specifically need to take to develope a search engine? please
I'm also interested. I'm looking into building a search engine for a group of less than 200 related websites only. This is an insightful video. Hopefully you you post the code. Thanks
Did you create your own search engine?
Was the number of links you parsed from the second batch of pages referring to the total links or the amount of unique links? It's pretty easy to think of an example of a set of Wikipedia pages that all link back to one article, or even two or more pages that all connect to each other and create endless loops.
Of course, looking for changes in website contents is necessary, but avoiding crawling over the same pages too often should speed up the rate of expanding the database and reduce the strain on the hardware in the long term.
you would have a process. search, in document, duplications, but you would still Identify if the are in deed duplications, and not just similar, and even if it was real duplications, it was still linking for a reason, unless it just the home button or something like that?
I'm ready to help.
Nice video 👌
WHATS THE LEVEL OF YOUR SERCHENGINE NOW
What coding tool do you use to create the search engine?
question: The search of pages can be done even with ip addresses, cycling every possibile number? Or only with links? Using links means you must have a list.
Good content
could you send your source code for the search engine
Yeah, I was trying to do this by myself. A link to the code would be great!
Nice
Thanks Adam!
Your videos are very informative, I just learned how to sell my child! :P (@4:18)
Haha I’m always happy to help. Glad you found that part useful
@@_equalo 🤣🤣🤣
How’s the project going now?
Its dead
But won't this method be too much time consuming?is there any other way to make it faster or automate it bro
9:10 lmfao that gif
More interested in the coding aspect you share source code or how to get started
2:57
when u open OBS for the first time:
👍
5:10 it wouldn't be a bad idea to chuck the word recursion in there and state that that's what your crawling and parsing is doing
Hey, how is your search engine going?
So where is your search engine? No link?
4:19 Did anyone see what he typed in the Search !! 🤣🤣🤣
Lol..🤣🤣🤣
"How to sell your child"🤣🤣🤣
Is this on github or you keeping the code private?
Can you give us the tutorial to make that search engine, sir?
Like it
Hope the project progressed beyond python scripts.
Your choice of wiki topic to search????
LOL, 7.35 She gushed over him. Sorry about that. Very interesting, thanks for the vid.
Please upload for me the download link of your search engine. Thanks
link?
I searched for it after whatspp's new january privacy up-to-date 😃😃
Why aren't you making another video?
I think this is the guy who has a channel named "what I've learned"
Bro go on title and headline base because then only you can get the information
5:27 so now you start indexing. keywords in the websites links to websites
What about Bing Users?
I have duck duck go! 🥳
From scratch just bothers me for some reason
Wait a minute
Hello r u still working on this
Any news on this?
try arango database for inverted index searching
duckduckgo is getting popular
can you share the code
7:45
is'nt scott morison the priminister
good try!
How can I contact you
PewDiePie's Got A Search Engine😀
I laughed a lot... watching your video
duck gang
i was planning to build something similar but ain't gonna happen 😂
"how to sell your child"
I have an idea do you want listen?
Yup
420th like
please share update.
petabytes=too expensive
Use folder and use letter to me the floder
😎My friends says -
You can't compete...
👩My mother says -
Never give up...
😇"I follow what my mother says"
Update please
Watch at 1.5x speed
"mom can we have google"
"we have google at home"
google at home
Did you make it, i want to make it and make it public and add free and zero tracking of people good gearh engine add free bias free...;)
Check out safenetwork.tech there are a few people pondering how to implement a search engine and you may enjoy the challenge.
Google no longer operates on the number of links that link back to it, but rather PRIORITIZATION
Can you email me really interested.
I'd prefer wikipedia api
You didn't even paritially cover indexing in this video
Vedio is just a look
Indexing is a big process
Want to know
We did something different
Check ✔️ Elliot Search
how
Am I the only one who uses duckduckgo
Yes.
Can't imagine someone not using Google😃
chuper
Are you on onlyfans?
Opera is better than Google