EXCLUSIVE: Google Gemini Pro & Flash 1.5 TESTED!
HTML-код
- Опубликовано: 17 май 2024
- I got early access to Gemini Pro 1.5--Google's professional model--and Flash 1.5--their lightweight high speed model--and torture tested them. How do they stack up against OpenAI's GPT-4o--and each other? The results are VERY surprising!
**If you are looking to purchase a new Tesla Car, Solar roof, Solar tiles or PowerWall, just click this link to get up to $500 off! www.tesla.com/referral/john11286. Thank you!
Join this channel to get access to perks:
/ @drknowitallknows
**To become part of our Patreon team, help support the channel, and get awesome perks, check out our Patreon site here: / drknowitallknows . Thanks for your support!
Get The Elon Musk Mission (I've got two chapters in it) here:
Paperback: amzn.to/3TQXV9g
Kindle: amzn.to/3U7f7Hr!
**Want some awesome Dr. Know-it-all merch, including the YEAR OF EMBODIED AI Shirt? Check out our awesome Merch store: drknowitall.itemorder.com/sale
For a limited time, use the code "Knows2021" to get 20% off your entire order!
**Check out Artimatic: www.artimatic.io
**You can help support this channel with one click! We have an Amazon Affiliate link in several countries. If you click the link for your country, anything you buy from Amazon in the next several hours gives us a small commission, and costs you nothing. Thank you!
* USA: amzn.to/39n5mPH
* Germany: amzn.to/2XbdxJi
* United Kingdom: amzn.to/3hGlzTR
* France: amzn.to/2KRAwXh
* Spain: amzn.to/3hJYYFV
**What do we use to shoot our videos?
-Sony alpha a7 III: amzn.to/3czV2XJ
--and lens: amzn.to/3aujOqE
-Feelworld portable field monitor: amzn.to/38yf2ah
-Neewer compact desk tripod: amzn.to/3l8yrUk
-Glidegear teleprompter: amzn.to/3rJeFkP
-Neewer dimmable LED lights: amzn.to/3qAg3oF
-Rode Wireless Go II Lavalier microphones: amzn.to/3eC9jUZ
-Rode NT USB+ Studio Microphone: amzn.to/3U65Q3w
-Focusrite Scarlette 2i2 audio interface: amzn.to/3l8vqDu
-Studio soundproofing tiles: amzn.to/3rFUtQU
-Sony MDR-7506 Professional Headphones: amzn.to/2OoDdBd
-Apple M1 Max Studio: amzn.to/3GfxPYY
-Apple M1 MacBook Pro: amzn.to/3wPYV1D
-Docking Station for MacBook: amzn.to/3yIhc1S
-Philips Brilliance 4K Docking Monitor: amzn.to/3xwSKAb
-Sabrent 8TB SSD drive: amzn.to/3rhSxQM
-DJI Mavic Mini Drone: amzn.to/2OnHCEw
-GoPro Hero 9 Black action camera: amzn.to/3vgVMrH
-GoPro Max 360 camera: amzn.to/3nORGYk
-Tesla phone mount: amzn.to/3U92fl9
-Suction car mount for camera: amzn.to/3tcUfRK
-Extender Rod for car mount camera: amzn.to/3wHQXsw
**Here are a few products we've found really fun and/or useful:
-NeoCharge Dryer/EV charger splitter: amzn.to/39UcKWx
-Lift pucks for your Tesla: amzn.to/3vJF3iB
-Emergency tire fill and repair kit: amzn.to/3vMkL8d
-CO2 Monitor: amzn.to/3PsQRh2
-Camping mattress for your Tesla model S/3/X/Y: amzn.to/3m7ffef
**Music by Zenlee. Check out his amazing music on instagram -@zenlee_music
or RUclips - / @zenlee_music
Tesla Stock: TSLA
**EVANNEX
Check out the Evannex web site: evannex.com/
If you use my discount code, KnowsEVs, you get $10 off any order over $100!
**For business inquiries, please email me here: DrKnowItAllKnows@gmail.com
Twitter: / drknowitall16
Also on Twitter: @Tesla_UnPR: / tesla_un
Instagram: @drknowitallknows
**Want some outdoorsy videos? Check out Whole Nuts and Donuts: / @wholenutsanddonuts5741 
Развлечения
Good test and rigorous. Thank you
I see a big improvement in GPT4o. In translation exercises, where an English sentence has to be translated to Japanese using a list of Japanese characters, GPT4 would always complete the translation but would not use the provided characters, it would just translate in what it thought was the optimal way. No matter how it was prompted, it would always fail to construct an answer using only provided characters. I also have tried the same task with Gemini/Bard and Claude without success.
GPT4o gets it right every time.
The white text background just killed my eyes everything else is super 😎
I have checked with GPT4o, and i see that no any other AI can compete with that model. even no Gemini 1.5 pro , Claude and etc...
2:19
2 ducks in front of 1 duck.
2 ducks behind 1 duck.
1 duck in the middle.
Ducks have facing. Try it with bowling pins.
2 in the front vs 2 in the back vs 1 in the middle is dependent upon their facing.
Or if you assume facing doesn't matter, the orientation is up to you. But for those three statements to be simultaneously true - implying but 1 perspective.
Just out of interest about half way through after you had to keep reloading - according to the screen recording - you ended up on Using Gemini Flash in both tabs, rather than Gemini Pro in one and Gemini Flash in the other. Did you notice? (The speed of "pro" was significantly faster after that happened, which was another clue that the model was now Flash and not Pro)
You just identified a function AI needs to be able to perform. Essentially, it needs to be able to control-F. Maybe a grep function too (compare two files and flag any differences). I'd expect eventually the AI would write and execute the necessary function, but not in such a short amount of time. Maybe?
CTRL-F is basically what the attention mechanism in transformers does.
A few observations:
You left the novel in the context so the math problems were far slower than they needed to be.
After the math questions, the model appears to have been on Flash for the Pro tests.
According to the GUI, Flash has a 1M context window so it could do the novel test as well.
Great tests though!
My old PC is having a hard time handling massive web browser windows, as all the whole conversation is kept alive. When you get up to 50k tokens the slowdown is very noticable, especially on older systems. But Gemini's a true wizard with python with very little mistakes, 9 out 10 times it produces executable code. It's just a question of the right prompting.
I am curious : I agree with you. And I tested both on coding.. gpt4 is way better... Are google aware of this ??
My coding results with Gemini are extremely good, with the correct prompt. At least, in my experience. What in your experience makes GPT-4 better?
@@paulmichaelfreedman8334 everything.. try something simple. Like I did
Ask it write a c++ function which takes an array of ints and does not sort it. But still prints out the 3 smallest elements...
@@paulmichaelfreedman8334 try this :
Write me a c++ function that takes an array of ints, and without sorting it prints the smallest 3 elements
I have no idea why my reply gets deleted...
Try this with both :
Write me a c++ function that takes an array of ints and without sorting the array, prints the three smallest elements
what's better g 1.5 pro or flash or gpt 4o?????
I have a recommendation for a question.
"Cutting stock problem".
You will plan a series of cuts from common stock to produce as few waste pieces as possible.
You have 3 foot lengths of wood as common stock and must cut them into:
Six 2 foot sections.
Six 1 foot 6 inch sections.
Six 6 inch sections.
[I am working on a similar project and haven't seen this type of question.
This isn't quite right yet. I want to prodice a question that ends with 1.5 feet of scrap. If you design it right you get 1 piece of scrap instead of three.
I am hesitant to use examples from on line because they may be trained in...]
Every single AI I've tested has failed with this problem, even with multiple clues from me as to how to solve it. Claude even ended up giving up and admitting it couldn't solve it:
There are three vampires and three virgins. They need to cross a river safely. Their only means of doing so is a boat that only seats two, and there's no boatman.
The problem is, if at any point the number of vampires is greater than the number of virgins, they'll succumb to their base instincts and bite the virgins.
How can they all get across the river safely?
And another one they have a problem with:
I have to set up races for 25 runners. I need to award the gold medal to the fastest overall runner, silver to the second fastest, and bronze to the third fastest. How they rank below that doesn't matter.
The problem is, my track only has 5 lanes, so I can only race 5 runners at a time. How can I find the three fastest runners in the fewest number of races?
The actual answer is 7. They have a tendency to come up with 10 as the solution, often in a way that improperly eliminates the third fastest runner.
Google/RUclips search already ignores your query in favor of algorithmically favored results. I don’t know why anyone would choose to use an AI developed by Google.
Google is wayyyy behind OpenAI at this point!
So far behind that I lost interest after 20 minutes. Comparing these 2 models to GPT-4o is chalk and cheese.
Thanks for the work, you're doing a great job!
The winner will not be the most logical or precise.... once they reach a certain point. It will be how they can interact with humans to the point of being a friend and confidant.
Why do you think Apple is considering doing a deal with OpenAI and GPT4-o?? It's all about human interaction baby.
This I think you are leaving out the most important metrics... human interaction and personality.
i dont get why there couldnt be 5 or 7 or any odd number of ducks. the statement fits just as well as with 3 ducks. are you sure you have the right idea about what logic is? logic is about whether a statement is true or false. nobody said, the answer should be the lowest number of ducks possible that still satisfies the question.
These "logic" questions and answers are subjective and arbitrary for all of these AI modules. This is no different than the codex programming for search engines and data indexing from 40 years ago. The answers will depend solely on the information fed into it. The tokens, the input, the pathways, and the output are all based on the algorithms. None of it is right, wrong, true, or false. Its output can not be accepted as FACT and relied upon for critical advice or preservation of humankind. In other words, BS.
Ask stupid questions, get stupid answers I guess