Hot take: Code-writing AI is hyped up so much because companies needed an excuse to lay off many people, and then hire them again later as independent contractors with far less pay and benefits...
Replacing coders with AI is like replacing doctors with AI. Who really needs replaced with AI is upper management. Heard thinking, commonality, same old mistakes sounds like a perfect use case for AI.
been using Claude everyday for a side project. It's not the greatest at debugging issues. You usually have to hold its hand and tell them the exact few lines or functions to focus on otherwise it will overcomplicate things by default. What it's great for is generating some boilerplate or template for a new function or component. It saves tons of time from looking up documentation, googling issues on stackoverflow. You just have to read over the code it generates to see if it makes sense. If not suggest a different approach and it's generally pretty good at adapting since apparently you're never wrong...
AI is pretty good if you are using it to "query" stuff that is in documentation. in fact i often drop a link to node docs for an example then ask a questing around node that i know will be really well documented.
I think the biggest problem is the marketing angle where AI companies are saying that it will replace developers instead of saying that this is something that will make developers 2 times faster/eficcient for ex, thus reducing your development cost. In it's current state Claude AI is good enough to convince me to buy a subscription. Problem is that 20$/mo is not nearly enough to finance AI R&D, thus companies choose to overhype the AI so they can fool VCs into funding them with insane amounts of money.
I doubt it will make devs 2 times faster I have used gpt4 but it's garbage, it generates really bad code that will cause problems in the future and it doesn't understand how to fix it when it is integrated with kinda complex codebase. Sure it can write boilerplate code for already popular libraries but you can easily google that.
I use claude and AI very effectively. Like right now I am writing a Kotlin app, but I only know javascript. But you have to know what you are doing to get real functionality out of it. I read every line of code and I am starting to really get Kotlin. But it's all about the pseudocode. You have to think through what you want and communicate it very clearly and expect to have to hone your communication several times. All of that makes me a better developer. No AI cannot replace me, but it sure does help me.
@@patrickjreid yes AI can definitely help. But does is really help more than looking the info up in Wikipedia or Google Search? I doubt that a little bit.
@@bpscast For me AI helps a lot more than Google + StackOverflow or Wikipedia. AI allows me to be faster in getting the information and/or suggestions I want, and if the information/suggestions are wrong I can quickly figure that out in my dev environment and iterate from there. Now even though the traditional method of Google + StackOverflow has less hallucination tendencies (sometimes the info is still wrong/outdated), there are multiple times when I used non-AI code from someone's SO post or blog and it didn't exactly fit my scenario, so I had to iterate all the same anyway. In any case AI has already given me a huge productivity boost in my development, and the AI models themselves plus tooling around it are only going to improve. I've been trying out Cursor for the past week, and I can see my productivity improve even more with some of their features. E.g. I used their chat function with context to code a new feature for me, and I basically acted as a code reviewer. Even though Cursor didn't get it right the first time, after one iteration with my own feedback it managed to get it right. Definite time-saver.
Same here as an attorney. Claude saves me so much time. AI skeptics who say that AI isn’t helpful either 1) know very little about prompt engineering, or 2) know that they are lying for some ulterior motive
I have been experimenting with the aider project recently. It has a few benefits for me, but big issues too. The biggest benefit is that it changes the dynamic of programming from pure creation to creation&collaboration. It creates mediocre code pretty often (backed w/ Sonnet35 or gpt4o), but it gets close enough that I change in the editor and move on. I'm prompting it extremely granularly too. 'add a new arg here', 'write a test for this one method', etc. If you try to do too much at once, everything falls apart so quickly. Also I'm a senior dev who can nearly instantly read a chunk of code (in my app at least) and see things wrong. I can't imagine a junior or even most midlevel devs doing anything useful with this without hitting so many speedbumps.
agreed , aider workflow is good at small iterative additions and changes , id guess about 75% of the time it does well . the main failure cases are typically when trying to do big changes across a lot of code , and when it gets into a repeating cycle of alternating between 2 wrong answers .
I dont understand why people are trying to crowbar LLMs into reasoning tasks. If they took the time to understand how they work, they’d stop wasting their time on this, and only use LLMs for things they are good at, which language stuff, not reasoning!!
Takeaways: 1. The code gens can at most do imperfect boilerplate stuff. 2. Coder monkeys are f***ed, and 3. All others will have more work proofreading and testing AI code. Fair assumptions?
Models won't get a lot better, coding is just the wrong category of problems for Neural Networks, it's not hard to understand, programming is not a fuzzy problem, which is what they actually do well...
Models will get substantially better, lol. Don't kid yourself. There are already new focuses based on revamping AI reasoning for this specific reason. There are multiple papers by Anthropic, Microsoft, and OpenAI on this.
@@yds6268 The biggest way is to add new hardware that has higher perf/watt. Which is exactly what the new Blackwell Chips do, apart from that there will need to be significant changes in energy expenditure and/or management in the long term. Not something easy of course, but as AI is able to take on more tasks. That is an investment companies will be willing to do.
The worst part about A.I that I've found at the moment, is that the API documentation that I usually rely on to get my day-to-day job done has either been purged or has been put behind an auth wall \ is not getting updated, in favour of "use our new A.I tool". - and the new AI tool is producing mediocre code at best or the purpose of me referring to the API documentation is to understand what a certain function, constant value or module does, and just getting an example snippet from a command line doesn't help with that, so I'm finding that I have to dig through definitions in the source code and take my best guess. Another part is what I would once refer to as an algorithm or set of algorithms within a given package or program are now being blanketed as "A.I."
I had to use image magic and convert recently on a Linux box. Darned huge number of hoops to get the SE to let me run it! I should have just spun up a VM, done the needful and sftp the results to the real machine.
Please never stop. This is such an uphill battle on something where it is so obvious. How can the sane people be the minority? Artificial intelligence should be the term used to describe the people who think the LLMs work well.
@@rasi_rawss Yes, and false advertising is illegal. You can't sell a product knowing it doesn't do what you said it can do. Look up Theranos. This crap is exactly the same and people should go to jail for it.
They don't care. As long as they can sell the illusion and make bank in the short-term, they could care less about the long-term consequences. I'm sure some of them even know that LLM capabilities are vastly overestimated and a correction will happen.
Claude can write really impressive code if you babysit it every step of the way. It will also destroy your code in really impressive ways if you don't and just blindly follow its instructions. For example, it had me merge two unrelated functions into one when I asked it to print an existing array as a debug line if there was an error. Not only did it end up not printing the array but its next suggestion was to split that function.
I asked Claude to create a Symfony (PHP) command to do simple checks that language binds of some forms existed in their corresponding language files. It only imported 3 classes that it never used, 5 variables that it never used, called 2 methods that do not exist, and used 1 method that was deprecated in 2018. It did not perform the check in a workable way and reported everything as an error. I've never wanted a junior developer who can't learn, but that's apparently what our team just got.
It's a fancy google search, it doesn't think or reason. What is the next most likely weighted token is the only thing that matters, and I saw a video where it very confidently said a Prime number was not prime, because the first token it matched on was "No" so the rest had to make sense. And it was confidently incorrect in it's reasoning (or the output was convincing to a human, no reasoning happened). Probably due to the fact, like you said, it's going on the most common response. And that could be a reddit thread with completely wrong individuals.
A project I'm working on started using AI code reviews & it only caught 3 issues, and 2 of them were for completely wrong reasons. All 3 of them were incredibly basic errors that I would have caught without it just by testing it once I was done setting up all the scaffolding. The rest was dozens upon dozens of misunderstandings of the code base or regurgitations of suggestions that read like they were from a cargo cult programming blog. It took me 30 minutes to sift through utter crap to get very little benefit out of it, and it was only that short for the amount of reviews it gave because they were so crap I could dismiss them out of hand. It's not even like my code is perfect either. My previous PRs on that same project have gotten plenty of meaningful reviews and adjustment requests from other human reviewers in the past, but the AI didn't catch anything of that nature.
Great one. I was bracing for you calling out mistakes I myself might make but nope, the demo and test were indeed garbage. Thx for these sane well-reasoned takes.
AI Gen Code, including Claude is just a tool 🧰. It's better than being stuck or being on a blank screen. It's better than just rubber ducking or random Googling. But, it's just a tool 🧰 and without trained operators, the tool 🧰 will not work so well. Clueless tech business leaders that are de-valuing and/or mistreating their best devs will realize this in 2-5 years. ⏳
Didn’t we go through the same hype cycle with self driving cars? The usury based financial system gives all the incentives for creating bubbles before they burst and the big companies are saved with taxation or inflation.
When *AGI soon* reached pitch fever middle 2023, I used to post the following comment, and got spanked for it: "Billions of $ + top talent + infinite amounts of data could not crack Self Driving Tech, but here we are, hoping AGI, something infinitely more complex, will be solved. What am I missing?"
We've been through this cycle countless times, as you say we've had self driving cars, the metaverse and NFTs, no-code replacing programmers, 3d TV's, cryptocurrencies, and a tonne more I can't even remember, and every time its "no no its real this time". Oddly every iteration seems to be more outlandish and unrealistic than the last, looking back on it 3d tv's seems almost quaint compared to what we're at now.
Self driving is pretty decent for highway driving, I use it on my Tesla regularly even though it's "enhanced driver assist" at this point. I expect LLMs to be the same, more as assist features, till we are able to take another significant leap in LLM tech.
@@maheshprabhu Let's start from here: the definition of "self driving". The tech is not ready, as many RUclips videos show. I am not trusting my cat inside those cars when the "self driving" thing is on.
There is hope, but I'm not going to lie - it's looking ugly in the short term. If it's any consolation - the AI you might get replaced with will do a lousy job, and it will get fired, and I expect there will be a bunch of jobs for us later cleaning up the mess the AI made,
@@zokpls8712 Not yet - at least in most places. But some companies are starting to make pretty big moves in a direction that's bad for programmers (in the short term). BP says they've reduced their programmers by 70%. ( www.webpronews.com/bp-needs-70-less-coders-thanks-to-ai/ )
LLMs are complex *text* pattern matching machines. They don't "think", they don't "reason", they don't even "know". Their effectiveness at producing plausible-looking text fools humans into attributing those qualities to them, similar to how people see faces in things.
I appreciate your channel. The technical component of all this has been the part that I've basically had to start from scratch to understand all the bullshit going on. As someone with a journalism undergraduate degree I am embarrassed by how the press have basically been stenographers for these companies. It's crazy how terrible they've been in covering this.
AI code assistants are used by major tech companies like Nvidia. They are just tools like but for sure they have made me more productive in generating efficient useful code but they make mistakes like humans. But as time goes by they will get better
I've been coding for just about 2 years only and have been using AI to learn most of the way. I do use Claude fairly often, but I just use it as a jumping off point and almost never actually use it completely. Especially for Svelte which was my last project, it knows a fair amount but the documentation and random blogs were far better resources
Welcome to reinforcement learning, where the goal is to create AI, but without the I. To be fair, what did we expect? If you reward someone based on whether you like what they say, won't you just create the most perfect sycophant that only tells you what you want to hear? And that's all that these LLMs are - a correlation of weights and biases that link tokens in the query to the output tokens it should generate +/- some randomness to create the illusion of creativity. These "AI" solutions will always "provide" you with a "solution" and even write beautiful little comments, as well as insert magic bullet functions from libraries that don't exist. It's cute, but alas, a waste of billions of dollars and untold man-hours of labor. Big tech can't admit that, since then they'd be held accountable by the investors. It will crash and burn someday, - a correction always happens - just not in the short-term.
you come off as a pure pseudointellectual when you act like this technology is a waste when clearly you just personally haven't found use for it (or tried to?). it has made my job as a programmer faster and it has helped me learn many things in life outside of my job. You say it will crash and burn but it is currently making billions of dollars in revenue. If you have used the models for what they are useful for then you will know they aren't just an illusion of creativity, they actual reason about things in ways that haven't been reasoned about before explicitly and that's their whole utility. Even my company is using these LLMs for actual business use-cases where we previously required human brains. The costs are also incredibly low and getting lower by the day...
A.I. Coding and its hype will lead to digital Idiocracy. "Why does the crop now grow, we are giving it electrolytes" will become "why does the software not do what it is suppossed to, we are using A.I." But then again, I like idiots as competition...
As soon as you said complicated ways to find prime numbers, I thought of the Sieve of Eratosthenes. I had the pleasure of coding that algorithm in ASM for a class project. I subscribed because it reminded me of, while tedious, how much fun it was to get to work.
And to explain algorithms, give ideas for algorithms, come up with APi's and libraries you did not know existed, be a better intellisense, right little utilities, critique small bits of code, suggest names for functions, find duplicate functions, write simple unitTests, search through a lot of data and analyse it, suggest fixes for problems quickly without going to stack overflow.
@@whig01 It doesn't make sense to say that a normal program is "effectively AI", just because it has some interactive options. That would make pretty much every app "effectively AI"
All these tools are basically goood summarisation of existing outputs from humans. For a very close question/query. They can retrieve a good summary. Nothing more, ask something a bit twisted, they will go complete kaput 😅
Its undoubtedly good at natural language processing, thats what it was made for, its really odd that we have this thing that can talk like us and just automatically believe that means it can do everything else like us too
It's Ok for coding. Definitely not great, but "not useful" is a bit much. You can rarely use what it gives you outright but it's very handy as a jumping off point.
It's certainly true that AI writes buggy code and "isn't there yet", but never underestimate the greed of corporations. Quality doesn't really matter to them. They'd be perfectly happy to lay off half their workforce in exchange.
I tried Claude 3 Sonnet and Claude 3.5 Sonnet for my own tasks (unit test generation for Python code, test generation for TF code) - I did not see any significant differences in the quality of the output. And no matter how detailed prompt I give, the produced test require ~80% of refactoring.
@@kyleyoung8179 No, append a trailing zero to 9.9, so it becomes 9.90. Now that we have 2 digit numbers after both decimal points, compare 90 to 11, and you see that 90 is greater. So we can conclude 9.9 > 9.11. The length of the string representations is such that len("9.9") < len("9.11"), but not the numbers they represent. A more interesting test case, that AI fails, is whether -2 is greater or less than +1. The correct mathematical answer of course, is +1 > -2. But depending on application, "greater" and "less than" may only be meaningful when comparing magnitudes, making what -2 represents be what most people would consider greater than +1. An example being electrical charge, where it is completely arbitrary what we call positive, and there is nothing "greater" about a sodium ion's +1 charge when compared with an oxide ion's -2 charge.
Most are getting this correct now. My 286 used to randomly crash in certain applications after I pressed the turbo button. Yes, computers used to come with a turbo button.
I agree with you that even in 5 years, AI will never be as good as a human programmer, I think you can concede that in 5 years it will be so much cheaper than a human programmer, that hiring more than one for a given project will not make sense. As a teacher you know that college CS majors currently do a better job at coding than AI, but you also know that it also literally takes years to learn how to code at your level. Thanks to Retrieval Augmented Generation (RAG), you can literally train an AI to code At Your Level in just a few hours. That's the real threat, it's not what it knows, it's the insane rate that it can improve, and once it gets to the level where it can self-improve without human guidance, then it's game over for everybody.
4:55 That's a pretty good summary of how outsourced codes have been coming back from the far eastern parts for 20+yrs. There is code, there are tests, they pass, except none of it is actually useful. Management loves it because it's cheap and fast.
I have too noticed AI could not perfectly write code. It just helps writing simple code that programmer can use and write a bigger logic. though it wrote unit test good. Good use case of AI could be in testing
You have been right about most things, but of course this is what this AI is all about, hype, a deflection, layoffs a 'necessary evil' since the dark objective is really about what most will understand as far as cybernetics is concerned, not robotics synthetic 'machines'' but of course this involved biology, even way back when they started messing around with neurology and publicly take what finalspark is doing and extrapolate on that. The darker objective is unethical, so whenever we essentially replace ourselves this will become clear. This is how they will achieve real AI or as they would say AGI, sure much of it will be mimicry but self-awareness, reasoning and emotions, the complexity of the human thought process spanning unrelated domains, all of that is a matter of course, the real question is, how close are they... 5 years, 50 years or are these things already walking amongst us and we are the proverbial Turing test. Regardless insofar as this, you are right, but they need to do this, fear and panic shuts down the prefrontal cortex and allows the limbic system to take over, crippling critical thinking, these things relating to 'ai' will fail, the current energy requirements to train the latest LLMs is insane so much so meta is (of perhaps this is done already) building a nuclear power plant for this very reason, a bit extreme, if people understand G P T tech they will get why scaling is a big problem... sure handy robots doing programmed tasks with some level of acting upon instruction from the user will continue and will improve but this GPT-like tech is likely the biggest most expensive con ever. I deal with machine languages (yes even MASM still) and also most of the others, be it scripting or otherwise, whatever I need for what I do, from robotics to SE and mostly now security (since that is what is affecting people the most)... I adapt to where my skills are needed. This AI code is crap, but I still thoroughly enjoy your exposition on it
I just asked GPT 4o: "Is 21577 a prime number?" - Answer "yes" (correct). Then I asked again: "Is 668887 a prime number?" (668887 = 21577 x 31). Answer: "Yes"
I tried this with my Excel spreadsheet with code I wrote for prime number detection, and got an overflow error when trying 668887. I revised my code to use long integers, and it properly detected 668887 as composite.
@@carultch I figured out that gpt generally has problems dealing with numbers. It tokenizes input and a number like 663837 is split into 663 837. Then.it has problems to comprehend the context because 664.837 are ultimately different things than number 664837...
The context is just too small still. I have an arduino code file thats only about 2000 or so lines, but it just cannot read it and answer questions on it. It just falls over and hallucinates loads. I was trying to get GPT and Claude to add some functionality and it just ignores whats already there and suggests duplicating functionality and ignoring the convention that is already there. I think for now Ai is only good for answering short small context questions. Nothing more.
@@breakoutgaffe4027 Trу swaрping Latin lеttеrs for a look-a-like Суrilliс and Greek letters with сompletely different рrоnunciations, and sеe if it confusеs the LLM. I did that in this сomment, and yoυ рrоbably didn't evеn noticе.
I've personally found good results with Claude by restricting it to be a helper tool. Writing very simple things or giving me some idea of how I might approach that almost like discussing with a colleague. I keep in mind that they are LLMs and only predict the most likely word to construct a sentence and have no concept of logic or design. For example using them to understand or create regex I have found them to be good. Manipulation of data in certain scenarios and writing very basic functions which aren't complicated but take time to write for example mapping data. The minute you ask it to go outside of this scope it isn't worth it
One example cannot represent all situations. I believe that we should view problems from a developmental perspective. At the present stage, AI code definitely still cannot replace humans, but it can assist coders in improving coding efficiency.
Since im not a coder i had no idea how to fix a broken plugin for a gameserver. The plugin was abandoned and i did not find anyone who would be bothered enough to have a look at it except of one guy, who said he would if i'd pay him 100 bucks an hour. So i tried my luck with claude 3.5 and it not only fixed it for me, but it added new functions to it at my request. It is now working again, and i don't care if you think the code is bad or not. IT WORKS! And i did not have to pay a fortune for it.
Good for you. It is no proof however that Claude can code in general, it just shows that in your specific case you found a way to use Claude as a tool to get the outcome you wanted. That's great and shows you're really smart. Re Claude, News at 5.
The only place I sort of trust the output of an AI is in one off short python scripts, but even then I usually have to change something. When doing professional or "serious" work I don't find myself using a chatbot all that often. I find that it's pretty good at pointing me in the right direction, but the code itself is not very good and accurate most of the time. Also having used chatpgt for a while and now claude 3.5 for a while, I'm considering going back to chatgpt. People say claude is better at programming, but honestly, that has not been my experience. It seems to missunderstand my questions a lot, regurgitate the same code I've given it and other things. It could be that I'm just more aware of these things and I'm going to have the exact same experience with gpt4 though..
I know they're hoping and praying that if they just keep throwing more money at the problem, eventually all the problems will go away, but... at what point do we just admit that we're falling for the sunk cost fallacy, and it's cheaper to just pay human beings to do the job?
Sadly upper management still has a goal of firing 60% of the engineers at my company within the next two years. Of course AI can't actually do our jobs but they may only learn that after the fact. It's demoralizing working for people who don't understand what you do and see you only as a cost center.
are we in the same company? our CEO (and CTO) started preaching laying off people with a hope that 30% of engineers would be replaced by the "AI". It shows how brainless management is.
ChatGPT: "here's your code for MSProject VBA" ME: "This is the 12th time I've indicated that you are providing Excel VBA code, not MSProject VBA code" ChatGPT: "you're right, that code is Excel code" ME: "If you were able to determine it was Excel code, after I pointed it out, why can't you simply pre-check the code before suggesting it"? ChatGPT: "er....uh....Context Management, Error Correction Mechanisms, Training Data Limitations...and btw, F U"
our CEO is all behind AI , its so depressing any meeting he literally just brings up chat gpt and say sees there is the answer like yea its *an* answer and it may work but its not that simple. feel like just uising Chatgpt all the time and not putting any thought in to it, as it seems like thats what people expect. its depresing and lame and sometimes these models dont even generate the right answer like if you asked it something it may generate it right but if you asked the same thing 20 times it will give some variation that is just wrong its all just random.
AI is not magic. It’s a tool that can be used effectively or poorly. The IDE integrations are useful when used well. LLMs are also great for debug messages, sometimes (but not always) better than google search. They are far from being able to autonomously develop on their own.
AI maths is an issue because of tokenization, which is a perf shortcut. This problem should mostly go away with the next generation of models. Or failing that the ones thereafter.
Hello sir, I just want to ask a not entirely related question, but have you already tried and tested the ChatGPT-4o-latest (2024-08-08)? If so, how does it compare to Claude 3.5 Sonnet?
meanwhile, ClownStrike doesn't even bother to write a test to validate input length for code going straight into the kernels of millions of machines. Maybe they used A"I"?
This is the paradox, because stackoverflow is one of the places where LLMs are learning from. There is no LLMs coding if there is no code written by humans to learn from.
I've heard somewhere that github copilot sure was making coders faster, but also brought up code churn rate (rate at which a line of code had to be changed). Might not be causation....
My feeling is that the AI pair programming thing is a little like the 'DreamWeaver' craze of the late 90's early 2000's... it looks like a winner, but of course you have no clue what all this code is if you aren't a coder and would leave it in there, a lot of it has security issues too! The AI will never remind you to run unit tests or stress the code at high load, thats all up to you and your knowledge of coding, and if you have none then you are like a bull in a china shop... So as a tool I use the pair coder a lot, cos its got all that stuff I used to copy, paste and modify from stack exchange... but its faster. However having been a coder since the 80's I KNOW WHAT I'M DOING!
You make great points, but i feel like you are missing the bigger picture. The current models are extremely primitive. You do a great job of highlighting problems with them. Future models will be trained with knowledge of the type of problems encountered and how to solve those problems. It will be an iterative process of improvement. How long the process will take is up for debate but I expect that most problems will be solved in the next 2 years. The fact that the output of LLMs is stochastic is something I see as a positive, not a negative. I want my models to compare different ways of writing the same function and to learn what works best. I am working on a coding assistant and I will use videos like this as guidance.
As a Full Stack Developer, I completely understand and agree with you. However, this field will eventually improve with time. We are coming up with new ways to improve models every day. We are still in the early days, and much of it is mostly hype. To be honest, I pay for these LLMs and use them too. They work better for writing, but for coding, it's still too early.
12:35: _"The bubble will pop"_ the bubble popped already the last week when the stock market fell, and the market analytics called it a "reset after the AI hype".
A lot less egregious than devin, but fair assessment. Albeit a lot of these issues could be solved or mitigated to a large degree by using something like typingmind with sonnet 3.5 + perplexity for example--to look up the most current and/or correct implementations. Claude 2 to Claude Opus to Claude 3.5 sonnet were all huge jumps in capabilities. Lets see what happens with 3.5 Opus. Lets also see if any of the reasoning changes actually pan out anytime soon.
Claude is generally very good at coding and picking on a bad demo doesn't change that fact. People often get a few views on their video for a topic and then get tunnel vision and become a one trick pony. It's fine point out a demo that's wrong, but to then generalize it as if this is a typical result when it's not (actually it's quite rare for Claude to give you just blatantly bad it wrong code) is wrong and misleading your audience. Im a dev of nearly 20 years who uses claude daily. I suggest you actually a) take the time to learn how to integrate it properly into your dev work flow, assess it by actually using it in real world scenarios on top of what you're doing here, in order to develop a more well rounded perspective. Or not... There's plenty of views and money in AI hating.
Sure, demos get ahead of current capabilities, but the assessment that "AI Generated code is bad" or the characterization of "AI Coding Crap" feels pretty extreme and clickbaity. I'm an experienced C++ SWE that has recently started using generative AI, and I think your take is too negative here. Most LLMs have had "one-shot" reasoning here, and we're in the very early days of LLMs being able to query themselves recursively, query other specialized models, and invoke deterministic software (compilers, interpreters, static analyzers, fuzzers, etc.). The long tail of integration with software engineering tooling will take years, but the net impact is going to be a hybrid thinking machine composed of generative and deterministic components. Requirement gathering, human judgement, code review, software architecture, integration engineering, etc. will likely stay in the human domain for a while, but software engineering is absolutely going to change, and fewer ICs are going to be needed. Some skills such as mastery of syntax, and memorization of standard library functions will not be as valuable in the future.
Behind the scenes interview with @Proof_news here: ruclips.net/video/xmnRpW7psQc/видео.html
Hot take: Code-writing AI is hyped up so much because companies needed an excuse to lay off many people, and then hire them again later as independent contractors with far less pay and benefits...
Replacing coders with AI is like replacing doctors with AI. Who really needs replaced with AI is upper management. Heard thinking, commonality, same old mistakes sounds like a perfect use case for AI.
Herd = moving cattle vs Heard = listening, then understanding
@@kathrynj.hernandez8425 Both Herd and Heard are apt in his original statement
your english vocabulary should be replaced by ai first
AI should know this...should
Spot on.
been using Claude everyday for a side project. It's not the greatest at debugging issues. You usually have to hold its hand and tell them the exact few lines or functions to focus on otherwise it will overcomplicate things by default. What it's great for is generating some boilerplate or template for a new function or component. It saves tons of time from looking up documentation, googling issues on stackoverflow. You just have to read over the code it generates to see if it makes sense. If not suggest a different approach and it's generally pretty good at adapting since apparently you're never wrong...
Lol😂😂😂
My man, never stop what you do here. Thanks for another great video calling out the nonsense.
@@breddygud6890 thanks for making me feel important again
AI is pretty good if you are using it to "query" stuff that is in documentation. in fact i often drop a link to node docs for an example then ask a questing around node that i know will be really well documented.
What model do you use for that?
@@therocktimist perplexity
Or just read the documentation...
I think the biggest problem is the marketing angle where AI companies are saying that it will replace developers instead of saying that this is something that will make developers 2 times faster/eficcient for ex, thus reducing your development cost.
In it's current state Claude AI is good enough to convince me to buy a subscription. Problem is that 20$/mo is not nearly enough to finance AI R&D, thus companies choose to overhype the AI so they can fool VCs into funding them with insane amounts of money.
Yup Copilot costs significantly more than $10/user/month
I doubt it will make devs 2 times faster I have used gpt4 but it's garbage, it generates really bad code that will cause problems in the future and it doesn't understand how to fix it when it is integrated with kinda complex codebase. Sure it can write boilerplate code for already popular libraries but you can easily google that.
I use claude and AI very effectively. Like right now I am writing a Kotlin app, but I only know javascript. But you have to know what you are doing to get real functionality out of it. I read every line of code and I am starting to really get Kotlin. But it's all about the pseudocode. You have to think through what you want and communicate it very clearly and expect to have to hone your communication several times. All of that makes me a better developer. No AI cannot replace me, but it sure does help me.
From my perspective its mostly cause you dont know Kotlin. Do you use AI for javascript?
@@patrickjreid yes AI can definitely help. But does is really help more than looking the info up in Wikipedia or Google Search? I doubt that a little bit.
@@bpscast For me AI helps a lot more than Google + StackOverflow or Wikipedia. AI allows me to be faster in getting the information and/or suggestions I want, and if the information/suggestions are wrong I can quickly figure that out in my dev environment and iterate from there. Now even though the traditional method of Google + StackOverflow has less hallucination tendencies (sometimes the info is still wrong/outdated), there are multiple times when I used non-AI code from someone's SO post or blog and it didn't exactly fit my scenario, so I had to iterate all the same anyway.
In any case AI has already given me a huge productivity boost in my development, and the AI models themselves plus tooling around it are only going to improve. I've been trying out Cursor for the past week, and I can see my productivity improve even more with some of their features. E.g. I used their chat function with context to code a new feature for me, and I basically acted as a code reviewer. Even though Cursor didn't get it right the first time, after one iteration with my own feedback it managed to get it right. Definite time-saver.
Same here as an attorney. Claude saves me so much time. AI skeptics who say that AI isn’t helpful either 1) know very little about prompt engineering, or 2) know that they are lying for some ulterior motive
I have been experimenting with the aider project recently. It has a few benefits for me, but big issues too. The biggest benefit is that it changes the dynamic of programming from pure creation to creation&collaboration. It creates mediocre code pretty often (backed w/ Sonnet35 or gpt4o), but it gets close enough that I change in the editor and move on.
I'm prompting it extremely granularly too. 'add a new arg here', 'write a test for this one method', etc. If you try to do too much at once, everything falls apart so quickly.
Also I'm a senior dev who can nearly instantly read a chunk of code (in my app at least) and see things wrong. I can't imagine a junior or even most midlevel devs doing anything useful with this without hitting so many speedbumps.
agreed , aider workflow is good at small iterative additions and changes , id guess about 75% of the time it does well . the main failure cases are typically when trying to do big changes across a lot of code , and when it gets into a repeating cycle of alternating between 2 wrong answers .
I dont understand why people are trying to crowbar LLMs into reasoning tasks. If they took the time to understand how they work, they’d stop wasting their time on this, and only use LLMs for things they are good at, which language stuff, not reasoning!!
Appreciate the injection of sanity. We shouldn't accept code from LLMs that we'd fail in review if it was from a human.
Takeaways: 1. The code gens can at most do imperfect boilerplate stuff. 2. Coder monkeys are f***ed, and 3. All others will have more work proofreading and testing AI code. Fair assumptions?
Models won't get a lot better, coding is just the wrong category of problems for Neural Networks, it's not hard to understand, programming is not a fuzzy problem, which is what they actually do well...
Thank you! I've got no idea why so many people don't understand that.
Models will get substantially better, lol. Don't kid yourself.
There are already new focuses based on revamping AI reasoning for this specific reason.
There are multiple papers by Anthropic, Microsoft, and OpenAI on this.
@@DESX312 what about the exponential increase in computational and energy resources needed to train new models? How do you deal with that?
@@yds6268 The biggest way is to add new hardware that has higher perf/watt.
Which is exactly what the new Blackwell Chips do, apart from that there will need to be significant changes in energy expenditure and/or management in the long term. Not something easy of course, but as AI is able to take on more tasks. That is an investment companies will be willing to do.
@@DESX312 your optimism is unfounded.
The worst part about A.I that I've found at the moment, is that the API documentation that I usually rely on to get my day-to-day job done has either been purged or has been put behind an auth wall \ is not getting updated, in favour of "use our new A.I tool". - and the new AI tool is producing mediocre code at best or the purpose of me referring to the API documentation is to understand what a certain function, constant value or module does, and just getting an example snippet from a command line doesn't help with that, so I'm finding that I have to dig through definitions in the source code and take my best guess.
Another part is what I would once refer to as an algorithm or set of algorithms within a given package or program are now being blanketed as "A.I."
I got distracted looking at ImageMagick Scripts just after the code part because that's the graphics manipulation library that I know well enough.
I had to use image magic and convert recently on a Linux box. Darned huge number of hoops to get the SE to let me run it! I should have just spun up a VM, done the needful and sftp the results to the real machine.
AI sells better at the moment. We all know this. The bubble is about to burst quickly.
Please never stop. This is such an uphill battle on something where it is so obvious. How can the sane people be the minority? Artificial intelligence should be the term used to describe the people who think the LLMs work well.
How? Marketing. Just like everything else, if you market it well, it will sell.
@@rasi_rawss Yes, and false advertising is illegal. You can't sell a product knowing it doesn't do what you said it can do. Look up Theranos. This crap is exactly the same and people should go to jail for it.
They don't care. As long as they can sell the illusion and make bank in the short-term, they could care less about the long-term consequences. I'm sure some of them even know that LLM capabilities are vastly overestimated and a correction will happen.
Claude can write really impressive code if you babysit it every step of the way. It will also destroy your code in really impressive ways if you don't and just blindly follow its instructions.
For example, it had me merge two unrelated functions into one when I asked it to print an existing array as a debug line if there was an error. Not only did it end up not printing the array but its next suggestion was to split that function.
I asked Claude to create a Symfony (PHP) command to do simple checks that language binds of some forms existed in their corresponding language files. It only imported 3 classes that it never used, 5 variables that it never used, called 2 methods that do not exist, and used 1 method that was deprecated in 2018. It did not perform the check in a workable way and reported everything as an error. I've never wanted a junior developer who can't learn, but that's apparently what our team just got.
It's a fancy google search, it doesn't think or reason. What is the next most likely weighted token is the only thing that matters, and I saw a video where it very confidently said a Prime number was not prime, because the first token it matched on was "No" so the rest had to make sense. And it was confidently incorrect in it's reasoning (or the output was convincing to a human, no reasoning happened).
Probably due to the fact, like you said, it's going on the most common response. And that could be a reddit thread with completely wrong individuals.
A project I'm working on started using AI code reviews & it only caught 3 issues, and 2 of them were for completely wrong reasons. All 3 of them were incredibly basic errors that I would have caught without it just by testing it once I was done setting up all the scaffolding.
The rest was dozens upon dozens of misunderstandings of the code base or regurgitations of suggestions that read like they were from a cargo cult programming blog. It took me 30 minutes to sift through utter crap to get very little benefit out of it, and it was only that short for the amount of reviews it gave because they were so crap I could dismiss them out of hand.
It's not even like my code is perfect either. My previous PRs on that same project have gotten plenty of meaningful reviews and adjustment requests from other human reviewers in the past, but the AI didn't catch anything of that nature.
Great one. I was bracing for you calling out mistakes I myself might make but nope, the demo and test were indeed garbage.
Thx for these sane well-reasoned takes.
AI Gen Code, including Claude is just a tool 🧰. It's better than being stuck or being on a blank screen. It's better than just rubber ducking or random Googling. But, it's just a tool 🧰 and without trained operators, the tool 🧰 will not work so well. Clueless tech business leaders that are de-valuing and/or mistreating their best devs will realize this in 2-5 years. ⏳
Didn’t we go through the same hype cycle with self driving cars?
The usury based financial system gives all the incentives for creating bubbles before they burst and the big companies are saved with taxation or inflation.
When *AGI soon* reached pitch fever middle 2023, I used to post the following comment, and got spanked for it:
"Billions of $ + top talent + infinite amounts of data could not crack Self Driving Tech, but here we are, hoping AGI, something infinitely more complex, will be solved. What am I missing?"
We've been through this cycle countless times, as you say we've had self driving cars, the metaverse and NFTs, no-code replacing programmers, 3d TV's, cryptocurrencies, and a tonne more I can't even remember, and every time its "no no its real this time".
Oddly every iteration seems to be more outlandish and unrealistic than the last, looking back on it 3d tv's seems almost quaint compared to what we're at now.
@hydrohasspoken6227
I mean Musk is a moron. So that isn't really anything to debate.
Literally leaps and bounds have already been made in this field.
Self driving is pretty decent for highway driving, I use it on my Tesla regularly even though it's "enhanced driver assist" at this point.
I expect LLMs to be the same, more as assist features, till we are able to take another significant leap in LLM tech.
@@maheshprabhu Let's start from here: the definition of "self driving". The tech is not ready, as many RUclips videos show. I am not trusting my cat inside those cars when the "self driving" thing is on.
This really needs a higher level of promotion. Well spotted and this is how code quality is going to go down hill...
Good content, something went wrong with the sound though 😢
So... there is hope that I won't lose my job tomorrow?
There is hope, but I'm not going to lie - it's looking ugly in the short term. If it's any consolation - the AI you might get replaced with will do a lousy job, and it will get fired, and I expect there will be a bunch of jobs for us later cleaning up the mess the AI made,
@@InternetOfBugsis this worse than outsourcing?
@@zokpls8712 Not yet - at least in most places. But some companies are starting to make pretty big moves in a direction that's bad for programmers (in the short term). BP says they've reduced their programmers by 70%. ( www.webpronews.com/bp-needs-70-less-coders-thanks-to-ai/ )
i use claude instead of hiring a developer
@@HoD999xThanks for making things worse
LLMs are complex *text* pattern matching machines. They don't "think", they don't "reason", they don't even "know". Their effectiveness at producing plausible-looking text fools humans into attributing those qualities to them, similar to how people see faces in things.
Pshh. I can be an incompetent coder *without AI*, thank you very much.
I appreciate your channel. The technical component of all this has been the part that I've basically had to start from scratch to understand all the bullshit going on. As someone with a journalism undergraduate degree I am embarrassed by how the press have basically been stenographers for these companies. It's crazy how terrible they've been in covering this.
Sometimes AI is quite good. It really can help you with an isolated small problem
AI code assistants are used by major tech companies like Nvidia. They are just tools like but for sure they have made me more productive in generating efficient useful code but they make mistakes like humans. But as time goes by they will get better
Tech bros: AI is about to replace all programmers!
Meanwhile the AI: circle = newSquare()
I've been coding for just about 2 years only and have been using AI to learn most of the way. I do use Claude fairly often, but I just use it as a jumping off point and almost never actually use it completely. Especially for Svelte which was my last project, it knows a fair amount but the documentation and random blogs were far better resources
Welcome to reinforcement learning, where the goal is to create AI, but without the I. To be fair, what did we expect?
If you reward someone based on whether you like what they say, won't you just create the most perfect sycophant that only tells you what you want to hear?
And that's all that these LLMs are - a correlation of weights and biases that link tokens in the query to the output tokens it should generate +/- some randomness to create the illusion of creativity.
These "AI" solutions will always "provide" you with a "solution" and even write beautiful little comments, as well as insert magic bullet functions from libraries that don't exist.
It's cute, but alas, a waste of billions of dollars and untold man-hours of labor.
Big tech can't admit that, since then they'd be held accountable by the investors.
It will crash and burn someday, - a correction always happens - just not in the short-term.
you come off as a pure pseudointellectual when you act like this technology is a waste when clearly you just personally haven't found use for it (or tried to?). it has made my job as a programmer faster and it has helped me learn many things in life outside of my job. You say it will crash and burn but it is currently making billions of dollars in revenue. If you have used the models for what they are useful for then you will know they aren't just an illusion of creativity, they actual reason about things in ways that haven't been reasoned about before explicitly and that's their whole utility. Even my company is using these LLMs for actual business use-cases where we previously required human brains. The costs are also incredibly low and getting lower by the day...
A.I. Coding and its hype will lead to digital Idiocracy. "Why does the crop now grow, we are giving it electrolytes" will become "why does the software not do what it is suppossed to, we are using A.I."
But then again, I like idiots as competition...
As soon as you said complicated ways to find prime numbers, I thought of the Sieve of Eratosthenes. I had the pleasure of coding that algorithm in ASM for a class project. I subscribed because it reminded me of, while tedious, how much fun it was to get to work.
What works is asking it to document and optimize code sometimes.
I had a decent experience getting it to document code.
And to explain algorithms, give ideas for algorithms, come up with APi's and libraries you did not know existed, be a better intellisense, right little utilities, critique small bits of code, suggest names for functions, find duplicate functions, write simple unitTests, search through a lot of data and analyse it, suggest fixes for problems quickly without going to stack overflow.
Documenting code is a fairly trivial task, which is why Non-AI programs for generating documentation existed for years
@@ZoranRavicTech But you can't interact as well with those programs, and if they are sufficiently interactive they are effectively AI.
@@whig01 It doesn't make sense to say that a normal program is "effectively AI", just because it has some interactive options. That would make pretty much every app "effectively AI"
All these tools are basically goood summarisation of existing outputs from humans. For a very close question/query. They can retrieve a good summary. Nothing more, ask something a bit twisted, they will go complete kaput 😅
Nice, weren't AI bros saying "I just use it to write tests" ? This stuff is just not useful for coding, it's just not what it's categorically good at.
disagree, i make games with it, check my other comments
@@HoD999x correction. You make low effort games, that takes a few hours of prompting to create
@@HoD999x go make Legend of Zelda Ocarina of Time using prompts. Then I will believe you make games with LLMs.
Its undoubtedly good at natural language processing, thats what it was made for, its really odd that we have this thing that can talk like us and just automatically believe that means it can do everything else like us too
It's Ok for coding. Definitely not great, but "not useful" is a bit much. You can rarely use what it gives you outright but it's very handy as a jumping off point.
It's certainly true that AI writes buggy code and "isn't there yet", but never underestimate the greed of corporations. Quality doesn't really matter to them. They'd be perfectly happy to lay off half their workforce in exchange.
Quality doesn't matter. But customers do. And if you ship a busted product you are toast (unless you are a monopoly).
Even if you are a monopoly, too many flops and your gone too@@calmhorizons
I tried Claude 3 Sonnet and Claude 3.5 Sonnet for my own tasks (unit test generation for Python code, test generation for TF code) - I did not see any significant differences in the quality of the output. And no matter how detailed prompt I give, the produced test require ~80% of refactoring.
you better generate code for your unit tests, not the way around...
llms think 9.11 > 9.9
Im confused, isnt 9.11 greater than 9.9?
@@kyleyoung8179It is not. 9.11 + 0.79 = 9.9, so 9.11 is the smaller of the two numbers.
@@kyleyoung8179 No, append a trailing zero to 9.9, so it becomes 9.90. Now that we have 2 digit numbers after both decimal points, compare 90 to 11, and you see that 90 is greater. So we can conclude 9.9 > 9.11. The length of the string representations is such that len("9.9") < len("9.11"), but not the numbers they represent.
A more interesting test case, that AI fails, is whether -2 is greater or less than +1. The correct mathematical answer of course, is +1 > -2. But depending on application, "greater" and "less than" may only be meaningful when comparing magnitudes, making what -2 represents be what most people would consider greater than +1. An example being electrical charge, where it is completely arbitrary what we call positive, and there is nothing "greater" about a sodium ion's +1 charge when compared with an oxide ion's -2 charge.
@@carultch ohh lol. I feel dumb
Most are getting this correct now. My 286 used to randomly crash in certain applications after I pressed the turbo button. Yes, computers used to come with a turbo button.
I agree with you that even in 5 years, AI will never be as good as a human programmer, I think you can concede that in 5 years it will be so much cheaper than a human programmer, that hiring more than one for a given project will not make sense.
As a teacher you know that college CS majors currently do a better job at coding than AI, but you also know that it also literally takes years to learn how to code at your level.
Thanks to Retrieval Augmented Generation (RAG), you can literally train an AI to code At Your Level in just a few hours.
That's the real threat, it's not what it knows, it's the insane rate that it can improve, and once it gets to the level where it can self-improve without human guidance, then it's game over for everybody.
4:55 That's a pretty good summary of how outsourced codes have been coming back from the far eastern parts for 20+yrs. There is code, there are tests, they pass, except none of it is actually useful. Management loves it because it's cheap and fast.
I have too noticed AI could not perfectly write code.
It just helps writing simple code that programmer can use and write a bigger logic.
though it wrote unit test good. Good use case of AI could be in testing
All I know is that it helps me
You have been right about most things, but of course this is what this AI is all about, hype, a deflection, layoffs a 'necessary evil' since the dark objective is really about what most will understand as far as cybernetics is concerned, not robotics synthetic 'machines'' but of course this involved biology, even way back when they started messing around with neurology and publicly take what finalspark is doing and extrapolate on that. The darker objective is unethical, so whenever we essentially replace ourselves this will become clear. This is how they will achieve real AI or as they would say AGI, sure much of it will be mimicry but self-awareness, reasoning and emotions, the complexity of the human thought process spanning unrelated domains, all of that is a matter of course, the real question is, how close are they... 5 years, 50 years or are these things already walking amongst us and we are the proverbial Turing test.
Regardless insofar as this, you are right, but they need to do this, fear and panic shuts down the prefrontal cortex and allows the limbic system to take over, crippling critical thinking, these things relating to 'ai' will fail, the current energy requirements to train the latest LLMs is insane so much so meta is (of perhaps this is done already) building a nuclear power plant for this very reason, a bit extreme, if people understand G P T tech they will get why scaling is a big problem... sure handy robots doing programmed tasks with some level of acting upon instruction from the user will continue and will improve but this GPT-like tech is likely the biggest most expensive con ever. I deal with machine languages (yes even MASM still) and also most of the others, be it scripting or otherwise, whatever I need for what I do, from robotics to SE and mostly now security (since that is what is affecting people the most)... I adapt to where my skills are needed.
This AI code is crap, but I still thoroughly enjoy your exposition on it
I just asked GPT 4o: "Is 21577 a prime number?" - Answer "yes" (correct). Then I asked again: "Is 668887 a prime number?" (668887 = 21577 x 31). Answer: "Yes"
I tried this with my Excel spreadsheet with code I wrote for prime number detection, and got an overflow error when trying 668887. I revised my code to use long integers, and it properly detected 668887 as composite.
@@carultch I figured out that gpt generally has problems dealing with numbers. It tokenizes input and a number like 663837 is split into 663 837. Then.it has problems to comprehend the context because 664.837 are ultimately different things than number 664837...
enterprise-ai AI fixes this. AI coding examples and demos.
*rubs hands*
Time for my weekly dose of watching this channel pour cold water on the hype.
The context is just too small still. I have an arduino code file thats only about 2000 or so lines, but it just cannot read it and answer questions on it. It just falls over and hallucinates loads.
I was trying to get GPT and Claude to add some functionality and it just ignores whats already there and suggests duplicating functionality and ignoring the convention that is already there.
I think for now Ai is only good for answering short small context questions. Nothing more.
As someone learning to code, I'm rapidly realising that I only make progress when I mostly ditch LLMs
Try switching z for s in realizing and you’ll progress even faster.
Que?
@@crypto_que the only Americanism I'm on board with is pronouncing it "router" instead of "rooter"
@@breakoutgaffe4027 Trу swaрping Latin lеttеrs for a look-a-like Суrilliс and Greek letters with сompletely different рrоnunciations, and sеe if it confusеs the LLM. I did that in this сomment, and yoυ рrоbably didn't evеn noticе.
I've personally found good results with Claude by restricting it to be a helper tool. Writing very simple things or giving me some idea of how I might approach that almost like discussing with a colleague.
I keep in mind that they are LLMs and only predict the most likely word to construct a sentence and have no concept of logic or design.
For example using them to understand or create regex I have found them to be good. Manipulation of data in certain scenarios and writing very basic functions which aren't complicated but take time to write for example mapping data. The minute you ask it to go outside of this scope it isn't worth it
One example cannot represent all situations. I believe that we should view problems from a developmental perspective. At the present stage, AI code definitely still cannot replace humans, but it can assist coders in improving coding efficiency.
Since im not a coder i had no idea how to fix a broken plugin for a gameserver. The plugin was abandoned and i did not find anyone who would be bothered enough to have a look at it except of one guy, who said he would if i'd pay him 100 bucks an hour.
So i tried my luck with claude 3.5 and it not only fixed it for me, but it added new functions to it at my request. It is now working again, and i don't care if you think the code is bad or not. IT WORKS! And i did not have to pay a fortune for it.
Good for you. It is no proof however that Claude can code in general, it just shows that in your specific case you found a way to use Claude as a tool to get the outcome you wanted. That's great and shows you're really smart. Re Claude, News at 5.
The only place I sort of trust the output of an AI is in one off short python scripts, but even then I usually have to change something. When doing professional or "serious" work I don't find myself using a chatbot all that often. I find that it's pretty good at pointing me in the right direction, but the code itself is not very good and accurate most of the time.
Also having used chatpgt for a while and now claude 3.5 for a while, I'm considering going back to chatgpt. People say claude is better at programming, but honestly, that has not been my experience. It seems to missunderstand my questions a lot, regurgitate the same code I've given it and other things. It could be that I'm just more aware of these things and I'm going to have the exact same experience with gpt4 though..
Hi Caps Admin I love ur music pls make more
I know they're hoping and praying that if they just keep throwing more money at the problem, eventually all the problems will go away, but... at what point do we just admit that we're falling for the sunk cost fallacy, and it's cheaper to just pay human beings to do the job?
Sadly upper management still has a goal of firing 60% of the engineers at my company within the next two years. Of course AI can't actually do our jobs but they may only learn that after the fact. It's demoralizing working for people who don't understand what you do and see you only as a cost center.
are we in the same company? our CEO (and CTO) started preaching laying off people with a hope that 30% of engineers would be replaced by the "AI". It shows how brainless management is.
You can find endless cases where AI sucks thats not where the challenge really is tho…
Often just properly prompting or regenerating the response gets you to the right answer.
Ask GPT4 how many "r" are in Barrier.
@@hydrohasspoken6227 I pray some day we'll find an effective solution to the "how many X letters are in this word" problem.
Thank you for sharing this videos. What do you think about Aider with Deepseek coder v2 in your own pc to help you to write code?
ChatGPT: "here's your code for MSProject VBA"
ME: "This is the 12th time I've indicated that you are providing Excel VBA code, not MSProject VBA code"
ChatGPT: "you're right, that code is Excel code"
ME: "If you were able to determine it was Excel code, after I pointed it out, why can't you simply pre-check the code before suggesting it"?
ChatGPT: "er....uh....Context Management, Error Correction Mechanisms, Training Data Limitations...and btw, F U"
our CEO is all behind AI , its so depressing any meeting he literally just brings up chat gpt and say sees there is the answer like yea its *an* answer and it may work but its not that simple. feel like just uising Chatgpt all the time and not putting any thought in to it, as it seems like thats what people expect. its depresing and lame and sometimes these models dont even generate the right answer like if you asked it something it may generate it right but if you asked the same thing 20 times it will give some variation that is just wrong its all just random.
AI is not magic. It’s a tool that can be used effectively or poorly. The IDE integrations are useful when used well. LLMs are also great for debug messages, sometimes (but not always) better than google search. They are far from being able to autonomously develop on their own.
Great video!
We are thankful.
AI maths is an issue because of tokenization, which is a perf shortcut. This problem should mostly go away with the next generation of models. Or failing that the ones thereafter.
The most effective way to tell an ai what to code is to code it yourself and provide it to the ai
Hello sir, I just want to ask a not entirely related question, but have you already tried and tested the ChatGPT-4o-latest (2024-08-08)? If so, how does it compare to Claude 3.5 Sonnet?
meanwhile, ClownStrike doesn't even bother to write a test to validate input length for code going straight into the kernels of millions of machines. Maybe they used A"I"?
Now I wonder... If instead of python you asked them to use C... how many exploits those AI would introduce to the code.
Or you could tell it to use Rust and forego all exploits.
enough to give us work for years to come...
Chat gpt is replacing stackoverflow
This is the paradox, because stackoverflow is one of the places where LLMs are learning from. There is no LLMs coding if there is no code written by humans to learn from.
I hope this AI hype is over before humanity decided to let AI write code for nuclear reactors.
I like these tools. As it helps me speed up 3-4 x. It is like working with a smart intern.
I've heard somewhere that github copilot sure was making coders faster, but also brought up code churn rate (rate at which a line of code had to be changed). Might not be causation....
My feeling is that the AI pair programming thing is a little like the 'DreamWeaver' craze of the late 90's early 2000's... it looks like a winner, but of course you have no clue what all this code is if you aren't a coder and would leave it in there, a lot of it has security issues too! The AI will never remind you to run unit tests or stress the code at high load, thats all up to you and your knowledge of coding, and if you have none then you are like a bull in a china shop... So as a tool I use the pair coder a lot, cos its got all that stuff I used to copy, paste and modify from stack exchange... but its faster. However having been a coder since the 80's I KNOW WHAT I'M DOING!
You make great points, but i feel like you are missing the bigger picture. The current models are extremely primitive. You do a great job of highlighting problems with them. Future models will be trained with knowledge of the type of problems encountered and how to solve those problems. It will be an iterative process of improvement. How long the process will take is up for debate but I expect that most problems will be solved in the next 2 years.
The fact that the output of LLMs is stochastic is something I see as a positive, not a negative. I want my models to compare different ways of writing the same function and to learn what works best.
I am working on a coding assistant and I will use videos like this as guidance.
Carl, Ai Generated code is not necessarily bad; lying by omission on the other hand- but any hype is generally good.
Wait for claude to utilize o1 approach.
ai dont need to get best at coding they just need to get better than top 90% software Engineerrs to replace them which currently is happening
Is it really? I see jobs being outsourced, not being replaced by AI.
Why comment without watching the video? Or are you actually saying this garbage is on the way to being that good ?
2:35 Wait how did you screenshot that code and it came out as editable in your IDE?
As a Full Stack Developer, I completely understand and agree with you. However, this field will eventually improve with time. We are coming up with new ways to improve models every day. We are still in the early days, and much of it is mostly hype. To be honest, I pay for these LLMs and use them too. They work better for writing, but for coding, it's still too early.
it's a bit unfair testing them on graphics
Now I'm curious to see how badly it messes up but in a lower level language while directly accessing the bitmap 🤭
and this will lower the wage and give more of a headache to whom ever has to clean up the AI
But the llms are passing the llm company's own benchmarks better than ever!
With a dad like you, that college freshman will be grading the professor's homework 😂
Judgement is precisely the problem here
12:35: _"The bubble will pop"_ the bubble popped already the last week when the stock market fell, and the market analytics called it a "reset after the AI hype".
Nasdaq 7.5% down after being 115% up over the last 5 years. I don't think you know what reset means, or maybe you read too many newspapers.
A lot less egregious than devin, but fair assessment.
Albeit a lot of these issues could be solved or mitigated to a large degree by using something like typingmind with sonnet 3.5 + perplexity for example--to look up the most current and/or correct implementations.
Claude 2 to Claude Opus to Claude 3.5 sonnet were all huge jumps in capabilities.
Lets see what happens with 3.5 Opus.
Lets also see if any of the reasoning changes actually pan out anytime soon.
Claude is generally very good at coding and picking on a bad demo doesn't change that fact. People often get a few views on their video for a topic and then get tunnel vision and become a one trick pony. It's fine point out a demo that's wrong, but to then generalize it as if this is a typical result when it's not (actually it's quite rare for Claude to give you just blatantly bad it wrong code) is wrong and misleading your audience. Im a dev of nearly 20 years who uses claude daily. I suggest you actually a) take the time to learn how to integrate it properly into your dev work flow, assess it by actually using it in real world scenarios on top of what you're doing here, in order to develop a more well rounded perspective. Or not... There's plenty of views and money in AI hating.
Why you're not talking about open ai project strawberry?
Worth noting that Claude has been noticeably downgraded in the past couple weeks. It was significantly better before.
Recently found this channel and find it informative, but this gentleman always looks like he just smelled something really vile.
Try to get past it, his content is good and unique.
You won't lose your job to AI. You'll lose it to someone using AI.
Which is the same as losing it to AI.
@@Woodroffskiuhm , no ?
@@emptycode1782 Of course it is. What will have changed mechanistically? The introduction of AI; the competing human workers were there before.
it only gets better. see you all on the acceptance stage.
it's the worst it will ever be. see y'all on the acceptance stage.
Much of the AI bloated code headaches remind of of MS Front Page days lol
@internetOfBugs But what if you gave it context of your code, so that it will learn from you? surely that would make it do well?
the future is belonging to the thinkers instead of executioners , we have a competitor in the market less cheaper .
AI overlord will be very pissed on you. You are gradually turning into their number one nemesis.
Sure, demos get ahead of current capabilities, but the assessment that "AI Generated code is bad" or the characterization of "AI Coding Crap" feels pretty extreme and clickbaity. I'm an experienced C++ SWE that has recently started using generative AI, and I think your take is too negative here. Most LLMs have had "one-shot" reasoning here, and we're in the very early days of LLMs being able to query themselves recursively, query other specialized models, and invoke deterministic software (compilers, interpreters, static analyzers, fuzzers, etc.). The long tail of integration with software engineering tooling will take years, but the net impact is going to be a hybrid thinking machine composed of generative and deterministic components.
Requirement gathering, human judgement, code review, software architecture, integration engineering, etc. will likely stay in the human domain for a while, but software engineering is absolutely going to change, and fewer ICs are going to be needed. Some skills such as mastery of syntax, and memorization of standard library functions will not be as valuable in the future.
Ask GPT4 how many "r" are in Barrier.
Ask it how many s's are in innocent, as a tribute to Homer Simpson's line from "The Boy Who Knew Too Much".