When ChatGPT was new, I used it to create a simple assertion library. This was mostly just an experiment, but I'm still finding bugs in it to this day.
My experience with Anything AI generated is that it's 1. Unpredictable, 2. Not always reliable. This makes doing anything as a "prompt to code" pathway highly problematic. You can use a prompt to get something ready for you, say some boiler plate, then mould it into shape. Even then, it's really only useful for small tasks. So, no, I don't see any reasonably near future where AI is doing any actual code replacement. Let's focus on tooling and not language. We are 40 years since SmallTalk and yet, not much stands above it. I mean, I still need to build and deploy so much code, without being able to edit and update it in real time.
Absolutely agree with that. If it can't even generate boilerplate reliably and with repeatability, I'd rather type it myself. I'm a pretty fast typist. I'm using Resharper, does much of what I need. For example, convert a simple property into one with backing variable, find myself doing similar thing like this a lot.
Holly makes some excellent points, but I can't help craving for better static analysis tools that do the job and do it correctly and reliably. The sort of tool where we aren't surprised when they find the correct answer.
AI software development differs from Human software development in that its much faster, and will replace human developers for 3 reasons Human Developers will think about things, overthink things, underestimate things - then write code and generally make a few a mistakes. Takes a few round trips to get it right, if at all. AI on the other hand will make shit up, and present it as facts. When corrected, it will make some adjustments, and eventually end up back where it started with the original random muck that never worked the first time around either. But the AI process is faster than the human process. AI will replace human developers, because 3 things : 1 - Its faster at generating incorrect code than humans 2 - Thanks to Agile, getting something (anything) "done" by the end of the 2 week sprint is more important that getting anything done properly 3 - Users have been gradually conditioned to accept that their software will be ridden with bugs, and will never actually work the way it was sold to them
@@royetter1777He spittin' fire in the language of FAXX. Behind every piece of cynicism is an opportunity to fix as much as we can for a better tomorrow.
It might be interesting to start with the codebase each time a change is needed, then ask it to generate BDD for the interesting part of the codebase likely to need changes, then tweak the BDD it generates, then ask it to make corresponding changes to the code.
So as an anecdote, 6 months ago the coding AIs could write good python and javascript, but they sucked at Rust, because they couldn't understand the borrowing rules. I was thinking that they probably wouldn't ever grok Rust, as it has complicated correctness rules. As of November, all the coding AIs generate passable Rust that is borrow-safe. They also regularly push back against buggy code, suggesting fixes from just reading it. So, I think while the tech is young and buggy and incomplete, it will very obviously get there. However, for interesting code, I still see the future being based around buddy coding with the AI as your buddy, and then having it generate all the boilerplate and plumbing from text specifications. For large chunks of software, I expect that these specifications will be the "source", moving forward.
Possibly, though the rate of improvement has slowed. Still, the code is not exactly the issue for me, it's the reliability and reproducibility. You wouldn't insert a "prompt" component when the code it generates is different every time it runs.
1:20 Almost like an ever evolving ad-hoc language. Why wouldn't we have the need to parse pictures, video, and audio? Parsing vague text that AI is supposed to "get the vibe with 100% or even 60% accuracy" is already impossible unless the prompt is so specific that we're just writing code at that point. Better informing us on what the code already is would be more feasible for big projects, but the context window is too small for even that use case to be viable for large legacy code.
Much like the singularity, the premise that AI can create something new, much like evolution, requires a system to increase in complexity without an input in information and can’t occur.
@ because if it is the same program and it prioritizes b over a because it is “learning” it isn’t evolution, but if it can’t magically rewrite itself into a new program by itself, it is like trying to pick yourself up by your own boot straps. And it isn’t any better if you have Simple AI try to rewrite Simple AI(1) into a better Smart AI, because Simple AI code isn’t smart enough to come up with the code for Smart AI. It is an accepted fallacy that bacteria1A became an ape over, so Simple AI can become Smart AI, but not only is the premise false, even if it weren’t, Life and AI aren’t the same type of complexity.
@@jsbrads1 well AI can write quite good code. And it doesn't need to bootstrap itself. Humans are bootstrapping AI, and it is getting quite close to the point that it will be able to write itself
Definitely not there yet. I'm typing away, and it offers me the next line and it's correct. Carries on correctly for half a dozen lines, then out-of-the-blue, it generates absolute garbage, away with the fairies. If it were combined with some kind of logic, might be usable. I've tried it to develop algorithms, just hopeless.
AI will make every programmers job easier, without a doubt. There's no programming task you can train a human to do that you couldn't train a model to do. The only question is what programming will look like when AI can write and rewrite your app in 5 minutes. Don't trust the AI code? Write tests.
I would argue that getting CSS correct would be quite an achievement. Most human developers can't get CSS correct. Though it's kind of debatable what does it even mean to get CSS correct. Just ask someone how to center a div. :D
Dave and guest, you're not thinking far ahead. Programming language mediate between humans and electrical signals and provide humans with the ability to create instructions to a "computer" that eventually regulates electrical currents. It's slow, laborious, fragile and error prone. Pure conjecture and dependent on risk tolerance for AI .... and I hope I'm wrong ... IMHO AI will take the place of programming languages and humans will interact through natural language. In other words, programming languages will largely go the way of the dodo bird.
Machines aren’t built on natural language but mathematical logic… Natural language is far more unstructured, full of contradictions and subtle hidden meanings full of human emotion and nuance.
I'm old enough to remember "computers will never be as good as human grandmasters in chess" and later "yeah, computers beat humans in chess, but they'll never be able to pass as human in a conversation" Just because the tech isn't there yet doesn't mean it won't get there, and at the current rate we can imagine it surpassing humans before the end of this decade
@@neko6 that’s completely false. Just because a computer can beat someone at chess? Sure the computer can beat me at fifa. But it cant design itself. It’s never going to happen. Actually it does because its limited unlike humans. Do you know how it works? It guesses
Fundamentally what is human logic? How does your brain decides if A follows from B? The answer is someone has shown you some examples and you created a statistical model. This is at least in principle the same thing we are doing with the AIs, albeit our current approach is definitely too primitive. I see no reason why this could never work...
@ its not even close. Then you do not understand that statistical model is not logic and never will be. Ive used this so called AI daily and have 21 years programming experience and a masters in computer science specifically the singularity. I can tell you its never going to happen
When ChatGPT was new, I used it to create a simple assertion library. This was mostly just an experiment, but I'm still finding bugs in it to this day.
My experience with Anything AI generated is that it's 1. Unpredictable, 2. Not always reliable. This makes doing anything as a "prompt to code" pathway highly problematic. You can use a prompt to get something ready for you, say some boiler plate, then mould it into shape. Even then, it's really only useful for small tasks. So, no, I don't see any reasonably near future where AI is doing any actual code replacement.
Let's focus on tooling and not language. We are 40 years since SmallTalk and yet, not much stands above it. I mean, I still need to build and deploy so much code, without being able to edit and update it in real time.
Absolutely agree with that. If it can't even generate boilerplate reliably and with repeatability, I'd rather type it myself. I'm a pretty fast typist.
I'm using Resharper, does much of what I need. For example, convert a simple property into one with backing variable, find myself doing similar thing like this a lot.
Holly makes some excellent points, but I can't help craving for better static analysis tools that do the job and do it correctly and reliably. The sort of tool where we aren't surprised when they find the correct answer.
AI software development differs from Human software development in that its much faster, and will replace human developers for 3 reasons
Human Developers will think about things, overthink things, underestimate things - then write code and generally make a few a mistakes. Takes a few round trips to get it right, if at all.
AI on the other hand will make shit up, and present it as facts. When corrected, it will make some adjustments, and eventually end up back where it started with the original random muck that never worked the first time around either.
But the AI process is faster than the human process.
AI will replace human developers, because 3 things :
1 - Its faster at generating incorrect code than humans
2 - Thanks to Agile, getting something (anything) "done" by the end of the 2 week sprint is more important that getting anything done properly
3 - Users have been gradually conditioned to accept that their software will be ridden with bugs, and will never actually work the way it was sold to them
That's some grade A post-truth cynicism there. Are you okay?
@@royetter1777He spittin' fire in the language of FAXX. Behind every piece of cynicism is an opportunity to fix as much as we can for a better tomorrow.
It might be interesting to start with the codebase each time a change is needed, then ask it to generate BDD for the interesting part of the codebase likely to need changes, then tweak the BDD it generates, then ask it to make corresponding changes to the code.
So as an anecdote, 6 months ago the coding AIs could write good python and javascript, but they sucked at Rust, because they couldn't understand the borrowing rules. I was thinking that they probably wouldn't ever grok Rust, as it has complicated correctness rules. As of November, all the coding AIs generate passable Rust that is borrow-safe. They also regularly push back against buggy code, suggesting fixes from just reading it. So, I think while the tech is young and buggy and incomplete, it will very obviously get there. However, for interesting code, I still see the future being based around buddy coding with the AI as your buddy, and then having it generate all the boilerplate and plumbing from text specifications. For large chunks of software, I expect that these specifications will be the "source", moving forward.
Possibly, though the rate of improvement has slowed. Still, the code is not exactly the issue for me, it's the reliability and reproducibility. You wouldn't insert a "prompt" component when the code it generates is different every time it runs.
@techsuvara maybe. Or perhaps it doesn't matter as long as over time you've described all the things that do matter in your prompt?
1:20 Almost like an ever evolving ad-hoc language. Why wouldn't we have the need to parse pictures, video, and audio? Parsing vague text that AI is supposed to "get the vibe with 100% or even 60% accuracy" is already impossible unless the prompt is so specific that we're just writing code at that point.
Better informing us on what the code already is would be more feasible for big projects, but the context window is too small for even that use case to be viable for large legacy code.
Much like the singularity, the premise that AI can create something new, much like evolution, requires a system to increase in complexity without an input in information and can’t occur.
Why can't an AI "evolve"?
@ because if it is the same program and it prioritizes b over a because it is “learning” it isn’t evolution, but if it can’t magically rewrite itself into a new program by itself, it is like trying to pick yourself up by your own boot straps.
And it isn’t any better if you have Simple AI try to rewrite Simple AI(1) into a better Smart AI, because Simple AI code isn’t smart enough to come up with the code for Smart AI.
It is an accepted fallacy that bacteria1A became an ape over, so Simple AI can become Smart AI, but not only is the premise false, even if it weren’t, Life and AI aren’t the same type of complexity.
@@jsbrads1 well AI can write quite good code. And it doesn't need to bootstrap itself. Humans are bootstrapping AI, and it is getting quite close to the point that it will be able to write itself
@ The last AI we create won’t be smart enough to create something smarter than it.
@@jsbrads1 good argument...
Definitely not there yet. I'm typing away, and it offers me the next line and it's correct. Carries on correctly for half a dozen lines, then out-of-the-blue, it generates absolute garbage, away with the fairies. If it were combined with some kind of logic, might be usable.
I've tried it to develop algorithms, just hopeless.
The problem is when AI is hiding away part of the "Essential complexity". To borrow the words from "out of the tar pit"
AI will make every programmers job easier, without a doubt. There's no programming task you can train a human to do that you couldn't train a model to do. The only question is what programming will look like when AI can write and rewrite your app in 5 minutes. Don't trust the AI code? Write tests.
it can’t even get css correct. it purely gives you back the weightings in what it predicts is next. it will be useless across the board
I would argue that getting CSS correct would be quite an achievement. Most human developers can't get CSS correct. Though it's kind of debatable what does it even mean to get CSS correct. Just ask someone how to center a div. :D
@JanVerny it means logic. humans can learn and be 100 correct end of story.
Have tried so many AI tailwind tools. All worth their weight in uselessness.
Dave and guest, you're not thinking far ahead. Programming language mediate between humans and electrical signals and provide humans with the ability to create instructions to a "computer" that eventually regulates electrical currents. It's slow, laborious, fragile and error prone.
Pure conjecture and dependent on risk tolerance for AI .... and I hope I'm wrong ... IMHO AI will take the place of programming languages and humans will interact through natural language. In other words, programming languages will largely go the way of the dodo bird.
Machines aren’t built on natural language but mathematical logic…
Natural language is far more unstructured, full of contradictions and subtle hidden meanings full of human emotion and nuance.
it can never work. inferences will never be as good as human logic. saying it will work even for medium complex projects is just wrong
I'm old enough to remember "computers will never be as good as human grandmasters in chess" and later "yeah, computers beat humans in chess, but they'll never be able to pass as human in a conversation"
Just because the tech isn't there yet doesn't mean it won't get there, and at the current rate we can imagine it surpassing humans before the end of this decade
@@neko6 that’s completely false. Just because a computer can beat someone at chess? Sure the computer can beat me at fifa. But it cant design itself. It’s never going to happen.
Actually it does because its limited unlike humans. Do you know how it works? It guesses
Fundamentally what is human logic? How does your brain decides if A follows from B? The answer is someone has shown you some examples and you created a statistical model. This is at least in principle the same thing we are doing with the AIs, albeit our current approach is definitely too primitive. I see no reason why this could never work...
@ its not even close. Then you do not understand that statistical model is not logic and never will be. Ive used this so called AI daily and have 21 years programming experience and a masters in computer science specifically the singularity. I can tell you its never going to happen
@JanVerny human logic is not a statistical model. Thats just incorrect