Holy Grail: FREE Coding Assistant That Can Build From EXISTING CODE BASE
HTML-код
- Опубликовано: 1 июл 2023
- UPDATE: I forgot a step in the install process. After cloning the repo and creating/activating the conda env, you have to install Aider by using “pip install aider-chat” or “pip install -r requirements.txt”
In this video, we review Aider, which is an AI coding assistant that can build entire codebases as well as iterate on existing code! Aider uses Universal CTags to map an existing codebase to provide all context to ChatGPT to help build on that codebase. It's truly amazing. I'll show you how to install and use it.
Enjoy :)
Join My Newsletter for Regular AI Updates 👇🏼
www.matthewberman.com
Need AI Consulting? ✅
forwardfuture.ai/
Rent a GPU (MassedCompute) 🚀
bit.ly/matthew-berman-youtube
USE CODE "MatthewBerman" for 50% discount
My Links 🔗
👉🏻 Subscribe: / @matthew_berman
👉🏻 Twitter: / matthewberman
👉🏻 Discord: / discord
👉🏻 Patreon: / matthewberman
Media/Sponsorship Inquiries 📈
bit.ly/44TC45V
Links:
Aider Github - github.com/paul-gauthier/aider
Universal CTags - github.com/universal-ctags/ctags
Homebrew - brew.sh/ 
Наука
UPDATE: I forgot a step in the install process. After cloning the repo and creating/activating the conda env, you have to install Aider by using “pip install aider-chat” or “pip install -r requirements.txt”
awesome video! do you have a solution for someone on windows10? I can't install brew to get the ctags... alternative ways to install?
😂we wait Till you Made a vscode Extension do Not worry
Thank You so much for this.
I seem to have missed the part about how aider becomes a binary on the path. Is this something that happens on the Mac that I'm not aware of?
on windos how to install ctags shuod it be installed is the same (-env)or sheud I install crome cmd?
Matthew, I wanted to let you know that I, a complete n00b to programming managed to get this working, and have actually gone a few steps extra on the snake game you made and have something that’s actually a little fun to play…
I can’t tell you how grateful I am to the work you are doing, thanks and keep it going! You’re gonna help a lot of people get through these crazy times :)
What's even crazier is that this is all free and open source (minus the OpenAI). Solopreneurship is the future!
The Ctags part was the most interesting part of this, I had never heard of it. That would be a great subject of its own video and compare it to fine tunings and embeddings
Agreed!
Hi matthew. Thx for the amazing tutorial. I just got Aider running on my Windows OS after installing Universal Ctags@@matthew_berman. But the Aider upon initialization, would only say "Repo-map: using 1024 tokens". Could it be that Aider is still using Ctags but just did not explicitly say it or that it is no longer using the Ctags? From my experience, the Aider now keeps "asking for the folder structure and content of files exactly the way a chatGPT would" does that indicate the latter possibility? Thx again!
❤ this was what i was looking for🎉 Please continue this topic do a project with this. Its amazing how far this is going. This can give a lot of material for your channel. If it wasn't for your video i might never have found this.
Makes me happy to hear!
Dude, thank you for finding these projects, that's amazing! Love your channel. 💪
You just gained a new subscriber. Always go straight to the point 😊
Thank you for keeping us up to date with AI. You are awesome.
Super freaking amazing. Got it up and running on my windows machine and also installed ctags. I told it about a pretty complicated program I have always wanted to write and it wrote the main file so we could iterate on it to get all the features right. Can't wait to bang on this for a few days.
this video is EPIC. how do you install ctags on windows? i'm using miniconda.
Ye I'd like to know too
Yes..I'd like to know too, thanks..
I'll have to check this out, it looks interesting. Thanks for the info man!
You got it
Great review of incredible software. Thank you for the consistent content.
As always a great video, good to follow along and very interesting! :)
Very good summary of this new technology. Thanks for sharing.
Feels like it's programming with extra steps. The effort you take to tell it what to do with many iterations could've been used on writing the code yourself.
Very exciting to help me finish some long standing "want to do" projects
Enjoy!
Very exiting. Will help me be unemployed and die pennyless
I actually got bing chat to write snake, and change it to my liking. I did this by first giving me a list of things it needed to code, and then instructed it to execute the steps it gave me.
this is truly amazing in so many levels! thanks for sharing, once gpt 4 16k is released the power of this tool will be enourmous! I've been in the waitlist for a while, hoping to get access soon!
Agreed and thank you!
Finally!! Thanks for making the video ❤
Thank you for introducing me to this project.
Great, I like your picks.
I'm working on code generation feature on my open source project, but without the need to see the code.
I think I'll benefit from aider project. Thanks.
I came across the tool a week ago. Decided to try it out with a new project. Though I installed some frameworks first and allowed it to work with the existing project. In afew days I think I have done weeks of work.
This is unreal. Thanks for the video!
This is really good, havent tried out the complex project feature but so far very promising.
Bro, You the man. Thanx for gathering all that info. help me out allot
This sounds amazing, I can't wait to try it out!
Hope you enjoy it!
Thank you very much. Iterating on existing projet is key!!
When we get proper AI coding and 3D modeling, I may get back into game dev.
The worst part of game dev is the time wasted on thinking how to code something, coding it and having to fix tons of bugs.
The second part is the graphics, you need lots of them and they can take a lot of time, AI art is going into the path of more control so there's not that many sacrifices on originality as before.
The other aspects aren't as bad. Music? I would never replace it. Sound? Very fast to use and reusable. Design? The whole reason why you started being the most fun part.
Yeah, I tried learning game dev but it's just too damn much work especially when you're alone.
I would love to create some 'games' that I can build myself to enjoy and spend some time.
I don't need money, I just want to see my thoughts/ideas come to life.
@@0AThijs It feels like if you want to become a game dev you have to sacrifice your happiness.
There are countless videos on RUclips about how to power through with midproject depression even though you started to be happy not sad.
I actually did make a game and it took 1 year, by the end of it I just wanted it to be over. It was just a 3 level shmup with simple graphics. I still want my ideas to become reality but there's no way I'm returning to that.
Those are all the things that make coding fun 😀 Why even code if you just want some GUI to make generic games without the fine-tuned control that it takes to be truly original? You can't be a very good game developer unless you *like* to code, even when it's NOT game-related. Learn to love coding first. To be a game developer, you have to know how to be a *developer*.
@@hashtagrich I am a coder in a programming course.
I assume you are talking from software coding experience and not game coding, game bug fixing is hell on Earth, and having to code AI for every enemy and the effect for every item escalates quickly.
@bazookaman1353 I worked as a professional game coder for a game console for a few years and it was great, but it is nearly impossible to make money making games these days. I love writing any kind of code though
This is awesome. Can't wait for it to integrate open source models.
Thank you. It's a great tutorial
I made the game tetris with this man! wow this is really cool!
Great! After my holiday, I will check how idiots-friendly it is. I have just a bit of an idea how to program, so it will have a hard time with me. 🤗
Thank you very much for your video!
I'm not sure that it doesn't just hallucinate the latest version though unless it is also using internet. It says latest "at the time of this writing" which sounds either hallucinated or best case taken directly from some random training sources.
Wow worked really well! Some glitches where sometimes instead of adding a change it only adds the change and deletes everything else. But I'm sure we're on the right path to have great ai tools to allow non programmers to make tools and software!
.. and programmers go live on welfare
@@a_mediocre_meerkatnot before we enable non doctors to perform neuro surgeries :)
@@drwhitewash it's a valid point, but i beg to differ.
Development is far less official and institutional than medicine. You have people with unrelated degrees or no degrees at all doing good development work.
So, yeah, i see it affecting developers,
Even if developers will get "boosted" - greedy companies will start downsizing many positions.
Eventually AI's should become good enough so in best case scenario there are some jobs to babysit it, rubber stamping it's code into the code base..
Good luck to us all, I hope i'm an idiot and i'm wrong
@@a_mediocre_meerkat why I mentioned neuro surgeons is because to me all this "citizen developer" hype makes as much sense as a "citizen neuro surgeon".
Both require high level of qualification and skills (institutionalization and rubber stamps are not important). What makes people think one skilled job can be done as a "hobby" besides something else, while another skilled job can't?
The other thing is, that these AI generator examples almost always 1) use Python and 2) develop trivial stuff or stuff that can already be found on multiple blogs.
But that's not what developers usually do in their real job.
@@drwhitewash well,
i think that there is no "citizen developer" or "prompt engineer" - it's either the AI can be trusted blindly or not..
i fully agree about current state of things, i'm more worried of how this may evolve
LLM's right now are not able to actually think and deduce logic, so they are not a threat
I am really worried about them evolving, big tech companies are really invested in making advancements, so yeah, incrementally it might evolve into something that can take the most complex requirement and deduce what it needs to do (until then we will surely have employment value)
To be fair, i think i just described AGI .. AGI scares me too, because it would crush the entire industry .. (and humanity too possibly ..idfk im not sarah connor)
Awesome !! Thanks 👌🏻
This is absolutely stunning, however not sure if I am missing something but could not find actual aider installation 'pip install aider-chat' in the video, I have installed it from the git page. Thanks a lot!
Probably in the readme?
great video as always
I appreciate that
For those on windows 11, at least for me, instead of export use set
jesus christ thank you. got depressed when got error
@@QuantumDynamics seens like it can be different if you run on cmd or powershell as well :P
@@EngineerAAJ right thanks ive been fiddling with it for a few hours. I have basically no experience with this stuff so every bit of info i got helped
Thanks for the tutorial! Which one do you think it's better in your opinion currently? SMOL or GPT-Engineer?
This is best given its use of universal c tags to allow for iterating on existing codebases
To anyone that's wondering, this doesn't work well with larger projects. Some numbers from my C# project:
2000 lines of code is around 15k -> 16k tokens, GPT3.5 max tokens is 16k, GPT4 max tokens is 32k.
Also after adding around 20 or so files into its context the aider command prompt gets incredibly slow and laggy.
So as described in their documentation, aider really only works well if you add a few smaller files into the context. So in practice this is mostly just a convenience tool that saves you time from copy and pasting code chunks into chatgpt, with some autocomplete for your codebase's symbols
Have you attempted to /clear the chat?
Yes, it will result in the loss of chat history, but it does enable you to begin typing as though initiating a fresh request.
It might also be beneficial to begin constructing smaller files/classes because large files don't tend to function well with AI anyway.
We can call this approach as creating AI-friendly code. 😅
Excellent tutorial and library! I was able to get EVERYTHING working. Running a 2021 MBP w/ M1 chip, GPT 3.5 API key, and pasting into PyCharm. Aider completely exceeded expectations. Only challenge was the copy paste...Aider outputs the code with a '+' symbol at the start of each line of code. My time is now spent deleting those and fixing indents unless someone has a suggestion on how to fix that 😂
You are doing something wrong, not sure what. What you are seeing is a DIFF, that's the 'visual output', the + means "added", a - would mean "removed". However, the actual file should be changed for you already, there should be no need to do it by hand. To be exact, after the +/- display, it should say "Applied edit to ".
@@WalterStroebel Thanks! I’ll give that a review and see what happens!!
That looks perfect. Like you said, using gpt4 on your coding projects. Is there a way to find out how much your usage is costing from aider ?
I fear it'll be costly, in my tiny experiments its using thousands of tokens for each change. Your milage may vary, seemingly depending on ctags support for your programming language.
Look at your bill at end of month :D
@@ViktorFerenczi That's a bit too late for me
Good question I’m not sure actually
Exactly the comment I was looking for... .03-.12 cents/1k tokens can really add up if it needs all the context tokens + it uses those tokens anyway in its response. But great if you got the $$ lol
Im in my project folder that contains loads of different project files. How do I get Aider to read them all? When I just run aider from the folder its as if its reading nothing.
Did I get the great value version? I have windows and when I tried to do your command desktop mkdir snake it said
I'm sorry, but I don't have the capability to create directories on your computer. Is there anything else I can assist
you with?
You can probably connect Aider to LM Studios API and run everything local
What if I modify the file manually? does it mess with it if it was open and I want to iterate after the manual change?
Just what I've been waiting for!!
I hope a webui will be available to those of us who can't setup locally due to hardware limitations.
??
What hardware limitations do you have?
Lack of internet? Lack of computer? It uses OpenAI GPT models, it doesn't run inference on a edge = no specific hardware requirements
@@matthew_bermanUnfortunately I am still operating on a 32bit Win7 os with 2GB ram.
is there a way to give to aider a specification file with the description of what the project is trying to accomplish? Especially for really new projects that do something that no one else before ever did, it could help.
Hello... I'm a hobbyist python-er and whenever (and I've tried lots of times) I've installed conda , it has arsed up (somehow) any of my python projects... I have normally always just created a venv and used pip - with success... where you demo stuff and use conda, can I instead create venv and use pip to install dependcies instead, or do I have to use conda?
Is this still one of the best for refactoring? Upgrading to Odoo v17 for my ERP, but need to migrate a handful of custom apps. Paying someone is insane. It’s just stupid stuff I could do, but there’s enough I don’t have that kinda time to manually commit. Just silly stuff they should have, like define fiscal position (tax) per category. They have you do it by product. 🙄 That’s insanity! Created an app to handle it. If Aider isn’t the shiznit anymore, what would you recommend?
Does it work with other languages like golang ot javascrip or is it only for python?
Thank you.
Thank you for this. Can you do a windows install of this as well please?
ok this appears to be exactly what i need but i get an error on the key part. is says there is no such command for export
For some reason it shows there is not Universal-ctags directory when i use "winget install universal-ctags" please help.
Can this be used for converting between two programming languages
Sounds interesting. What languages is it good for? I mainly use C++ and C#.
Im having an issue with it where its opening the past github repos I created with it and there doesnt seem to be an option for working with a new instance/git repo. Anyone know how to fix?
This is quite cool, actually.
How do you find the difference in output quality between gpt4 and 3.5x? Im currently on the waitliest like everyone else but i've tried running gpt-engineer with 3.5 turbo and the results were not great. I"m assuming that since Aider uses the same model, the output would be similar?
You can say that GPT4 is richer in experience and gets things done faster in coding terms. It's more optimized and structured. 3.5 on the other hand, is more like a junior coder, not knowing it's issues BUT it can output more Code at once while trying something to fulfill the request.
If you want to have larger code in gpt 4, just shorten your functions and, if needed, leave a comment at the function. This way will boost your coding context as gpt still understands what this is about !!if you use forward function names or have comments on it!! instead of the full code if not needed. May this give you an idea about this 🤷♂️
To be honest takmais I was absolutely delighted when I got access...and for some complex tasks yes, its your man... but two things to note... With a lot of these models I think 80% of the power is in the prompt not so much the model... and on top of that... well GPT 4 is really expensive... like from my bill its like 10 times the price of 3.5. For boilerplate inference its GPT3.5... unless you own a bank man... okay in fairness I went a little berserk when I got access trying it out... but ummm... be careful okay!
I haved fatal experience with Visual Studio 2022 and my C# project and with GPT-4. I have too much bugs and misunderstandings.
GPT-4 gives a good quick start, some skeleton code to fill in. It saves some research time, reading tutorials, watching videos and such. But it cannot replace real developer skills/experience (yet).
Yes, my experience as well. Most likely it is not using ctags as it should on your C# code, I had the same problem with Java. ctags seems to work fine but aider is not using it so chatGPT runs out of tokens big time and starts spitting out nonsense. Especially gpt3.5, gpt4.0 might be better but will still choke on large code bases. And break the bank.
Lol..do you have your API key exposed? 😅 - time to create a new one my friend! Great tutorial as usual.
He always creates new ones 👍🏽
Yep, I always rotate the keys before publishing :)
Can you use it for Flutter or React?
But if we use Claude 2 we have 100k context length so do we really need Ctags in the future?
I am not able to install CTags on windows. Can you please help
You are the man 🫡🫡🫡
Which one of these is the best?
1. Smol developer
2. GPT Engineer (his previous video in which he made the snake game with prompt)
3. Or this AI Assistant?
Opinions would be appreciated :)
Pure speculation but I think it will really come down to this project or gpt-engineer. Smol had crazy potential but I don't see any true value add above engineer.
I would say that engineer's true power come from the complex pre and post processing and tuning allowing you to truly reach the goal of having the project you are looking for. Right now, this assistant is just like magic. I want to say Aider hands down but I would have said the same thing as smol and then engineer and now im saying it about Aider. Everything is so early with these projects, things are changing literally every hour so it's really hard to tell.
I'm leaning Aider because of "raw" power. However, I am not sleeping on engineer - I mean what they have done already with the "steps" you can take to fine-tune, if you will, who knows what they've got up their sleeves. Either way, the fact we're even having this conversation is amazing :)
@@alexcooperstone5530 Thanks for your reply. That was insightful. Btw what do you think, how will these technologies going to help total non-developers to in the process of creating applications and stuff..
Also, Aider is capable of making web and android apps as well as chrome extensions??
None of them, I would say a true KISS standard hasn't been set, even SuperAGI which has lots of neat plugins isn't user friendly. To truly make a mark, they need to be easy to log into or use offline like ChatGPT ==> but fully fledged agents that do exactly as instructed. None of these tools meet this standard so far, they are ALL cumbersome and rife with errors for new users. As you can see, many people who are posting videos like this are quite skilled with coding and even they face hurdles getting these things to work.
Hey, I got stuck at export, it says it's not a command, am I missing something?
can you not open existing projects without GPT-4?
Is it possible to have an assistant like this or like GPT-Engineer that doesn't require an Open AI paid account? Especially because I am very afraid that my credit card could be charged with high costs without being able to control it.
Look what I have found:
Model: Code Interpreter
What version are you?
I am an instance of OpenAl's GPT-3 model, specifically the chat model known as ChatGPT.
The training data I was trained on includes information up until September 2021. Since the model itself does not have version numbers like software might, it's not possible to give a specific version number. Any updates or improvements made to the model since my training data cut-off wouldn't be reflected in my responses.
Model: Plugins • Enabled plugins:
What version are you?
As an Al developed by OpenAl, I don't have a specific version number like software applications do. However, as of my last training data, I'm based on the GPT-3 model. Please note that my training data only includes information up until September 2021, and I may not have the most recent updates or information beyond that date.
Model: GPT-4
What version are you?
lam based on the GPT-4 architecture, which is an iteration of OpenAl's language model. The 4 Q information I am providing is accurate up to September 2021, which is when I last received an update in my training data. After this date, I can't provide real-time data or updates.
Your the best!!!
thanks!
Do a video on how to set up ctags? I tried installing it but I can't set the Repo-map coming on.
Great video, thanks, I'm finding your channel very useful. Do you know of any project / model, that would help me test a website?
Test in what way?
Wish there was a GIST for this.
which batter this one or gpt-engineer ?
How many API calls does aider do? I would love if you could list the costs incured by your commands in the videos!
It would need to load up all your repo codebase and feed it through the API's, so not cheap! So load only what you work on now, and save.
It is costly. 0.35 USD for me and I did barely anything, few files, maybe 2000 lines of input code. Ans that's on gpt3.5, gpt4 should be many times more expensive.
@@WalterStroebel thanks!
does anyone know how to have aider run on other versions of GPT 4 (for example 4-0314)?
Export for the OpenAI API key doesn't work, where do I put it in manually?
You can place your api key in an environment variable:
* `export OPENAI_API_KEY=sk-...` on Linux or Mac
* `setx OPENAI_API_KEY sk-...` in Windows PowerShell
Or you can create a `.aider.conf.yml` file in your home directory.
Put a line in it like this to specify your api key:
```
openai-api-key: sk-...
would anyone know how to use the 32k version in aider? I have access now but not quite sure how to adjust the config
found it : aider --model gpt-4-32k
the holy grail!
Excellent 👍 does it work with v3.5?
Barely but yes.
For future videos can you have these ai models make something more complicated like tetris or something
I really wish that the first expert AI that they built wasn't a coder bot, but instead a lawyer or an accountant.
We won't really need lawyers and accountants then though. We'll need AI that can help us create such a state of abundance for everyone that will make laws and money obsolete.
You gotta try some new examples but this seems somewhat promising with making less mistakes
Instead of snake you mean?
i tried it, it says my api key does not support gpt4... any idea
Really want to get this working, but running into error messages. I installed anaconda, aider, universal ctags and homebrew. When I open aider, I get the prompt as shown, and can interact. However, any command I give it results in a series of retries, then error messages (13 files listed), and then a rate limit error, saying that I exceeded my current quota limit, but when I go to Chat-GPT4, I'm able to interact, so haven't run out of tokens. Anyone help?
this isnt associated with the chat-gpt web interface but rather their api. Have you loaded credits specifically onto your api account?
Hmm
a) My Chat-GPT key seems to be not chat-gpt4 capable.. Model: gpt-3.5-turbo-16k (lame) b) I can't seem to get it to recognize my ctags.. I am running it in the directoiry where the tag file has been generated.. Repo-map: disabled
Great! Is it possible to use gpt 3.5 turbo ?
Yes it is!
There are issues with using 3.5 turbo as it only has a context of 4k tokens. gpt 4 has 32k
@@roscoevanderboom8449 It is visible on the video that it falls back to GPT-3.5-Turbo with 16k context length now. It has been available for a week. Also, not that many people have access to the 32k GPT-4 either, most have only 8k access.
@2:36 - what if we were on a a windows computer?
is it better than smol?
Will gpt4 key produce better results than gpt3.5 key?
Yes
It's soo good to get rid of our jobs. I'm so pumped!
Well, I have close to 3 Tb of models so far, what another few gb's?
This uses ChatGPT so no need to download a model.
@@matthew_berman Oh, then another project that I will get frustrated over trying to change the code to work locally. Need more local models! And if I wasn't doing other stuff while listening to you I would have heard you say this at the beginning :( .
I guess those code are not open src where the ref data? Just saying that where those code come from but I guess it from github and there should be copy right stuff in there some where.
so which cool things have been made with this ?
I would really like to see you do something slightly more ambitious than snake. Can you ask it to create a new game concept? I'm really interested to see it do something novel.
Snake is just obvious and well understood. I could ask it to make a web app of some sort.
@@matthew_berman I totally get why you are using Snake for the general LLM evaluation, but since Snake is something you can find online and would clearly be part of the training data, it's one step removed from "creative coding".
How about dating app for twitter profiles? That sounds weird and random enough.
To marry it with something like WizardCoder and it will be great tool for offline and private use. I don't like the idea of feeding all my sources to OpenAI
Hey what about windows users?? we dont have brew... :(
I've been unable to get aider to work on Windows. Installs fine but the " 'aider' is not recognized as an internal or external command,
operable program or batch file" is all I get. I can't figure out how to use it
You have to make sure you add the aider path to the environmental variables. GPT4 will explain how to set up your dev environment and most other stuff as well.
I had the same, albeit on Linux. Did you run 'pip install aider-chat'?
@@WalterStroebel yes I did.
@@garyjurman8709 I can't figure out what the path is.
@@jimmytaylor1279 My "DOS" knowledge is a bit rusty, basically "set PATH=..." in the prompt window where you want to run aider in. It lives in (HOME)/miniconda3/envs/aider/bin/aider if you follow the instructions on using conda.
any open-source one yet?
It's been 5 months🤔
Does anyone ever say you look like and sound like Paulie Shore( probably miss spelled)? Why wouldn't I use gpt4 instead of their API? Doesn't it cost more to use the API? What are the benefits besides running on a local machine? Enjoyed the video !!
It is exactly a wrapper around the OpenAI API, like a locally running plugin if that makes any sense. Basically it does a lot of copying and pasting. It also uses ctags like a kind of local vector db but that seems somewhat buggy.
@@WalterStroebel thank you for the reply. It was very helpful in my understanding.