Cole, you missed an super unique feature that do with ChatLLM everyday: the advisory council feature. Abacus doesn't call it that but I do. It is the ability to chat with multiple LLMs at once. Start with LLM#1 > get response, then switch to LLM#2 and say "anything missing?" and LLM#2 has the chat history for context to push the answer further. You can't do this anywhere else and it gives you the ability to get an cross AI-enabled consensus as well as the most complete answer to your question.
Thank you! They are very similar tools - I've personally used Chat LLM more though just because their routing feature (taking a request and picking the right LLM) has worked incredibly well for me.
It looks good but I would be skeptical. Searching reddit and youtube there are many comments suggesting that they hit rate limits quickly and support are very vauge about this. In the footer their company section all points back to the main page, I cant see any terms of service or more details about the company. I am just cautious that things just dont seem 100% right maybe its just me. Appreciate your content though. I will keep my eye on this to see if I get any follow up responses.
I was searching through their site and you are limited to 2M tokens across all llm’s. Honestly not sure if that’s a lot but with gpt4 on api it’s pennies for 1M tokens. Correct me if I’m wrong please.
Dang my comment got deleted?? That's not cool, I'm seriously a supporter of Cole and what I said was legit and if it was wrong I asked to be corrected but censorship is to followers is not cool man.
Totally fair, but I haven't hit any rate limits myself! Ive enjoyed using the platform a lot even though I have seen some similar things on Reddit/RUclips. Possible it wasnt as good at one point but they made it better!
@SouthbayCreations I'm sorry your comment got deleted! I definitely didn't and don't have any moderators, so I am not sure what happened. I know sometimes RUclips deletes comments automatically sometimes and it can seem random. 2M tokens is dozens and dozens of messages but still not a ton considering the API pricing like you said. Where did you see that on the site?
@@ColeMedin You need to get past the part where the keep asking for your CC# and go to your account and click billing details. I took a screenshot of it because it would auto redirect back to enter my CC#
When testing perplexity or "you" that also offer multiple LLM's I noticed that there was a huge difference using for example GPT4 there than using it with CHATGPT. Even though you are using the same model it's not the same chat logic. The same happens using tools like anythingllm.
@@tiagovasc You can use it as a playground and see the results for yourself. I encountered issues where, after just two or three answers, perplexity, for example, lost the topic and started providing responses without the initial context. I tested this with pro accounts and the same questions. This was 3-4 months ago; perhaps things have changed since then. Also, an issue with multiple LLMs Saas will be that they will always lag behind, and promised features like prompt caching posibly won't work with multiple LLMs anyway. Besides that, you never know how many tokens are actually available, as it seems that this type of issues could be the bottleneck of these solutions. The only thing that makes this chat LLM interesting is the price. For many use cases, it is still very useful. And than what about privacy concerns.
@@tiagovasc problem is ChatGPT is not the same as GPT4, so what you get from the API is not what you get when you chat directly with ChatGPT using the interface. With Claude there is even more to it, because I do not think this ChatLLM Teams supports the Artifacts or Projects feature. Sure I guess "chat with large documents" given some RAG solution can do something similar for you but... Artifacts and Projects for me replaces most the tools he talked about using. The framework around these LLMs are very important. Then there is Perplexity which has some really good search features, it does not replace that either and what about ChatGPTs new Artifacts like feature? What about o1 and being able to see its reasoning steps? Edit: Okay it does at least seem to have an artifacts like feature. I guess I will have to try it out to see how well it works. Edit 2: Well, it can run code in the browser, webcode, but its not as good as artifacts. Does not update the same code and show all the versions. But it is very nice how you can switch between so many LLMs, and it has some very advanced features, like agents, video generation and more
@ricozuares and @KasperSOlesen I really appreciate your thoughts here! You both are right with the downsides to using a platform like this, and I appreciate you both saying at the same time there is a place for it because of the convienience!
Hey @ColeMedin, Love what you're doing. Not sure how to get a feature request to you but here are my two suggestions. The ability to upload images so AI can use it as a guide is hugely beneficial. I see it's on the priority list. The other feature is to add internet access.
Hi Cole, thanks for the cool alternative. 👍The only thing that is a bit confusing is the use of the Agent Platform. 🤔I don't find it very tidy / clear and the documentation isn't really helpful either. I wanted an agent that gives me information about a table, but even the AI Engineer fails. Potential for frustration. 🤨
You bet!! Their agent platform is in beta so they are still working on fleshing it out! I agree that it isn't super user friendly yet but hopefully it will be soon!
That's awesome, I'm glad you're enjoying it! I'm not totally sure how to integrate with n8n but ChatLLM has an API so theoretically it is possible! But it's not one of the default supported AI models for the agents in n8n unfortunately.
I have been testing this a bit. Very interesting service for sure. Annoying that the Playground and Code feature are not one feature like Claudes Artifacts. When you use Playground you get the full code readout in the response instead of the playground being a possible tab for the code window. Claude really did a great job with that artifact feature I think. Still this service has a lot of great things to offer, especially that all of this works with so many LLMs. I tried the video generation, its nice to have that there as well. The custom agents and projects with multiple agents feature seems very interesting. I will have to play around with all of this. But when it comes to code I think I will stick with Claude for now. But it does sound like they keep adding features, so I will have to keep an eye on this. They might very well improve on the coding / playground feature. Have you made any videos going into those advanced agent features? Seems you can make a project and create several agents for it.
Indeed! There is a time and place for local LLMs and a time and place to use a service like this, as I mention in the video! For just bouncing ideas off of LLMs (not creating agents), I love using ChatLLM! But coding custom agents and things I love using local LLMs.
That's the million dollar question! It's because the use the API under the hood and rely on the fact that most people won't use the LLMs as much as they could with their generous rate limits.
Your video seems to be more informative than other on this site. However: how many compute points does a message with say Claude 3.5 Sonnet cost? I heard you get 2.000.000 compute points but I see no table what those amount to per LLM. They also apparently have rate limits on how much you chat there. Can you say something about that?
@@ColeMedin Insanely good is quite subjective. So they offer 2M compute points, but how much compute points does each LLM cost? Are they variable compute point costs per token use?
Seriously, bro! I just signed up to OpenAI's pay as you go plan and now this. I felt betrayed. I'll try gpt for a month and then will check this out. I'm still working on your other tutorials anyways and my son won't let me use MY pc atm. One question though, why is OpenAI's API keys very long, 165 characters? Is this normal?
I think they have their own APIs to all these different models and that comes to be a lot cheaper than paying for ChatGPT. So they built the platform, and structured their business that way. It’s not uncommon. Think cell phone carriers who also sell Netflix, Hulu subscriptions if you buy their tv service. Not that much different
I think they rely on most users not using up too many tokens on average. I rarely hit the limits Claude or ChatGPT has every 3 hours or whatever it is, but the API calls if used directly are not that expensive. Claude actually does not give you an exact amount of queries, they seem to give you tokens, so they also warn you that if you give Claude AI really large documents and / or prompts that will make you run out of the token limit you seem to have faster. ChatGPT has just set a set amount of queries which might be in their favor and it might not. But with Claude it means you will be able to make a lot of simple prompts and probably wont run out of queries / tokens, but you will end up having a lot fewer queries if you end up making really long chats, that makes the input tokens a lot larger.
I know this varies by service and potentially by model depending on their arrangement with the different providers, but how can I see what data is being shared and stored by these services? What are ways I can harden my system usage and reduce attack surfaces? Any help or direction would be greatly appreciated.
I appreciate you asking! Though I am not entirely sure how to answer, because really any data or prompts you put into the system is going to be stored in these services. So any documents you upload will be stored in ChatLLM. Or are you asking something else?
comparing 01 preview directly vs abacus o1 preview 15.00 per 1M input tokens 7.50 per 1M cached* input tokens 60.00 per 1M output** tokens on abacus i used i used input 39,000 characters ≈ 9,750 tokens o1 preview output 68,000 characters ≈ 17,000 tokens thus x 10 it seems i would run out after roughly 267,500 tokens for 10 dollars so lets say we 10 x 17,000 for output = 170,000 now we take 1,000,000 divide by 170,000 = 5.88 and we x that by the 10 bucks we paid we get around 58.80 basically 60 bucks. I butchered the estimate but it seems in the ball park of 60.00 dollars 1M output tokens lemme know if im out to lunch on this.
There was another comment that mentioned it was a scam service with a small context window, was that person trolling you Cole? This definitely looks interesting and worth trying out
@@ColeMedinI’ve been using this service since July, and never looked back. It’s amazing for the price, and not a scam at all. And you didn’t even mention chatbots, ai agents, recent integration with GitHub… It’s an amazing tool.
![Noob To Max With DRAGON REWORK In Blox Fruits [FULL MOVIE]](http://i.ytimg.com/vi/LnBMOinoOvA/mqdefault.jpg)
Cole, you missed an super unique feature that do with ChatLLM everyday: the advisory council feature. Abacus doesn't call it that but I do. It is the ability to chat with multiple LLMs at once. Start with LLM#1 > get response, then switch to LLM#2 and say "anything missing?" and LLM#2 has the chat history for context to push the answer further. You can't do this anywhere else and it gives you the ability to get an cross AI-enabled consensus as well as the most complete answer to your question.
I agree I should have covered this feature for sure, it is fantastic!! Thank you for calling it out!
Not diamond has similar features and for free
@@akashpaul4248 Nice. Didn't know about Not Diamond. Open source options coming along fast!
Haven't heard of not diamond, thanks for sharing!
This.
This is a great LLM Cole, thank you for sharing this video. How would you compare this with myninja LLM?
Thank you! They are very similar tools - I've personally used Chat LLM more though just because their routing feature (taking a request and picking the right LLM) has worked incredibly well for me.
It looks good but I would be skeptical. Searching reddit and youtube there are many comments suggesting that they hit rate limits quickly and support are very vauge about this. In the footer their company section all points back to the main page, I cant see any terms of service or more details about the company. I am just cautious that things just dont seem 100% right maybe its just me. Appreciate your content though. I will keep my eye on this to see if I get any follow up responses.
I was searching through their site and you are limited to 2M tokens across all llm’s. Honestly not sure if that’s a lot but with gpt4 on api it’s pennies for 1M tokens. Correct me if I’m wrong please.
Dang my comment got deleted?? That's not cool, I'm seriously a supporter of Cole and what I said was legit and if it was wrong I asked to be corrected but censorship is to followers is not cool man.
Totally fair, but I haven't hit any rate limits myself! Ive enjoyed using the platform a lot even though I have seen some similar things on Reddit/RUclips. Possible it wasnt as good at one point but they made it better!
@SouthbayCreations I'm sorry your comment got deleted! I definitely didn't and don't have any moderators, so I am not sure what happened. I know sometimes RUclips deletes comments automatically sometimes and it can seem random.
2M tokens is dozens and dozens of messages but still not a ton considering the API pricing like you said. Where did you see that on the site?
@@ColeMedin You need to get past the part where the keep asking for your CC# and go to your account and click billing details. I took a screenshot of it because it would auto redirect back to enter my CC#
Woow been following you Cole and now first comment. Keep bringing the content
damn. beat me to it
I sure will, thank you!
When testing perplexity or "you" that also offer multiple LLM's I noticed that there was a huge difference using for example GPT4 there than using it with CHATGPT. Even though you are using the same model it's not the same chat logic. The same happens using tools like anythingllm.
Most likely just driven by a bunch of safety features in the system prompt of gpt. Doesn't mean the results are worse.
@@tiagovasc You can use it as a playground and see the results for yourself. I encountered issues where, after just two or three answers, perplexity, for example, lost the topic and started providing responses without the initial context. I tested this with pro accounts and the same questions. This was 3-4 months ago; perhaps things have changed since then. Also, an issue with multiple LLMs Saas will be that they will always lag behind, and promised features like prompt caching posibly won't work with multiple LLMs anyway. Besides that, you never know how many tokens are actually available, as it seems that this type of issues could be the bottleneck of these solutions. The only thing that makes this chat LLM interesting is the price. For many use cases, it is still very useful. And than what about privacy concerns.
@@tiagovasc problem is ChatGPT is not the same as GPT4, so what you get from the API is not what you get when you chat directly with ChatGPT using the interface.
With Claude there is even more to it, because I do not think this ChatLLM Teams supports the Artifacts or Projects feature. Sure I guess "chat with large documents" given some RAG solution can do something similar for you but... Artifacts and Projects for me replaces most the tools he talked about using. The framework around these LLMs are very important.
Then there is Perplexity which has some really good search features, it does not replace that either and what about ChatGPTs new Artifacts like feature? What about o1 and being able to see its reasoning steps?
Edit: Okay it does at least seem to have an artifacts like feature. I guess I will have to try it out to see how well it works.
Edit 2: Well, it can run code in the browser, webcode, but its not as good as artifacts. Does not update the same code and show all the versions. But it is very nice how you can switch between so many LLMs, and it has some very advanced features, like agents, video generation and more
@ricozuares and @KasperSOlesen I really appreciate your thoughts here! You both are right with the downsides to using a platform like this, and I appreciate you both saying at the same time there is a place for it because of the convienience!
Hey @ColeMedin, Love what you're doing. Not sure how to get a feature request to you but here are my two suggestions. The ability to upload images so AI can use it as a guide is hugely beneficial. I see it's on the priority list. The other feature is to add internet access.
Love the suggestions, thank you! What would you see internet access helping the most with?
You are the Man.....🐐🐐🐐
Hi Cole, thanks for the cool alternative. 👍The only thing that is a bit confusing is the use of the Agent Platform. 🤔I don't find it very tidy / clear and the documentation isn't really helpful either. I wanted an agent that gives me information about a table, but even the AI Engineer fails. Potential for frustration. 🤨
You bet!! Their agent platform is in beta so they are still working on fleshing it out! I agree that it isn't super user friendly yet but hopefully it will be soon!
Cole thanks for your work and efforts you are putting in bolt ai. Can you advise can I use abacus ai within bolt.new. Thanks
You bet!! I don't think Abacus AI has an OpenAI compatible API so it would be pretty tough to integrate with Bolt.new/oTToDev
Thank you for sharing I registered today and loving it. How I can use this LLM in n8n workflow?
That's awesome, I'm glad you're enjoying it! I'm not totally sure how to integrate with n8n but ChatLLM has an API so theoretically it is possible! But it's not one of the default supported AI models for the agents in n8n unfortunately.
Nice one this is a perfect combination!
It really is! Thanks!
I have been testing this a bit. Very interesting service for sure. Annoying that the Playground and Code feature are not one feature like Claudes Artifacts. When you use Playground you get the full code readout in the response instead of the playground being a possible tab for the code window. Claude really did a great job with that artifact feature I think.
Still this service has a lot of great things to offer, especially that all of this works with so many LLMs. I tried the video generation, its nice to have that there as well. The custom agents and projects with multiple agents feature seems very interesting. I will have to play around with all of this.
But when it comes to code I think I will stick with Claude for now. But it does sound like they keep adding features, so I will have to keep an eye on this. They might very well improve on the coding / playground feature.
Have you made any videos going into those advanced agent features? Seems you can make a project and create several agents for it.
Thanks for your thoughts here Kasper, your plan sounds good! I haven't made a video on the agent features in ChatLLM yet, but I might in the future!
Also cNian we use custom gpts extensions ?
If I pay for that, can I connect to Cursor with a API Key just like with normal GPT?
Great question! I think it would require a more custom setup, but ChatLLM does have an API so theoretically it is possible!
Cole, weren't you build these llms locally on you computer the other day???
Indeed! There is a time and place for local LLMs and a time and place to use a service like this, as I mention in the video! For just bouncing ideas off of LLMs (not creating agents), I love using ChatLLM! But coding custom agents and things I love using local LLMs.
How is this possible to pay less while the others who are the inventors charge more ?
That's the million dollar question! It's because the use the API under the hood and rely on the fact that most people won't use the LLMs as much as they could with their generous rate limits.
can i connect it to visual code and does it have there own api so i can connect my llmlite
Great questions! They do have an API and developer SDK so theoretically it is possible! I haven't looked into doing it myself yet though!
Your video seems to be more informative than other on this site. However: how many compute points does a message with say Claude 3.5 Sonnet cost? I heard you get 2.000.000 compute points but I see no table what those amount to per LLM. They also apparently have rate limits on how much you chat there. Can you say something about that?
They do have rate limits but I've never hit them and I know they are insanely good!
@@ColeMedin Insanely good is quite subjective. So they offer 2M compute points, but how much compute points does each LLM cost? Are they variable compute point costs per token use?
Is there a free self hosted thing that does the same thing and I can use my own api keys?
Nothing as good as ChatLLM but there are projects out there like Route LLM I would check out!
lmsys.org/blog/2024-07-01-routellm/
N8n usable ?
In what way?
Seriously, bro! I just signed up to OpenAI's pay as you go plan and now this. I felt betrayed. I'll try gpt for a month and then will check this out. I'm still working on your other tutorials anyways and my son won't let me use MY pc atm. One question though, why is OpenAI's API keys very long, 165 characters? Is this normal?
Sounds like a good plan!
Where are you seeing that long OpenAI API key? It shouldn't be that long!
@@ColeMedin whenever I create an API key. I still haven't checked if it's working though, will try first. Thanks!
Sounds good, you bet!
how is this economically viable for them?
I think they have their own APIs to all these different models and that comes to be a lot cheaper than paying for ChatGPT. So they built the platform, and structured their business that way. It’s not uncommon. Think cell phone carriers who also sell Netflix, Hulu subscriptions if you buy their tv service. Not that much different
I think they rely on most users not using up too many tokens on average. I rarely hit the limits Claude or ChatGPT has every 3 hours or whatever it is, but the API calls if used directly are not that expensive. Claude actually does not give you an exact amount of queries, they seem to give you tokens, so they also warn you that if you give Claude AI really large documents and / or prompts that will make you run out of the token limit you seem to have faster. ChatGPT has just set a set amount of queries which might be in their favor and it might not. But with Claude it means you will be able to make a lot of simple prompts and probably wont run out of queries / tokens, but you will end up having a lot fewer queries if you end up making really long chats, that makes the input tokens a lot larger.
A very good question! The other replies to your comment hit the head on the nail 👍
I can't find a limit, that's wild
Yeah it is!
I know this varies by service and potentially by model depending on their arrangement with the different providers, but how can I see what data is being shared and stored by these services? What are ways I can harden my system usage and reduce attack surfaces? Any help or direction would be greatly appreciated.
I appreciate you asking! Though I am not entirely sure how to answer, because really any data or prompts you put into the system is going to be stored in these services. So any documents you upload will be stored in ChatLLM. Or are you asking something else?
is there any limit for how many chats we can do?
There is a limit but it's VERY generous!
Does it have APIs?
Yes it does!
So basically just Mammouth AI.
how is that? is it worth going that route instead of playing chatgpt seperately?
Similar but I like the features of Chat LLM a lot more!
comparing 01 preview directly vs abacus
o1 preview
15.00 per 1M input tokens
7.50 per 1M cached* input tokens
60.00 per 1M output** tokens
on abacus i used
i used input 39,000 characters ≈ 9,750 tokens
o1 preview output 68,000 characters ≈ 17,000 tokens
thus x 10 it seems i would run out after roughly 267,500 tokens for 10 dollars so lets say we 10 x 17,000 for output = 170,000 now we take 1,000,000 divide by 170,000 = 5.88 and we x that by the 10 bucks we paid we get around 58.80 basically 60 bucks. I butchered the estimate but it seems in the ball park of 60.00 dollars 1M output tokens lemme know if im out to lunch on this.
That sounds about right!
hum are you saying that it ends up more expensive than 01 preview?
I thought he was saying it ends up being cheaper than using the API for o1 but maybe I misread it!
@@ColeMedinyeah the wording is unclear haha 😅
How it compares to boxchat?
Similar for sure, but ChatLLM has much better pricing and my features that I've enjoyed!
I am using Anything LLM GUI interface and free on Linux.
Yeah Anything LLM is fantastic!
How many VRAM do u have ?
There was another comment that mentioned it was a scam service with a small context window, was that person trolling you Cole?
This definitely looks interesting and worth trying out
Seemed like it, but I know some people have that concern even though I haven't experienced it myself!
@@ColeMedinI’ve been using this service since July, and never looked back. It’s amazing for the price, and not a scam at all.
And you didn’t even mention chatbots, ai agents, recent integration with GitHub…
It’s an amazing tool.
That's fantastic Antonio, thank you for sharing!
What's the difference? Getmerlin is already providing. Can you explain?
They are similar but ChatLLM is half the price and has some nice features that Merlin doesn't have like video generation/comprehension!
It is free bro ?
Not free but it's $10/month, much more affordable than using other services!
@@ColeMedin how chat llm manage all gtps it's fake gtps ?
Sorry could you clarify?
😎
I just signed up and they don't even have gpt 4? Wtf
They have GPT-4o and GPT-4o-mini! Were you referring to GPT-4-Turbo?
🎉🎉🎉🎉
Killer
how to get chatgpt voice?
You can record audio and upload it into the platform! Otherwise there isn't a way to chat with voice directly right now