Can you use AWS credits for a Claude AI subscription and then use Claude AI with this setup. I recall seeing Anthropic listed which seems to be behind ClaudeAI
great video! Thanks for doing this. After watching and experimenting on my own, what really worries me is the fact that such a simple setup is so slow. 10 - 20 seconds for an answer is a dealbreaker on so many applications. Imagine a chatbot where the user needs to wait a half a minute to get a response if they type "hi". Why is Bedroc so painfully slow? How can anyone sanefully envision building an app with Bedrock given this latency?
how can we speed up the response time? has anyone tried using different models to see what is fastest? i guess prompt engineering can also reduce latency.
Thanks for this! Please do make a video about the one using websocket for the streaming data. It could be more practical for real use case.
Here's the code for those who need it:
import boto3
import json
def lambda_handler(event, context):
bedrock = boto3.client(
service_name='bedrock-runtime',
region_name = 'us-east-1'
)
input = {
'modelId':'cohere.command-text-v14',
'contentType':'application/json',
'accept':'*/*',
'body': "{\"prompt\":\"What is AWS? \",\"max_tokens\":400,\"temperature\":0.75,\"p\":0.01,\"k\":0,\"stop_sequences\":[],\"return_likelihoods\":\"NONE\"}"
}
response = bedrock.invoke_model(body=input['body'],
modelId=input["modelId"],
accept=input["accept"],
contentType=input["contentType"])
response_body = json.loads(response['body'].read())
print(response_body)
+1 for websockets video !!
Fascinating. Would love to see a websocket demo as well
Yes please make a video with websocket and berock!
Can you use AWS credits for a Claude AI subscription and then use Claude AI with this setup. I recall seeing Anthropic listed which seems to be behind ClaudeAI
Amazing!! 🎉 I believe streaming works with JavaScript
+1 for websockets video
where is the git code for lambda ?
+1 for websocket, eventually plain websocket with lambda, not strictly scoped to bedrock
Thanks for the feedback! Here's a plan websocket one I made a while ago using Lambda + API Gateway
@@BeABetterDev yeah, sorry, I figured out after the comment. Thanks anyway
you are badass
love it! So juicy!
hey where is the code :)
Nice
is it only me or code is not in video description?
I also would love to the the websocket implementation
Amazing video!
I would like to see the video about web sockets
great video! Thanks for doing this. After watching and experimenting on my own, what really worries me is the fact that such a simple setup is so slow. 10 - 20 seconds for an answer is a dealbreaker on so many applications. Imagine a chatbot where the user needs to wait a half a minute to get a response if they type "hi". Why is Bedroc so painfully slow? How can anyone sanefully envision building an app with Bedrock given this latency?
How this work if you have your lambda private subnets?
thanks for posting this. Could you add the python code in the description notes?
How can I give it context, with information from a text that I previously extracted with texttract? pls.
yes go ahead and make the video on websockets, you dont need our permission to do so. Permission-less mentality.
Please make a websocket + bedrock, that would be awesome!
how can we speed up the response time? has anyone tried using different models to see what is fastest? i guess prompt engineering can also reduce latency.
Try moving the connection string outside the handler