How to get LIVE cryptocurrency data with Python using the Binance API / ERROR FIXING
HTML-код
- Опубликовано: 12 окт 2021
- This video is about getting live cryptocurrency data via Python using the Binance Socket Manager. It is just a solution to an error message a lot of you have experienced.
Please provide me feedback if this is solving your error message! I would highly appreciate it!
Source:
python-binance.readthedocs.io...
#Python #Crypto #Live
![How To Build a Cryptocurrency Live Trading Bot with Python using the Binance API [SMA Crossover]](http://i.ytimg.com/vi/aWgJYd5kVfo/mqdefault.jpg)
![How To Build a Cryptocurrency Live Trading Bot with Python using the Binance API [SMA Crossover]](/img/tr.png)
Hi bois and girls,
PLEASE READ:
I dedicated some time to further investigate on this issue. Before you apply the createframe function you have to make sure you really received a message.
You can do that easily by just applying a simple if condition after res = await tscm.recv()
Please provide me feedback, if it works for you.
Example:
async def main():
client = await AsyncClient.create()
bm = BinanceSocketManager(client)
ts = bm.trade_socket('BTCUSDT')
async with ts as tscm:
while True:
res = await tscm.recv()
if res:
print(createframe(res))
await client.close_connection()
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Sorry but this doesn't solve the "queue overflow" error, I still have it. I'm python dev, we can talk about this?
First of all thank you for all you do. It has been a great help to me, I only started learning programming 1 month ago. This solution fixed the download problem, but now I want to upload it in the sql db. But it doesn't work. I'm trying to implement some steps from your video "How To Build a Simple Cryptocurrency Live Trading Bot with Python using the Binance API" but that doesn't work for me. I watched different video from you about SQL, but that also didn't work. Under the code I added the lines: createframe(msg) and engine = sqlalchemy.create_engine('sqlite:///BTCUSDTstream.db'), this didn't work. Then I added the lines: frame = createframe(msg) and frame.to_sql('BTCUSDT', engine, if_exists='append', index=False) after while True: in the async def main():. This also didn't work. Could you please help me fix this problem? Keep up the great work 👍
What is your version python sir?
I LOVE u and ur Cripto series! 🌵 Thanks !
Thanks buddy 🌵
6 Days ago, i woke up, saw on of your videos, today i have a bot and i keep adding layers for optimizing, and im trading , earned 2 usd and just runing test, prob was just luck , but in 3 or 4 das more , this thing will help me a lot.
Thank you for inspiring and been a good mentor,
Thank you bro, muchas gracias hermano, has cambiado una vida.
That's great to hear! Would be interested if you were able to make profits over the longer term.
Thank you for providing an update to your tutorial. I’ve tried this approach and the data stream loads perfectly. I appreciate that you are providing support and great content to the community.
Thanks for you feedback! Highly appreciated.
Thank you! This works for me. It prevents the error I was getting with the sqlalchemy data base. Where it'd generate frames for nearly a minute, but then stop and give me KeyError: "None of [Index(['s', 'E', 'p'], dtype='object')] are in the [columns]"
Hi buddy, please check my new pinned comment. That should finally solve the issue.
Thanks for the update - the data loads perfectly now.
Awesome. Thanks for your feedback! :-)
Thank you for providing such good content and addressing feedback/issues of your viewers! I think the topics you've made content on is really captivating and has given me the motivation to start learning python. Really like how you started with videos on fundamental mechanics and have progressed to adding more analysis.
Thank you so much for the feedback. That's awesome to hear!
Thanks a lot for the great content. I am looking into your stuff now for 6 days straight and am quite amazed by my own learning curve given your videos. I am a python beginner with some knowledge in data structures and "hndling" only so this is great
Thanks a lot for checking out my stuff man. Really appreciate it!
Hey Algovibes, habe erst vor kurzem deinen Kanal entdeckt und finde die Videos echt super lehrreich und interessant! Auf diese Art und Weiße funktioniert der Datastream wieder einwandfrei, nur bekommt man die Daten in viel zu hoher Frequenz. Hast du da schon eine Methode rausgefunden, wie man das regeln könnte? :) Außerdem würde mich mal interessieren, ob du schon eine Trading Strategie gefunden hast, welche wirklich dauerhaft besser performed als die Performance des Assets? Wenn nicht, werde ich auf alle Fälle selber weitersuchen :D
Danke Dir :-)
Mich stört die Frequenz nicht. Du kannst aber einfach ein sleep einbauen (from time import sleep) und dann im while loop ein oder mehrere Sekunden einbauen. Dauerhaft besser performed -> nein. Ich spiele damit rum und lass ab und an mal bots laufen - einige laufen gut, andere weniger.
Ich investiere (nicht trade) seit Jahren in cryptos. Es sind Spielereien - mehr nicht.
@@Algovibes ach ja so werde ich das mal ausprobieren. Von deinen Spielereien kann man aber auf alle Fälle viel lernen, danke dir! :)
I've been looking into using RL in a testing environment with a live data stream. I was curious about your opinion or insight into that. Great videos!
Awesome. Thanks a lot for your feedback :-)
Excellent !
Thx a lot !!!!!
Work without any errors for me ;-)
awesome, thanks a lot for your feedback!
Hello men, i got some issues when i try to run it with:
def createframe(msg):
df = pd.DataFrame([msg])
df = df.loc[:,['s','E','p']]
df.columns = ['symbol', 'Time', 'Price']
df.Price = df.Price.astype(float)
df.Time = pd.to_datetime(df.Time, unit='ms')
i just put it and appears
NONE
NONE
NONE...
i[m using VSC and also a question should not be res instead msg in createframe function?
Hi buddy,
this comment didn't show up for me. Sorry!
msg is just the argument of the function. If you put in res that's perfectly fine.
Were you able to solve the problem?
Some great videos, very nice, subbed!
Can you do a video on how to detect (hidden) divergence in Python?
Thank you man! Appreciate your support. Also thanks for the suggestion!
Thanks! it work for me :)
Thanks again for your feedback.
Hi mate! awesome channel! New suscriber here. I've done two bots from your "guides" with no problems, but the third one altcoin one is bugging me!
I have an error like some of the comments there and you directed them here, but I cant figure it out how to implement this in to that code... Some help would be apreciated!
Btw I love all of your videos, keep it up man!
Hi man, thank you very much. Did you check out the approach using the Unicorn library in one of my newer videos?
E.g. here:
ruclips.net/video/Ml6LUH2wxgc/видео.html
Thank you sooo much
Thanks for watching mate :-)
It worked for me, thank you bro
Nice :-) Thanks for your feedback buddy.
The code works fine, thanks for the video.
Awesome. Thanks for the feedback!
Great video. Do you know if it's possible to add a SL or TP to a futures order on binance?
Thanks mate. On the platform Binance itself? I am not quite sure.
useful video, thanks for posting!
Thanks for your feedback! :-)
When running the code I received a ImportError. The code could not import name AsyncClient from Binance
But did you install the python-binance lib?
Hi Bro, I have two error -module 'asyncio' has no attribute 'apply' ?? And the same probleme when I try to run the last loop : RuntimeError: This event loop is already running ?? I use Python 3.7. Any sugestions ? Thnx.
Sorry it s just working now. wrong usage !!! 😉
Thank you, this is working great for me. No errors.
I am uncertain now how to leverage this to use it on the RSI, stoic, MACD code. I have been at it for 2 weeks. Yes, I have never seen code before November 2021.
Also, I was thinking that the RSI, stoic, MACD could benefit from the MTF EMA, where one line is 15 minutes 50, the second line is 60 minutes 50. Any documentation I can use. Thanks.
Hi Alfred, thanks a lot for your feedback.
I am planning on release a video on trading technicals with this approach. I have covered it already here:
ruclips.net/video/g04GeHe-dJw/видео.html
and in specific here:
ruclips.net/video/nQkaJ207xYI/видео.html
Let me know if this is helpful for you or what you are still missing.
Works great! However it requests much faster than once per second. What is the best way to limit the rate of requests?
I've tried using the "user_timeout=1" parameter in the Socket Manager function but it doesnt seem to change anything but I may be misunderstanding how the parameter works. The other thing I've tried, and seems to work well enough, is on each message recieve taking the current date time and the last time I recorded data and checking to see if the difference is 1. That seems to work well but does add some discrepancies. Thank you for the videos!
thank you, yes, that's how it works
Thanks for watching mate. I have shown another more handy approach in my newer vids btw. Be kindly invited to check them out!
Hello, I have the error message:
As of 3.10, the *loop* parameter was removed from Queue() since it is no longer necessary
Are you using Python 3.10 in the above video? If so are there any fixes to this?
Fixes on that would be to use Websocket. I have covered it e.g. here:
ruclips.net/video/mDNIAkEZChg/видео.html
and here:
ruclips.net/video/8p240qonj0E/видео.html
Let me know if this was helpful!
@@Algovibes okay thanks so much! keep up the great content :)
Thank you, Can you try to do a pairs trading strategy on python ?
Thanks for watching and also thanks for the suggestion :-)
How much time you need for getting live data about around 300 crypto? in seconds/minutes
not more than some seconds
Hello, I tried to apply this solution to the tutorial on the other video, but i still get the following error:
{'e': 'error', 'm': 'Queue overflow. Message not filled'}
KeyError: "None of [Index(['s', 'E', 'p'], dtype='object')] are in the [columns]"
Can you please give me some help?
Thanks!
Could you try the following (I am referring to the while Loop in THIS video not the one in the strategy video)
After while True:
put in a sleep function -> time.sleep(2) or any other number of seconds.
Could you do that and try it again and give me feedback? Much appreciated!
@@Algovibes Thanks for the reply, I tried to add the time.sleep but i keep getting the same issue, I am trying to merge both solutions in order to have it working, but not sure why i just can't make this work.
Work!!!
nice, thanks for watching :-)
I have noted that after some time the loop somehow stops and we get no msg. I wonder what is the solution to automatically restart the socket perhaps providing custom time out to close connection and restart. Could you please explain how to do this way?
Could you provide more details? When does the stream stop exactly? It disconnects after 24 hours automatically btw.
Thank you for your videos. I have some issues with this code before. With this fix it working. Can you show live candlestick stream with dataframe.
Awesome. Thanks for your Feedback, Spasibo :-) The Kline stream is pretty straightforward. You just change the socket to:
ks = bm.kline_socket('BNBBTC', interval=KLINE_INTERVAL_30MINUTE)
source: python-binance.readthedocs.io/en/latest/websockets.html
Hello friend! Its been a long time since you posted this content but surely i had to give you a feddback. It worked just fine for me. I had issue with the previous approach but this one is working.
So, i want very much to thank you. First for sharing such a great knowledge for free, and also for keeping it alive by clearing the eventual issue that may occur. God bless you!
Thank you for your kind comment Fred. Appreciate it! BTW I actually recommend using the unicorn - super easy to use and works well on multiple coins - check it out here (and the linked videos below the vid):
ruclips.net/video/hope9jJICRs/видео.html
in python 3.9 I don't have any problem to run the socket stream, but in 3.10 I get the error that there is no running loop. I already tried your fixes in this video. Can you help me to solve this? Thanks for your great videos!!
hmmm...do you have more details on your setup (which IDE,...)?
@@Algovibes I use PyCharm and a virtual environment for the interpreter... I know that something has changed in asyncio with 3.10 but I don't know what and how to fix
Hi nice vid. How to split the frame minute by minute ? Thnx.
Can you check out the videos in the cryptobot playlist please? I was using data manipulations and stuff like that there.
Great videos! I'm getting a "loop.stop ()" problem and I'm getting an error message:
----> 1 loop.stop ()
NameError: name 'loop' is not defined
If I take out the line then the code will run.
Thanks mate. This is not part of the code. Just delete it. I just wanted to check if I can stop the Loop without crashing the kernel (which is another loop running) but I couldn't. Just ignore that.
@@Algovibes ohhh, right!
Thanks!
Thank you, you did so much good work, you never had a desire to create a graphical application so that everything is already in place and a database of quotes for quick analysis and your candlestick charts, and baсktesting strategies and a robot for trading
Thank you mate! Of course I have the desire to build nice stuff. Anyhow I probably wouldn't benefit much of a GUI like you described. Or where do you see the advantages?
Thanks for all your videos! Very helpful and clearly explained!
I've tried this new code and I don't get an error I just get the >>> cursor!! That is all!!
Any idea why that may be?
Running 3.9 and the IDLE on a Mac....
Thanks in advance!
Very welcome!
Not quite sure but I am showing how you are doing it in an IDE here somewhere around minute 9:
ruclips.net/video/nQkaJ207xYI/видео.html
how to draw incoming bitcoin prices in real time on a chart in colab or IDE with different timeframes? Please show it
Does anyone else get the problem that 'DataFrame' has no attribute 'Time'?
No, where did you deviate from the code?
@@Algovibes haha i deviated you are right. I already fixed it. Thanks for answering though!
Loving your content, but I must say my first thoughts as a web programmer by profession when watching your videos was that you are also teaching a lot of people how to hammer the binance api, which I guess could lead to people being temporarily blocked. But I wasn't sure if there was some built-in throttling as I don't use Python much daily. Still, your content is great and I'm learning a lot. I'm trying to get up to speed, but still on your early crypto/python tutorials.
I'd love to see an example of creating a strategy and not creating orders, but actually pretend to do the orders, add them to a sql database, and then being able to simulate how your strategy works on live data, if it made loss or profit, or maybe trying it on a different coin, different time etc. . So you can adjust it, and just duplicate it and make it do the orders after being satisfied.
Thanks for your comment! The streams are the recommended way to pull the data. I did some examples to show the main concepts. Where do you see hammering api calls in this video?
BTW also thanks a lot for your suggestion!
@@Algovibes It wasn't meant as criticising as I'm not sure how it works. Does the socket actually control how many requests are sent to the API? The reason I ask is that I figured a while loop would ask hundreds of times per millisecond otherwise, like it would process as fast as it possibly can and start over :)
@@neXib The way I understand it is that a Binance Client is a persistent TCP connection. From network security perspective, if the script created numerous new connections in the while loop( too many requests per second) Binance API server would consider it as a DOS attack and block the IP the script runs from. it is probably not an issue in this case if all the requested data by while loop is transferred within that single one established TCP connection. Please correct me if I am wrong here about the Binance Client concept
OI muito prazer curti de mais esse tutorial me ajudou muito a esclarece o erro do asyncio, mais eu agora estou com dificuldades de fazer esse mesmo processo em múltiplas moedas 🤖🤖🤖🤦♂
Hey buddy, thanks for your comment. Was using a translator as I am not fluent in Portuguese. You are invited to check out my cryptobot playlist, in specific the newer videos. I am using a livestream on multiple cryptos there - so exactly what you are looking for.
Good job bro, JESUS IS COMING BACK VERY SOON;WATCH AND PREPARE