Finally...Someone gives an informative lecture/demo on Python's logging engine. This was excellent and would have saved me a bit of time and confusion a couple of years ago. I use a separate "logger" for each module for large apps. This way I know what module (and in many cases the class and/or function) that generated the message. The only problem with this type of logging is that the module level logger (by definition) is a "global" variable. Recently I implemented a color-coded console-only logger also. Once you understand the various "logging" components (handler, formatter, name) it's really not that hard to do. Excellent Video !
OK, now I understand why you have 1.1M subscribers: exceptional content delivered exceptionally well. First time seeing your content, instant subscriber. Keep up the good work.
I haven't really bridged the gap into intermediate level projects yet. I've used logging a couple of times before, but I can def see how it would be more useful in larger, more mature, programs. Thanks for the video!
Loguru is my go-to logging module, makes it sooooo easy to define custom levels, multiple logging sinks, and i just parse the data into a logging SQL Server table and it is GLORIOUS
I've been following you since under 100k subscribers... somehow you consistently produce content that aligns with what I am in the process of learning. Thank you! (I'm a former gryphon too!)
By the way, log.log is unchanged not because you are not using it but because you overwrite it with the same log text. All log comes from a logger eventually goes to its parent logger (in this case: root logger).And if you don’t want this behavior, you need to set logger.propagate = False.
thanks for this, you have explained logging in a really simple and straightforward manner. You have probably saved me a lot of effort in trawling through documentation. Much appreciated
Amazing video! I'd like to add, however, that most linters recommend using the 'args' of the logging levels to embed variables. For example: logging.info(f"the value of x is {x}") -> wrong logging.info("the value of x is %d" % x) -> wrong logging.info("the value of x is %d", x) -> correct approach Do note that all three ways shown above do work - it's just that the third method is recommended for use.
The second version is fine, too. Note that the docs state that no formatting substitution is done on the message string if no formatting arguments are supplied. Compare, say, printf(3), which always does formatting substitution even when it doesn’t make sense. Python is a bit smarter.
lol, reading all the comments about people searching for logging and praising your video, like it came out the same day they commented.. then i realized the video came out 24 hrs ago, and i too searched for logging the same day the video came out xD
Listen i won't say that i intended to use print as my main debugging tool but it just naturally happened because i didn't realize that debugging already did most of the work for you, heck at first i didn't even really know what debugging was. Great tutorial however I'll end up using it quite a bit in the long term and it saves me some work so nice.
Dont use f-string when logging. Because, if you use logger.debug("text %s", variable), the text formating will only happen, if the debug level is active. If you use f-string, the string will be formated, even before it is decided wether the msg will be logged, thus losing performance
@@Anequit they can't, because it depends on what you are evaluating in the fstring. But given that this is a debug level statement, it shouldn't be too expensive. I personally wouldn't sacrifice the readability and still use fstrings tho.
If you care about performance, why not use other languages 😂 Seriously, python is a flexible and high readability language with reasonable performance. Why do you need to sacrifice a readability? Are you throwing a fuckin Ackermann function to fstring?
Tim, as a couple other people have pointed out in the comments, but it bares repeating, you should never use f-strings, simple concatenation or .format when logging messages. Instead, you should use %s or similar in the log message as a placeholder, and then pass the variable that you want to merge into the message as an argument to the logging method. For example: in_val = 105 logging.info(“Tip amount entered: %s, exceeds 100 percent”, in_val) This way, the system doesn’t have to do the extra work of formatting the log message, if for example the log level is set to error. When you pass the variable as an argument to the logging function this way, the system will defer the formatting of the message until it absolutely knows that the log message is going to be output.
I do miss one thing though; this video shows how to use logging in one file, but what I think most people are having an issue with is multiple loggers. An idea for a "logging -part 2" could be, how do we use (different) loggers across a module? In the video above there's not really a reason to use "logger", you can use "logging" directly thus I think it would really be great to have an example where (and how) you would use various loggers
finnaly found, why the "X" user can"t launch my script => it's because this user has not some library installed this logging think is very useful thx man.
Hi, nice content ! thx a lot ! starting in Python and your video really help. I'm just wondering about your interface. I saw you use PyCharm in your other video and i'm use to the 'VBA' type of interface. What are you using now and what do you suggest using. Thx in advance !
I've never encountered the need to use a logger in 1 person projects, no matter large or small. However, if you are working as a team, having a common standard for logging is extremely useful
I have noticed, when using the logging.exception() call it adds the exc_info on end of the preformated text. This is problem as I created the format to be Json and this exc_info is always put outside of the JSON which breaks the format. Can I somehow use the logging formatting to store this exc_info in my json format? I tried to use exc_info but that spits out multiple lines, like text what I can see on screen when error occurs. But I want just the Traceback what is return in function logging.exception(), as there it seems to be always the exact point of failure rather then all the dependencies what I have no power over. Thank you
I'm sorry, TIm, i almost never say how much your content helps. Thank you, i'm sorry that we are moneyless-bums and you have to rely onto a separate revenue stream.
The greatest problem I face as a programmer is not ERRORs, ..it is erroneous results. ..and for that the LOGGING module is not helpful.....you need to trace every step of the program ...Errors are not hard to find....getting a 5, instead of a 55 though....can take you 2 days to figure out sometimes...because there is no real error associated with it. Thats what takes time , not things like 1/0...or adding strings to integers.
Hey Tim.. I have been learning react and django to make full stack web applications. Do you have any suggestions on what to do when you get stuck and can't seem to find the solution with google?
Sure, the logger is cool, but print isn't going to make my script reach out to a remote LDAP server and execute arbitrary code. So I think I'll just keep using print.
The logging module just gives you more control over message output, that’s all. If you use sys.stderr.write, then you have to do your own implementation of verbosity levels, for example. The standard Python module has already pre-invented this whole wheel (and much else besides), so you don’t have to.
doesn't know about that, i was creating my "own log file" ^^" for my little python programmation this gonna maybe help me, to know whatt the name of the user my script are lauch thx
WARNING: Do not use logging to replace common messages on your system! Use print instead. You will be flooded with unwanted messages from third-party packages! The use of logging in these cases is considered bad practice!
Finally...Someone gives an informative lecture/demo on Python's logging engine. This was excellent and would have saved me a bit of time and confusion a couple of years ago. I use a separate "logger" for each module for large apps. This way I know what module (and in many cases the class and/or function) that generated the message. The only problem with this type of logging is that the module level logger (by definition) is a "global" variable. Recently I implemented a color-coded console-only logger also. Once you understand the various "logging" components (handler, formatter, name) it's really not that hard to do. Excellent Video !
seriously?? that's a copy and pasted video from corey schafer
OK, now I understand why you have 1.1M subscribers: exceptional content delivered exceptionally well. First time seeing your content, instant subscriber. Keep up the good work.
1.6 at the time I subbed, I agree ^^^^^
I was *literally* just searching about logging in python for a project yesterday... just in time Tim 👍
I swear he has us figured out
X2
Same!
I haven't really bridged the gap into intermediate level projects yet. I've used logging a couple of times before, but I can def see how it would be more useful in larger, more mature, programs. Thanks for the video!
Loguru is my go-to logging module, makes it sooooo easy to define custom levels, multiple logging sinks, and i just parse the data into a logging SQL Server table and it is GLORIOUS
I've been following you since under 100k subscribers... somehow you consistently produce content that aligns with what I am in the process of learning. Thank you! (I'm a former gryphon too!)
no joke, i was just looking for a logging in python video. thanks a lot Tim ❤️
By the way, log.log is unchanged not because you are not using it but because you overwrite it with the same log text. All log comes from a logger eventually goes to its parent logger (in this case: root logger).And if you don’t want this behavior, you need to set logger.propagate = False.
Thank you so much, I was literally facing this issue... Thank you, infact in this video the log.log has got updated indeed
@@ShivamPatel-yg3kd Thank you so much
thanks for this, you have explained logging in a really simple and straightforward manner. You have probably saved me a lot of effort in trawling through documentation. Much appreciated
Amazing video! I'd like to add, however, that most linters recommend using the 'args' of the logging levels to embed variables. For example:
logging.info(f"the value of x is {x}") -> wrong
logging.info("the value of x is %d" % x) -> wrong
logging.info("the value of x is %d", x) -> correct approach
Do note that all three ways shown above do work - it's just that the third method is recommended for use.
I come to comment on this +1
can i ask why? ive never heard of a correct way of formatting strings?
The second version is fine, too. Note that the docs state that no formatting substitution is done on the message string if no formatting arguments are supplied.
Compare, say, printf(3), which always does formatting substitution even when it doesn’t make sense. Python is a bit smarter.
lol, reading all the comments about people searching for logging and praising your video, like it came out the same day they commented.. then i realized the video came out 24 hrs ago, and i too searched for logging the same day the video came out xD
Listen i won't say that i intended to use print as my main debugging tool but it just naturally happened because i didn't realize that debugging already did most of the work for you, heck at first i didn't even really know what debugging was.
Great tutorial however I'll end up using it quite a bit in the long term and it saves me some work so nice.
Dont use f-string when logging. Because, if you use logger.debug("text %s", variable), the text formating will only happen, if the debug level is active. If you use f-string, the string will be formated, even before it is decided wether the msg will be logged, thus losing performance
Can you post benchmarks of the performance loss?
Was about to comment the same thing. Such a minor point but is considered best practice.
@@Anequit they can't, because it depends on what you are evaluating in the fstring. But given that this is a debug level statement, it shouldn't be too expensive. I personally wouldn't sacrifice the readability and still use fstrings tho.
If you care about performance, why not use other languages 😂
Seriously, python is a flexible and high readability language with reasonable performance. Why do you need to sacrifice a readability? Are you throwing a fuckin Ackermann function to fstring?
seems the small gain in performance is absolutely not worth the readibility you may lose so ok but mostly doesn't matter
i have been debugging all day with those print's. thanks for another way
Amazing channel. Has helped me so much when studying to become a software developer. Amazing work really!
best thing that i found today about logging-python
Cheers Tim! Another Python skill in the bag!
For real this video what at the perfect time, last night i was searching for that and i understood nothing until this time i saw this video . Thanks
short & crisp explanation. Perfect!
Nice.We would glad to see about Golang (logging, unit testing and other advanced topics )
Very valuable and time saving tutorial. Thanks. Fixed up my logging implementation
Crazy useful! I am done with print statements now, thank you!!
Tim, as a couple other people have pointed out in the comments, but it bares repeating, you should never use f-strings, simple concatenation or .format when logging messages. Instead, you should use %s or similar in the log message as a placeholder, and then pass the variable that you want to merge into the message as an argument to the logging method.
For example:
in_val = 105
logging.info(“Tip amount entered: %s, exceeds 100 percent”, in_val)
This way, the system doesn’t have to do the extra work of formatting the log message, if for example the log level is set to error. When you pass the variable as an argument to the logging function this way, the system will defer the formatting of the message until it absolutely knows that the log message is going to be output.
Hey Tim ! great content as always. can you do tutorial on how to write unit test for Flask applications ?
legend finally made it
Outstanding, works great first time, lots of good ideas.
I do miss one thing though; this video shows how to use logging in one file, but what I think most people are having an issue with is multiple loggers.
An idea for a "logging -part 2" could be, how do we use (different) loggers across a module?
In the video above there's not really a reason to use "logger", you can use "logging" directly thus I think it would really be great to have an example where (and how) you would use various loggers
Everything i needed in just 15 Minutes! Perfect
Perfect timing ! Thanks 😀
Very helpful! Thank you!
Thanks a lot , very informative !!
life saver
thanks
👍
Great tutorial! Thanks :)
finnaly found, why the "X" user can"t launch my script => it's because this user has not some library installed
this logging think is very useful thx man.
It's so perfect timing
Thanks for the good content
Great tutorial
Hi, nice content ! thx a lot ! starting in Python and your video really help. I'm just wondering about your interface. I saw you use PyCharm in your other video and i'm use to the 'VBA' type of interface. What are you using now and what do you suggest using. Thx in advance !
thank you it was a helpful video
Very useful 👍☺️
Hey Tim when are you going to open startup?
Thanks so much!
Could we get more information on db handler? I'd love to learn how to use it :)
You're the man!
Epic! thanks
Is it possible to give a short description of the log file at the beginning of each log file, whenever it is created?
Got a list of scripts that I need to turn logging on for. Need to add some logging functions to class object I have made for it
Hi! Any idea when is Kite going to be up?
very useful thanks
I've never encountered the need to use a logger in 1 person projects, no matter large or small. However, if you are working as a team, having a common standard for logging is extremely useful
really useful
Nice Video! The music was a little bit too loud tho
i didnt even realize music was playing til you pointed it out
@@waqaspathan3337 Interesting. It’s probably just me
Didn't know about the exception thing
Is there any append mode instead of overwrite mode for logging?? I want to store my logs of a service in an S3 bucket
I have noticed, when using the logging.exception() call it adds the exc_info on end of the preformated text.
This is problem as I created the format to be Json and this exc_info is always put outside of the JSON which breaks the format.
Can I somehow use the logging formatting to store this exc_info in my json format?
I tried to use exc_info but that spits out multiple lines, like text what I can see on screen when error occurs. But I want just the Traceback what is return in function logging.exception(), as there it seems to be always the exact point of failure rather then all the dependencies what I have no power over.
Thank you
Hello, I'm new to programming and I don't know anything about it. I would like to know what you recommend
What if an existing value exists on the logger? Is it going to skip logging the existing value or will it add the same value?
Thank you for the video. I really hope you would get rid of the background music.
DSA chapter is it covered in programming expert ?
I'm sorry, TIm, i almost never say how much your content helps. Thank you, i'm sorry that we are moneyless-bums and you have to rely onto a separate revenue stream.
The greatest problem I face as a programmer is not ERRORs, ..it is erroneous results. ..and for that the LOGGING module is not helpful.....you need to trace every step of the program ...Errors are not hard to find....getting a 5, instead of a 55 though....can take you 2 days to figure out sometimes...because there is no real error associated with it. Thats what takes time , not things like 1/0...or adding strings to integers.
do you read my mind
I am dealing with an error for the last 3 day
and I print to debug
print("this is painful")
Hey Tim.. I have been learning react and django to make full stack web applications. Do you have any suggestions on what to do when you get stuck and can't seem to find the solution with google?
Could you do something about linear programming please?
thank you
thank you
thank you
Sure, the logger is cool, but print isn't going to make my script reach out to a remote LDAP server and execute arbitrary code. So I think I'll just keep using print.
Which Code Editor you used?
What do u use as sublime theme and colour scheme 🌚
i'm new to debugging professionally, idk why to do logging
Hi Tim, may I know the autocomplete package your are using?
The logging module just gives you more control over message output, that’s all. If you use sys.stderr.write, then you have to do your own implementation of verbosity levels, for example. The standard Python module has already pre-invented this whole wheel (and much else besides), so you don’t have to.
which IDE/code editor you are using?
Sublime text I think
@@einstein_god thanks
What is your favorite IDE?
my test.log file was empty till the time i did not add the logging.basicConfig, can someone explain why?
how do you log in file as well as on screen ?
I was looking to put a logging system in my app today! How did you know?
Up
Logging in FastAPI gives me headaches. Dublicate lines, and no rollover in Windows.
Damm, I just spent the last 2 days builing my logger template for work.
doesn't know about that, i was creating my "own log file" ^^" for my little python programmation
this gonna maybe help me, to know whatt the name of the user my script are lauch
thx
What ide?
print("like")
WARNING: Do not use logging to replace common messages on your system! Use print instead. You will be flooded with unwanted messages from third-party packages! The use of logging in these cases is considered bad practice!
It’s literally a built-in package but ok
💪💪💪
what is this ide?
vs code
i mean sublime text
Yep yep thats the stuff
7:32
Looking for the past 1 year
"Stop using print debugging" - what sort of madness is this? :D
hi
4th
You stop using print debugging!
Jesus loves you. Start a relationship with Him today before its too late.
I started a relationship with h8m last night. Nnow my balls are empty. Halleluja
Yes, please stop teaching print()
1.
ratio
Those people, trying to use Python like a full-fledged language instead of the hacky script beast it really is, are hilarious
+1
Great tutorial