More under the hood videos please. In modern times we have got so much abstraction going on most of don't know how things work at their core. Would love this series.
I would honestly watch an entire online course on how GUI programs work under the hood. There's tons of information on the internet on how to write GUI applications with popular toolkits and frameworks, but none that explain how the operating system actually deals with all that stuff.
@@TheAdriyaman You're not incorrect, but for many people it really helps connect the understanding of these high level frameworks when you can learn exactly how all the low level operations work.
I agree. I feel like with a lot of abstracted concepts, you'll just get told to use a framework. But that doesn't tell me how things _really_ work. My question is, how did they know _how_ to implement these things in the first place (when there was no framework)?
Having any meaningful understanding of a compiler pipeline is going to require more time than Computerphile videos run. Maybe they could do a multi-parter as a high-level intro to the topic, but it really isn't a small topic.
It's super charming that someone in the Year 2019 says something like "whether you are programming for RiscOS, Atari or Amiga..." :D yup, these are still the relevant platforms. At least in my mind.
Its not hardware... Its basically a protocol on how cpu recieves and executes commands. Only instruction set is open source, and the actual hardware to use RISC-V is a trade secret
@@Studio-df7ge My understanding is that the ISA is open source and how you construct your CPU to use it is up to the producer to share or not. So in a broad term, open source "hardware".
6:10 A common use for mouse-moved events is changing the mouse pointer, e.g. making it an I-beam pointer when the user moves into a text field, to indicate that they can type there.
That's a mouseover event, not a mouse-move event: you want the OS to do the neccessary work to determine whether it's over a relevant part of your window: working it out yourself is error-prone if you have scaling, font rendering etc to consider. An example of where mouse-move is relevant is looking around in a 3D environment or panning the screen via dragging (although that could be a separate drag event?)
I love Dr Steve Bagley; even though most of the time the very last sentence/utterance of his current train of thought makes little sense, he is exceptionally erudite for someone who is seemingly just speaking from the top of his head! Editing probably helps a tiny bit, but I love the speed and eloquence with which he can disect such a complex topic as how to code a gui OS!
2:57 One subtle point worth noting is that the GUI framework usually handles keyboard autorepeat. That is, when you ask for another event, no “real’ event is pending, and a key is still down from the previous one, and the autorepeat interval has elapsed, then the framework will generate an autorepeat key-down event for you. Note that these autorepeat events are not buffered, so they are only generated as quickly as your code can handle them. Thus, autorepeat can never run away from your program’s ability to respond!
Yea... modern browser rendering engines are crazy complicated. You go through 4 separate steps - compute>layout>paint>composite. Some of these happen on the CPU, some on the GPU. When playing around with complex CSS/JS, some changes trigger all 4 steps, some trigger less and a web developer needs to be mindful of all these.
Would love a bigger dive how the OS handles the message loop. How one could implement such GUI without an operating system like, for example when using a custom SoC or microcontroller. Love all your work and the way the speakers present the info. Keep up
Truth is, it is not a loop: At the top of the loop is an instruction to wait for an event. Therefore a better way to see it, is a routine that runs to completion, every time an event comes in. And all events are eventually (as you trace them back) interrupts.
It comes from a poorly designed window manager that Windows has. The whole processing messages thing is a nightmare to handle. No other operating system has this issue because window manager is the one handling painting of buffers not the application itself.
4:05 In fact, in windows under Wayland compositors don't get any information on their locations or z-order relative to other windows. 14:10 Wayland compositors can implement transparency should they choose to. Due to the design decision of windows not knowing their z-order, all it comes down to is asking the windows underneath to redraw their surfaces.
I mean, when you're replacing Xorg wholesale with a completely new design, everything is inevitably going to break. No need to worry, all the toolkits have been ported to wayland, so as long as you're using GTK+ or Qt or SDL or EFL, you won't notice a thing. (This comment was sent from my Wayland session.)
There are now widely adopted protocols for screenshotting and screen recording, and if there is a large enough need for capturing footage from a wayland surface, someone will likely design a protocol to allow that. So yes, you can fix those design choices. It was just a design choice to not make those protocols part of the core spec. EDIT: actually, the core protocols don't even specify how to display a window... Even xdg-shell is an extension
4:03 Actually, Emacs calls them "frames", and reserves "windows" to refer to divisions within frames. Like, if you type C-x 2, the current window gets split into two. Yeah, it's non-standard terminology.
I think what you call "windows" are actually the buffers. You can have many frames (e.g. UI Windows) of emacs and each frame has it's buffers (either one, or split in any way)
@@AnastasisGrammenos Nope. Buffers are the things that hold the contents of files (and other things). A window can be viewing any buffer at a given moment. You can even have multiple windows viewing the same buffer, possibly at different places. The C-x 2 I cited activates the command "split-window-below", with this description: "Split the selected window into two windows, one above the other. The selected window is above. The newly split-off window is below and displays the same buffer." Like I said, non-standard terminology. Might be because Emacs originates from before window systems (1976, specifically). ...Man, that's almost half a century ago!
The reason for that should be obvious: it’s because Emacs was originally doing “windowing” on text terminals, before GUIs became commonplace. Rather than change their terminology, they simply added another nonstandard term, “frames”.
Yeah, Wayland does that, Wayland's entire thing is composting just like Aero and whatever the heck Apple calls their window manager these days. There's plenty of window managers for Xorg that do it too now. Most likely if you're using a full desktop environment, it's not using a traditional stacking window manager anymore. Also, it's been a while since I've done Windows GUI stuff, I know on Qt 4 (Not sure about 5.) you DO have MouseMove events, or can. The way the MOC on Qt works is simply not to include any handler for events that aren't there at all. I personally don't understand why more toolkits don't use callbacks instead of forcing the use of enormous switch structures, though, sort of akin to what was described in Windows. Rather than do a bit switch and an inefficient loop, just simply have the OS call a function within a program registered to the event instead.
I want a video on how multiple objects (Windows/widgets) are composited on the framebuffer. I am working on a graphics library in C for fun. Basically it can draw lines, boxes, etc as of now. I have to draw multiple such objects onto the screen and they might move based on user’s inputs. I’m thinking to have virtual framebuffer with functions that draw the respective objects from back to front so that the objects in front masks out the window in the back. A video on that would be nice.
I love these videos! This was a nice trip down memory lane, to back in the days when we acutally interacted with message queues when writing applications. 😊
@@lawrencedoliveiro9104 Indeed I am. Modern GUI apps work like event-driven, like in Hollywood: "We'll call you if needed, don't call us." There is no need to write messageloops anymore.
Could Dr Steve Bagley also create a video about how file formats work under the hood? How are they laid out in memory and read by the program? For example, how are pdf, word documents, video, and audio files stored in the memory and how can we write a simple C program to parse this information and do something useful with it?
Ah that’s why the xcode / Android Studio / Flutter programs are so big and have so many files. When I as hobby programmer create an ‘Hello world’ app for my iPhone most of the work is already done. Nice explanation by the way!
I'd say that it's more common that they're implemented using MouseEnter, MouseLeave or Mouse[H]Over events provided by the OS than interpreting raw MouseMove events to determine those by yourself.
Isn't in most cases the mouse position accessible as a 2D vector variable? Using some pseudocode here, you can activate drag on Mouse.LeftButtonDown(), use the vector to handle the drag, and end the drag on Mouse.LeftButtonUp().
@@LudwigvanBeethoven2 MouseMove needs to be carefully controlled. For example, the WaitNextEvent() call he describes takes a mouse-region argument, and a mouse-move event is only generated when the mouse leaves this region.
I'm surprised they didn't touch on the move to embedded browsers to create GUI's using web technologies to save money. A given it is still the same underlying principals with an abstraction layer.
Oh wow, this really underscores to me just how advanced the Amiga windowing environment was in comparison to other systems. But I guess it had to be, given such a polling loop would have been a very bad idea in a multitasking system running on a 7MHz 68000 CPU.
It's meant to make it easier for the user, not easier for the developer. If you want something that makes you happy as a developer, try Python or Ruby.
In human-computer interaction, WIMP stands for "windows, icons, menus, pointer",[1][2][3]denoting a style of interaction using these elements of the user interface. It was coined by Merzouga Wilberts in 1980.[4]
Yeah, this works far better in written form, as you can tell the difference between the common noun "windows" and the proper noun "Windows" by the capital letter. Ah, but this is Microsoft through-and-through, happily trying to take monopolistic control over the English language itself: "Windows™", "Word™", "Excel™", "Access™", "Paint™", etc. - all of which are just singular plain English words - and then they just slap a trademark symbol on it. And they really did try to sue "Lindows" over its portmanteau of "windows" and "Linux" - but, thankfully, this failed in the court because "windows", without the capital letter as a common noun, is a standard English word and also the generic term for a screen portal, so their arrogance backfired on them there, as it was deemed that "Linux + windows" are two things that have nothing to do with Microsoft (and the capital "L" in the portmanteau comes from "Linux", which is a proper noun and NOT a common English word). So the court ruled "mind your own business and go away" on that one.
I wonder what the latest WIndows 10's Reveal Highlight feature works under the hood, where interactive controls are highlighted near where the cursor is positioned.
Use mouse move messages to know when you need to draw, draw brightness per pixel depending on the part of the control and distance to the mouse. It's surprisingly easy!
When the mouse xy position send to system how system find click window or button from many windows? By a loop determine window border insider(If stacked)?
Regarding naming windows "windows". In framework called wxWidgets the "windows" are called "Frames" and basically "Widgets" (buttons etc.) are called... "Windows". Just a little curiosity.
1:54 One problem with the Windows approach was that every GUI element (every button, menu, possibly even every menu item) was a “window” as far as the OS was concerned. And storage for every “window” was allocated from an area known as the “system resource heap”, which only had a small, finite size. So your machine could have lots of RAM, but if the GUI was too complex (or you had too many of them running at once), you could still run out of system resources!
@@lawrencedoliveiro9104 it doesn't happen always, the dialog must be synchronous. Then the event loop for the window will "pause" and switch to the dialog - this allows you to simply write response = openDialog() in your code and block the program on that line until that dialog closes. If you want both the back window and the dialog to be responsive, the dialog has to be asynchronous.
@ Look at GIMP, for example, for which GTK was originally developed; in filter settings dialogs, there is usually a “preview” button, which lets me see the effect of the settings on the image in the main window. I can move the dialog around, scroll around the image etc and have it update in real time until I do “OK” to confirm the filter application or “Cancel” to restore the previous state of the image.
5 лет назад
@@lawrencedoliveiro9104 yes, as I said, it depends on the dialog/window that the programmer chooses. If you choose a synchronous dialog/window (usually used for simple yes/no prompts or alerts), the window in the background cannot update, otherwise it couldn't be synchronous in the code. Of course this does not make sense for a preview dialog and therefore the dialog that you describe is asynchronous, which makes it a bit harder to handle in code. You can see examples of synchronous dialogs in web browsers - a simple prompt/alert is synchronous and therefore you cannot use the tab in the background while it's active (although this is not a typical example as I suspect it's simulated in the browser, because for example video playback continues during an alert). Error alerts and prompts in Windows apps typically used to be synchronous.
@4:07 :sigh: now I miss my Atari ST GEM Desktop...
5 лет назад+2
Why are you so visibly uncomfortable in mentioning Linux? You struggle to avoid bringing it into question on this video. Is it because Linux has differing GUI toolkits like GTK+, Qt and wx?
in fact Windows programs have both: a message loop and event handlers for the windows. but most modern solutions burry the message loop deep in the framework and expose only the event handlers to the programmer.
"burry the message loop deep in the framework and expose only the event handlers to the programmer." That is because Windows is so poorly designed. Burying like that are work around that flaw.
@@philipfry9436 You mean that if YOU stall the message loop by your heavy function then it won't be able to process other messages? Well yes, that's single threaded program design, basicly UI runs in the main thread in Windows. What are alternatives to that?
@@MrSparker95 I am not stalling the message pump, it get updated every frame. Drag or resize the window around and it become stalled by Windows (the operating system). You need co-routine, threading, or calling your update function from the window proc to work around that. It's a garbage api. There should be only a GetMessage call. No dispatch. Dispatch is widget or application space.
If you're developing for Windows and have decided to punish yourself by using plain Win32 APIs, you have to code all the redrawing yourself, unless you're using some built-in controls. Even in that latter case, you still have to tell those controls when to redraw.
If it were accurate, sure. It's not though. He's not explaining the different roles of compositor and window manager and assumes GUI toolkit does everything, which is not true.
@@brod515 Yes you can. But using it as your main message receiving call at message loop results in either busy waiting or unnecessarily complex code (or both). Steve's code doesn't work, because he doesn't check if there is message in the queue and therefore tries to translate and dispatch messages which aren't there.
Window repaint is supposed to be a quintessential NOT MY PROBLEM thing ever since the stone age. Get with the program and use a compositing window manager like, you know, normal people.
Ah, looping through WF_NEXTXYWH trying to work out how to draw part of your windows. 'Now you could if you were a lazy programmer redaw everything... and just clip to the area'... Guilty, m'lud.
I miss my wobbly windows and the fancy alt-tab switchers. All I got now is what's offered by compiz. So, you know, transparency and drop shadows. So boring.
I'm a bit surprised that you said under the hood instead of under the bonnet, was that for the benefit of Americans or do you generally use under the hood in a non-car context? Or do they not call it a bonnet in the UK anymore?
This video does a great job of highlighting how "Windows" is a terrible name for a GUI! I get that it was 35 years ago, but what were you thinking, Microsoft engineers?
I think this was back in the day when Microsoft rarely tried to do clever names for most of their products and just named them after what they do. Windows, Office, Disk Operating System...
I think that was deliberate to help non-programmers understand. It is not obvious what logical not means just out of the word "not" and it is not obvious that "!" means logical not.
while(!quit) is perfectly readable. A reason to prefer this form is that global symbols are initialized to zero and therefore is automatic initialized correctly. Perhaps you nooblets should wait to be more experimented with programming before dissing experts.
Wait, Python or C family? How on earth is Python C-like at all? Java is strongly influenced by C (semicolons, curly braces, syntax conventions like for loops), but Python isn't even in the same species, let alone family. Although I will admit, I've never been a fan of how strictly the language enforces OOP, I much prefer C++ where I can easily drop into straight functional C if I need to, along with the ability to (mis)use pointers.
Anyone know of a good book that addresses gui design from a theory standpoint? Obviously there are a ton of books on specific toolkits and computer graphics, but I'd like one that addresses all of the chaos involved in human-computer interaction; e.g., object-drawing and event handling.
out of all the things that are special and how a OS works that i know and can make sense of... i really struggle with what happens after you press power and the BIOS appears and how that part works... like i wonder how putting power in a "abacus" is equal a BIOS, even tho i can understand how mathematically the Abacus can make games play in 3d... like why can i Type a letter or a command on the BIOS and why are a set of Characters equal a function... how did they define that in the BIOS? so yeah, why do pressing power = seeing a BIOS and being able to type... like you have to define letters some where, you have to define a screen able to display that... an OS cannot do anything without the BIOS having defined that for it before hence how the heck did they make a BIOS without a BIOS?
@@theepicslayer7sss101 i would guess that bios is like machine code, so basically that data is read and took verbatim by the circuitry in the form of electronic signals, so, highs and lows on voltage, logic gates, transistors and stuff. So, the hardware level stuff rather than some yet another software thing.
He failed to explain what compositor is and what's the role of window managers. In properly designed operating systems, not Windows, application will request a buffer from window manager/compositor. This buffer is located on GPU memory. Compositor will then make sure that order of displayed things is correct and redraw only "dirty" parts. Toolkit in turn needs to make sure to update that buffer when window manager requests it to. That way there are no solitaire effect Windows often gets when application stop responding. Windows works in a different ways. There system will send bunch of messages to the application window and if there's no response for few seconds you get that not responding message. Among these messages is the repaint request. So if application is not responding nothing will happen and you get solitaire effect. In this repaint message windows will provide a "handle" on to which content will be painted. Hardware acceleration is basically where buffers are located and how fast they are painted. If possible compositor will store those buffers on GPU and then create final buffer that will end up being displayed by layering all the application buffers in proper places. Eventually it just tells GPU "display this". When there's no hardware acceleration, all the layering is done on CPU which is not specialized for this kind of operation and ends up being slower.
other computerphile commenters: Something something CPU something library in C something CSS/JS something something Atari ST GEM something "frames" something shirt compression me 15:45 onwards: **scrolling window up and down obsessively** _Ah?! I'm moving an already moving video! This is real-life magic!_
That's all he does in his videos, ramble on and on an on, maybe with a little doodling. Just do a quick doodle, explain what we're seeing and then just code, code, code, but I'm not sure he's able to do that.
1:50 WTF are you talking about. Windows is garbage. The window proc stall the thread forever and you have to use co-routine or threads to get around that.
@@colin-campbell Your solution to the garbageness of windows is to add a hack that call user update function within the window proc? We got the operating systems that we deserve...
Philip Fry There is a solution if you know your way around pointer routines and maybe a little bit of XML injection if you’re not great with pointer routines.
the frame is just the outmost container for the content, so you could argue that the frame in itself doesnt really represent the window, rather, as the name suggests, the window frame.
that just builds on the window analogy and refer to the frame of the window, then that thing holds multiple "panels" window is the whole thing. then the windows GUI has multiple windows hence windows macs has multiple baskets which hold apples linux has multiple pockets which hold lints
More under the hood videos please. In modern times we have got so much abstraction going on most of don't know how things work at their core. Would love this series.
I would honestly watch an entire online course on how GUI programs work under the hood. There's tons of information on the internet on how to write GUI applications with popular toolkits and frameworks, but none that explain how the operating system actually deals with all that stuff.
"In modern times we have got so much abstraction going on most of don't know how things work at their core" -> Not necessarily a bad thing
@@LiamMayfair Modern operating systems don’t tie themselves to a particular GUI layer. Microsoft and Apple do, but Linux doesn’t.
@@TheAdriyaman You're not incorrect, but for many people it really helps connect the understanding of these high level frameworks when you can learn exactly how all the low level operations work.
I agree. I feel like with a lot of abstracted concepts, you'll just get told to use a framework. But that doesn't tell me how things _really_ work. My question is, how did they know _how_ to implement these things in the first place (when there was no framework)?
Would love an under the hood video on the gcc compiler :/
that would be a long and painful road :D JRR Tolkien would look like a total noob compared to GCC pipeline :D
10 minute loop of a man screaming
Llvm is more sane
No, you really dont.
Having any meaningful understanding of a compiler pipeline is going to require more time than Computerphile videos run. Maybe they could do a multi-parter as a high-level intro to the topic, but it really isn't a small topic.
It's super charming that someone in the Year 2019 says something like "whether you are programming for RiscOS, Atari or Amiga..." :D yup, these are still the relevant platforms. At least in my mind.
This is a humble request for a video about open source hardware like RISC-V
THIS PLEASE!
Its not hardware... Its basically a protocol on how cpu recieves and executes commands. Only instruction set is open source, and the actual hardware to use RISC-V is a trade secret
@@Studio-df7ge My understanding is that the ISA is open source and how you construct your CPU to use it is up to the producer to share or not. So in a broad term, open source "hardware".
6:10 A common use for mouse-moved events is changing the mouse pointer, e.g. making it an I-beam pointer when the user moves into a text field, to indicate that they can type there.
That's a mouseover event, not a mouse-move event: you want the OS to do the neccessary work to determine whether it's over a relevant part of your window: working it out yourself is error-prone if you have scaling, font rendering etc to consider. An example of where mouse-move is relevant is looking around in a 3D environment or panning the screen via dragging (although that could be a separate drag event?)
comes from apple camp
I love Dr Steve Bagley; even though most of the time the very last sentence/utterance of his current train of thought makes little sense, he is exceptionally erudite for someone who is seemingly just speaking from the top of his head! Editing probably helps a tiny bit, but I love the speed and eloquence with which he can disect such a complex topic as how to code a gui OS!
2:57 One subtle point worth noting is that the GUI framework usually handles keyboard autorepeat. That is, when you ask for another event, no “real’ event is pending, and a key is still down from the previous one, and the autorepeat interval has elapsed, then the framework will generate an autorepeat key-down event for you.
Note that these autorepeat events are not buffered, so they are only generated as quickly as your code can handle them. Thus, autorepeat can never run away from your program’s ability to respond!
Also, programs usually don't handle WM_KEYDOWN and WM_KEYUP, but they _do_ handle the WM_KEYPRESS which is synthesized by Windows.
Yea... modern browser rendering engines are crazy complicated. You go through 4 separate steps - compute>layout>paint>composite. Some of these happen on the CPU, some on the GPU. When playing around with complex CSS/JS, some changes trigger all 4 steps, some trigger less and a web developer needs to be mindful of all these.
Would love a bigger dive how the OS handles the message loop. How one could implement such GUI without an operating system like, for example when using a custom SoC or microcontroller.
Love all your work and the way the speakers present the info. Keep up
Truth is, it is not a loop: At the top of the loop is an instruction to wait for an event. Therefore a better way to see it, is a routine that runs to completion, every time an event comes in. And all events are eventually (as you trace them back) interrupts.
Have you got anything more related to this ? I have bern searching for articles, Videos. Could not find out
@@mrrdelorenzi8478 i think this above comment by Abbas Ramish was asking you, so, have you got any resource related to this?
I personally prefer a crispy program to a gooey one
Better for number-crunching. *Wipes crumbs from mouth*
Ho!
I cannot not focus on his shirt pattern that is blurred and turned sharp again over and over. Now apply knowledge from picture compression video.
Yes I had a bit of a shocker with focus on this video! >Sean
I always get recommended something by computerphile I didn't even know I wanted to learn about but I love it every time.
Now I understand where this Windows XP Not Responding Art comes from :D
Exactly the reason. Also when the Windows detects a program has not processed any events for a while, it does the "not responding" message.
Indeed, that's when the underlying window doesn't process it's WM_PAINT messages properly.
I think that default timeout is 5 seconds. If program does not process any messages then you get that "not responding" message.
@@ataksnajpera yeah. Especially for programs that process data in long chunks without sending messages saying that it's still working.
It comes from a poorly designed window manager that Windows has. The whole processing messages thing is a nightmare to handle. No other operating system has this issue because window manager is the one handling painting of buffers not the application itself.
Very nice video! Perhaps it could be interesting to discuss the contrast with intermediate mode GUIs as well...
4:05 In fact, in windows under Wayland compositors don't get any information on their locations or z-order relative to other windows.
14:10 Wayland compositors can implement transparency should they choose to. Due to the design decision of windows not knowing their z-order, all it comes down to is asking the windows underneath to redraw their surfaces.
Wayland breaks everything part MCXVII
I mean, when you're replacing Xorg wholesale with a completely new design, everything is inevitably going to break. No need to worry, all the toolkits have been ported to wayland, so as long as you're using GTK+ or Qt or SDL or EFL, you won't notice a thing.
(This comment was sent from my Wayland session.)
@@GammaFn. You can't really fix some design choices; like not being able to see another software's viewport. It broke ShareX for instance.
There are now widely adopted protocols for screenshotting and screen recording, and if there is a large enough need for capturing footage from a wayland surface, someone will likely design a protocol to allow that.
So yes, you can fix those design choices. It was just a design choice to not make those protocols part of the core spec.
EDIT: actually, the core protocols don't even specify how to display a window... Even xdg-shell is an extension
@@GammaFn. Oh cool, I didn't know.
I never thought I'd be learning about this stuff in such depth. I love computer science.
“Object oriented frameworks” O.O.F.
Introducing the "Open Object Oriented Framework"
oof
foo
Great detail! Video games were the original GUIs :)
4:03 Actually, Emacs calls them "frames", and reserves "windows" to refer to divisions within frames. Like, if you type C-x 2, the current window gets split into two. Yeah, it's non-standard terminology.
Also JFrame in java.
I think what you call "windows" are actually the buffers. You can have many frames (e.g. UI Windows) of emacs and each frame has it's buffers (either one, or split in any way)
@@AnastasisGrammenos Nope. Buffers are the things that hold the contents of files (and other things). A window can be viewing any buffer at a given moment. You can even have multiple windows viewing the same buffer, possibly at different places. The C-x 2 I cited activates the command "split-window-below", with this description: "Split the selected window into two windows, one above the other. The selected window is above. The newly split-off window is below and displays the same buffer."
Like I said, non-standard terminology. Might be because Emacs originates from before window systems (1976, specifically). ...Man, that's almost half a century ago!
The reason for that should be obvious: it’s because Emacs was originally doing “windowing” on text terminals, before GUIs became commonplace. Rather than change their terminology, they simply added another nonstandard term, “frames”.
Harry Hill refers to them as frames as well and when you select one the operating system sends a message saying, "You've been framed"
The event polling is similar to SDL
SDL Learned from the big
They are just one, big, while true loop.
Lol that windows message handling stuff reminded me of win32 programming in C. Those were fun times haha
Yeah, Wayland does that, Wayland's entire thing is composting just like Aero and whatever the heck Apple calls their window manager these days. There's plenty of window managers for Xorg that do it too now. Most likely if you're using a full desktop environment, it's not using a traditional stacking window manager anymore.
Also, it's been a while since I've done Windows GUI stuff, I know on Qt 4 (Not sure about 5.) you DO have MouseMove events, or can. The way the MOC on Qt works is simply not to include any handler for events that aren't there at all.
I personally don't understand why more toolkits don't use callbacks instead of forcing the use of enormous switch structures, though, sort of akin to what was described in Windows. Rather than do a bit switch and an inefficient loop, just simply have the OS call a function within a program registered to the event instead.
The more I think about it, the more in awe of that shirt I am.
I want that shirt
I tried to see constellations, but I failed.
His shirt may be blue, but the noise on it isn't.
I want a video on how multiple objects (Windows/widgets) are composited on the framebuffer. I am working on a graphics library in C for fun. Basically it can draw lines, boxes, etc as of now.
I have to draw multiple such objects onto the screen and they might move based on user’s inputs. I’m thinking to have virtual framebuffer with functions that draw the respective objects from back to front so that the objects in front masks out the window in the back.
A video on that would be nice.
Might be interested in looking up blitters.
Non-rectangular clipping regions are your friend.
David McGill Thanks! Will do!
Sounds cool! What's your library called?
Albert Tománek I haven’t got a name for it. It’s still work in progress. Interested to collaborate?
You do get loads of mouse move events in most gui frameworks.
xev is a handy program for demonstrating this. ;)
I love these videos!
This was a nice trip down memory lane, to back in the days when we acutally interacted with message queues when writing applications. 😊
What’s the alternative? You’re not suggesting multithreading the GUI, are you?
@@lawrencedoliveiro9104 🙄👀
@@lawrencedoliveiro9104 Modern frameworks isolate the developer from the low-level message stuff.
@@PaulaJBean You’re not suggesting we use callbacks instead?
@@lawrencedoliveiro9104 Indeed I am. Modern GUI apps work like event-driven, like in Hollywood: "We'll call you if needed, don't call us." There is no need to write messageloops anymore.
Sometimes to prevent ambiguity, people refer to windows as "frames", though this usually refers to a specific "sub-window" within a top-level window.
Could Dr Steve Bagley also create a video about how file formats work under the hood? How are they laid out in memory and read by the program? For example, how are pdf, word documents, video, and audio files stored in the memory and how can we write a simple C program to parse this information and do something useful with it?
Ah that’s why the xcode / Android Studio / Flutter programs are so big and have so many files. When I as hobby programmer create an ‘Hello world’ app for my iPhone most of the work is already done. Nice explanation by the way!
MouseMove events are very common and used to implement things like hover effects
I'd say that it's more common that they're implemented using MouseEnter, MouseLeave or Mouse[H]Over events provided by the OS than interpreting raw MouseMove events to determine those by yourself.
MouseMove is bad, MouseEnter/Leave is much better, only two events required
@@LudwigvanBeethoven2 but then you can't drag and drop without another extra command.
Isn't in most cases the mouse position accessible as a 2D vector variable?
Using some pseudocode here, you can activate drag on Mouse.LeftButtonDown(), use the vector to handle the drag, and end the drag on Mouse.LeftButtonUp().
@@LudwigvanBeethoven2 MouseMove needs to be carefully controlled. For example, the WaitNextEvent() call he describes takes a mouse-region argument, and a mouse-move event is only generated when the mouse leaves this region.
I'm surprised they didn't touch on the move to embedded browsers to create GUI's using web technologies to save money. A given it is still the same underlying principals with an abstraction layer.
Oh wow, this really underscores to me just how advanced the Amiga windowing environment was in comparison to other systems. But I guess it had to be, given such a polling loop would have been a very bad idea in a multitasking system running on a 7MHz 68000 CPU.
The Amiga could never handle non-rectangular windows, though. The Mac could.
is this brand new Computerphile spatial technology at 4:07? Mind blown...
Tl;Dr: it's a huge mess eventhough it feels simple at first
It's meant to make it easier for the user, not easier for the developer. If you want something that makes you happy as a developer, try Python or Ruby.
It's not as huge of a mess as it is poorly explained.
In human-computer interaction, WIMP stands for "windows, icons, menus, pointer",[1][2][3]denoting a style of interaction using these elements of the user interface. It was coined by Merzouga Wilberts in 1980.[4]
We stopped using that purposefully
Does the operating system also handle debouncing of button inputs?
Gets confusing when talking about windows... by windows, do you mean windows or windows.
windows
windows or Windows would be less confusing in the written form...
I like to use the term “Dimdows” for the Microsoft product. Everybody knows what I mean. ;)
Yeah, this works far better in written form, as you can tell the difference between the common noun "windows" and the proper noun "Windows" by the capital letter.
Ah, but this is Microsoft through-and-through, happily trying to take monopolistic control over the English language itself:
"Windows™", "Word™", "Excel™", "Access™", "Paint™", etc. - all of which are just singular plain English words - and then they just slap a trademark symbol on it.
And they really did try to sue "Lindows" over its portmanteau of "windows" and "Linux" - but, thankfully, this failed in the court because "windows", without the capital letter as a common noun, is a standard English word and also the generic term for a screen portal, so their arrogance backfired on them there, as it was deemed that "Linux + windows" are two things that have nothing to do with Microsoft (and the capital "L" in the portmanteau comes from "Linux", which is a proper noun and NOT a common English word). So the court ruled "mind your own business and go away" on that one.
Suggest looking at Dear ImGUI, it's a great example of immediate GUI's.
what's "immediate GUI" ??
I wonder what the latest WIndows 10's Reveal Highlight feature works under the hood, where interactive controls are highlighted near where the cursor is positioned.
Use mouse move messages to know when you need to draw, draw brightness per pixel depending on the part of the control and distance to the mouse. It's surprisingly easy!
Tkinter master race.
When the mouse xy position send to system how system find click window or button from many windows? By a loop determine window border insider(If stacked)?
Great short!
Regarding naming windows "windows". In framework called wxWidgets the "windows" are called "Frames" and basically "Widgets" (buttons etc.) are called... "Windows". Just a little curiosity.
1:54 One problem with the Windows approach was that every GUI element (every button, menu, possibly even every menu item) was a “window” as far as the OS was concerned. And storage for every “window” was allocated from an area known as the “system resource heap”, which only had a small, finite size. So your machine could have lots of RAM, but if the GUI was too complex (or you had too many of them running at once), you could still run out of system resources!
Windows is windows turtles all the way down.
umh, "one prob with Win OS was" --> was, or still is??
if was, then how was it mitigated??
As I understand it, this is still a problem.
many programs these days I believe use their own custom rendering engines, so that shouldn't be a problem these days
Great job as usual, keep it up!
As for another name for "window", on Windows they are commonly called forms at least in the development scene
I've ju;t got my head around this stuff recently, if I'd only had this video sooner.
And now you know why the background window goes dormant/unresponsive when a GUI dialog pops up :-)
It doesn’t for me. What OS are you using?
@@lawrencedoliveiro9104 it doesn't happen always, the dialog must be synchronous. Then the event loop for the window will "pause" and switch to the dialog - this allows you to simply write
response = openDialog()
in your code and block the program on that line until that dialog closes. If you want both the back window and the dialog to be responsive, the dialog has to be asynchronous.
@ Look at GIMP, for example, for which GTK was originally developed; in filter settings dialogs, there is usually a “preview” button, which lets me see the effect of the settings on the image in the main window. I can move the dialog around, scroll around the image etc and have it update in real time until I do “OK” to confirm the filter application or “Cancel” to restore the previous state of the image.
@@lawrencedoliveiro9104 yes, as I said, it depends on the dialog/window that the programmer chooses. If you choose a synchronous dialog/window (usually used for simple yes/no prompts or alerts), the window in the background cannot update, otherwise it couldn't be synchronous in the code. Of course this does not make sense for a preview dialog and therefore the dialog that you describe is asynchronous, which makes it a bit harder to handle in code.
You can see examples of synchronous dialogs in web browsers - a simple prompt/alert is synchronous and therefore you cannot use the tab in the background while it's active (although this is not a typical example as I suspect it's simulated in the browser, because for example video playback continues during an alert). Error alerts and prompts in Windows apps typically used to be synchronous.
Sorry, it is me again.
May I humbly ask, if this is, in essence, NOT a video about GUI?
Dialog is sometimes used instead of window.
@4:07 :sigh: now I miss my Atari ST GEM Desktop...
Why are you so visibly uncomfortable in mentioning Linux? You struggle to avoid bringing it into question on this video. Is it because Linux has differing GUI toolkits like GTK+, Qt and wx?
yeah, i was noticing lack of GNU+linux as well
Don't you mean "Under the Bonnet".
Bonnet is children's version (explanation) of hood
2:24 @ TJ "Henry" Yoshi
in fact Windows programs have both: a message loop and event handlers for the windows. but most modern solutions burry the message loop deep in the framework and expose only the event handlers to the programmer.
"burry the message loop deep in the framework and expose only the event handlers to the programmer."
That is because Windows is so poorly designed. Burying like that are work around that flaw.
@@philipfry9436 Could you elaborate why it is poorly designed?
@@MrSparker95 copypasted from my other comment: The window proc stall the thread forever and you have to use co-routine or threads to get around that.
@@philipfry9436 You mean that if YOU stall the message loop by your heavy function then it won't be able to process other messages? Well yes, that's single threaded program design, basicly UI runs in the main thread in Windows. What are alternatives to that?
@@MrSparker95 I am not stalling the message pump, it get updated every frame. Drag or resize the window around and it become stalled by Windows (the operating system).
You need co-routine, threading, or calling your update function from the window proc to work around that. It's a garbage api. There should be only a GetMessage call. No dispatch. Dispatch is widget or application space.
thanks you for such a nice video
Surely frameworks handle these things and every Dev isn't writing code for showing part of a window??
If you're developing for Windows and have decided to punish yourself by using plain Win32 APIs, you have to code all the redrawing yourself, unless you're using some built-in controls. Even in that latter case, you still have to tell those controls when to redraw.
MacOS AN IOS have a responder chain and keeps passing it up until a class wants it. Windows are classes in macOS. As are all UI elements.
It's turtles all the way down.
This is is very helpful knowledge for a software engineer
If it were accurate, sure. It's not though. He's not explaining the different roles of compositor and window manager and assumes GUI toolkit does everything, which is not true.
Nice haircut!
Thanks you too lol
7:45
NO! You don't use PeekMessage() on message loop. You use GetMessage().
I think you can still use PeekMessage to check for any available messages. but I do understand what you mean
@@brod515
Yes you can. But using it as your main message receiving call at message loop results in either busy waiting or unnecessarily complex code (or both).
Steve's code doesn't work, because he doesn't check if there is message in the queue and therefore tries to translate and dispatch messages which aren't there.
8:58 direct it to a specific window behind...
Window repaint is supposed to be a quintessential NOT MY PROBLEM thing ever since the stone age. Get with the program and use a compositing window manager like, you know, normal people.
03:57 Windows are called forms, aren't they? 😉
English caption is awful. Why won't you just enable auto generated one.
I just posted the same thing. The English caption is awful in all of his videos. I don't know why.
Hello!
Ah, looping through WF_NEXTXYWH trying to work out how to draw part of your windows. 'Now you could if you were a lazy programmer redaw everything... and just clip to the area'... Guilty, m'lud.
Press F to pay respect to wobbly windows on old GNOME desktop
I miss my wobbly windows and the fancy alt-tab switchers. All I got now is what's offered by compiz. So, you know, transparency and drop shadows. So boring.
@@aitchpea6011 But compiz is what powered the wobbly windows in the first place...
I'm a bit surprised that you said under the hood instead of under the bonnet, was that for the benefit of Americans or do you generally use under the hood in a non-car context? Or do they not call it a bonnet in the UK anymore?
Alert boxes and disabling components can be a way of forcing the user to obey your software
And they will break it anyway
source code ?
What source code are you looking for? He is talking about code you don't need to write... the stuff that runs under the hood.
@@VBKing2 hahaha
This video does a great job of highlighting how "Windows" is a terrible name for a GUI!
I get that it was 35 years ago, but what were you thinking, Microsoft engineers?
Also, 'Excel' should have been named 'Table'.
they made squillions of dollars so it wasn't that bad
Windows always been the name for the screen viewports since the beginning of gui.
I think this was back in the day when Microsoft rarely tried to do clever names for most of their products and just named them after what they do. Windows, Office, Disk Operating System...
@@YaroKasear ahw nice
Nice RUclips cutout from Dave’s Geek Stuff 😉
Ah, the good old times when we had to include a message loop in all our programs. Today's kids don't need to know what a WM_PAINT is anymore...
Ahh, yes yes, WM_ERASEBKGND and WM_PAINT were always such a pleasant boon to productivity!
while(!quit) :/
I think that was deliberate to help non-programmers understand. It is not obvious what logical not means just out of the word "not" and it is not obvious that "!" means logical not.
C99 lets you use the words “and”, “or” and “not”. They’re defined in the standard include file iso646.h.
I regularly see Python code like: *if flag == True: dosomething* and *if len(s) > 0: dosomething* . That makes me cringe.
Ratio of time spent reading versus writing is well over 10 to 1. Code readability is more important than saving few bytes.
while(!quit) is perfectly readable. A reason to prefer this form is that global symbols are initialized to zero and therefore is automatic initialized correctly. Perhaps you nooblets should wait to be more experimented with programming before dissing experts.
Anyone else tried to program in Java, just given up and moved back to Python or other C family?
Learned just enough Java to pass my OOP class and never touched it again.
I am the only one that actually likes Java?
Java is a C like language
A GUI is pretty concurrent by nature. Try a concurrency oriented programming language. Try Erlang/OTP.
Wait, Python or C family? How on earth is Python C-like at all? Java is strongly influenced by C (semicolons, curly braces, syntax conventions like for loops), but Python isn't even in the same species, let alone family.
Although I will admit, I've never been a fan of how strictly the language enforces OOP, I much prefer C++ where I can easily drop into straight functional C if I need to, along with the ability to (mis)use pointers.
I've always said women can't multitask, they can just handle more events in their single threaded event loop.
i like ncurses as a gui
How can you? I have never managed to write anything without being utterly frustrated.
@@MladenMijatov i am a little more patient with it , i mean i was going to suggest QT
Please make a video on utility of structures and unions.
That's a lot of windows
10 of them
Ah, that well known operating system, Atari OS... 🤷♂️
AntneeUK It was called TOS. (And the GUI was actually GEM.)
Coding your own GUI's isn't too bad. Not as hard as coding your own IDES (something else I also do).
here is your cookie 🍪
My co-workers always scolded me for using Comic Sans MS as a programming font in my IDE editor, but Comic Sans is making a comeback.
@@PaulaJBean NOOOOOOOOOOOOOOOOOOOOO
IDEs or IDES??
@@yash1152 IDE's.
Anyone know of a good book that addresses gui design from a theory standpoint? Obviously there are a ton of books on specific toolkits and computer graphics, but I'd like one that addresses all of the chaos involved in human-computer interaction; e.g., object-drawing and event handling.
There are not a ton of books that specific toolkits, lol.
The closed captioning is awful in this video.
out of all the things that are special and how a OS works that i know and can make sense of... i really struggle with what happens after you press power and the BIOS appears and how that part works... like i wonder how putting power in a "abacus" is equal a BIOS, even tho i can understand how mathematically the Abacus can make games play in 3d... like why can i Type a letter or a command on the BIOS and why are a set of Characters equal a function... how did they define that in the BIOS?
so yeah, why do pressing power = seeing a BIOS and being able to type... like you have to define letters some where, you have to define a screen able to display that... an OS cannot do anything without the BIOS having defined that for it before hence how the heck did they make a BIOS without a BIOS?
@moo i guess what i mean is, when you press power on, to when you see the BIOS, like what happens to get there.
@@theepicslayer7sss101 i would guess that bios is like machine code, so basically that data is read and took verbatim by the circuitry in the form of electronic signals, so, highs and lows on voltage, logic gates, transistors and stuff. So, the hardware level stuff rather than some yet another software thing.
I make a guess now: when You need to redraw a more complicated screen with lot of stuff to put into it they use the GPU to speed things up.
He failed to explain what compositor is and what's the role of window managers. In properly designed operating systems, not Windows, application will request a buffer from window manager/compositor. This buffer is located on GPU memory. Compositor will then make sure that order of displayed things is correct and redraw only "dirty" parts. Toolkit in turn needs to make sure to update that buffer when window manager requests it to. That way there are no solitaire effect Windows often gets when application stop responding.
Windows works in a different ways. There system will send bunch of messages to the application window and if there's no response for few seconds you get that not responding message. Among these messages is the repaint request. So if application is not responding nothing will happen and you get solitaire effect. In this repaint message windows will provide a "handle" on to which content will be painted.
Hardware acceleration is basically where buffers are located and how fast they are painted. If possible compositor will store those buffers on GPU and then create final buffer that will end up being displayed by layering all the application buffers in proper places. Eventually it just tells GPU "display this". When there's no hardware acceleration, all the layering is done on CPU which is not specialized for this kind of operation and ends up being slower.
Leonardo Di Caprio's cousin😃
other computerphile commenters: Something something CPU something library in C something CSS/JS something something Atari ST GEM something "frames" something shirt compression
me 15:45 onwards: **scrolling window up and down obsessively** _Ah?! I'm moving an already moving video! This is real-life magic!_
Chrome probably keeps drawing of each tab in seperate piece of memory
As soon I saw that he had dyed his eyebrows black I thought he was on drugs. And then he said "windows does something better" and I knew he was.
Oh for goodness sake, use a camera tripod! They are cheap
tripods make videos feel more rigid and tend to drop attention faster
there is a reason so many movies/shows intentionally avoid stabilization
Could you please not use a white-board marker on paper ever again? The sound is really annoying.
They use those in most Computerphile videos. :S
why are you talking about it?
That's all he does in his videos, ramble on and on an on, maybe with a little doodling. Just do a quick doodle, explain what we're seeing and then just code, code, code, but I'm not sure he's able to do that.
7:01 I think he meant command-Q
1:50 WTF are you talking about. Windows is garbage. The window proc stall the thread forever and you have to use co-routine or threads to get around that.
Philip Fry
Not strictly speaking true if you know your way around pointer routines.
@@colin-campbell Your solution to the garbageness of windows is to add a hack that call user update function within the window proc?
We got the operating systems that we deserve...
Philip Fry
You’re close to the solution but overdoing it a little, think about the system in a cleaner fashion.
@@colin-campbell There is no solution. Let alone a clean one. Windows can't be fixed. X11 is _literally_ easier to use.
Philip Fry
There is a solution if you know your way around pointer routines and maybe a little bit of XML injection if you’re not great with pointer routines.
U can call the window, frame.
the frame is just the outmost container for the content, so you could argue that the frame in itself doesnt really represent the window, rather, as the name suggests, the window frame.
that just builds on the window analogy and refer to the frame of the window, then that thing holds multiple "panels"
window is the whole thing. then the windows GUI has multiple windows hence windows
macs has multiple baskets which hold apples
linux has multiple pockets which hold lints
The Grasslehoff, the outmost content is most of the time called “Desktop”.
Most OS’s have multiple desktop frames with multiple window frames.
This is garbage. Nothing about different ui design (immediate mode vs retrained mode). Different ways of hit testing (color picking, etc... garbage
Friends : I want to pursue CS degree what do you think?
Me : Just watch computerphile from 0 till end...
Nah... is there a lot of elaborate opinion about GUI poured into a basic speech bubble?
No offense, but please do not do that.