What Is an Image? - Cpp Computer Graphics Tutorial, (GPU, GUI, 2D Graphics and Pixels Explained)
HTML-код
- Опубликовано: 5 фев 2025
- cppcon.org/
---
What Is an Image, Anyway? - Cpp Computer Graphics Tutorial, (GPU, 2D Graphics, GUI, Colorspaces, YUT, Planter Formats & Pixels Explained) - Will Rosecrans - CppCon 2022
github.com/Cpp...
We all have at least a vague idea of what an image is, and that they are useful. It has even been proposed that 2D graphics functionality be added to the C++ language. But many developers haven't gotten too far into the weeds with the topic. What is an image? Sure, it's a 2D array of pixels. But what are pixels? What do the values in the pixels really mean? And, are you sure it's always just a single 2D array? Or that every pixel format actually stores discrete pixels? The reality is that the concept of an image can be blurry and filled with corner cases that will bewilder the newcomer and exasperate even experts.
Take a journey from the absolute basics of images in a computer, through a brief survey a more advanced topics. How much memory you need to allocate for n pixels. Alignment of pixels for efficient processing. Types like 12 bit int and half-float that aren't native C++ types. Almost (but not quite) compatible pixel formats across some common API's. Colorspaces. Colorspaces, but as defined from a slightly different niche. Colorspaces. Subsampling used in video formats. Some of the complexities from using a GPU, like compressed formats and mixed type formats. The performance implications of needing redundant copies of pixel data to interoperate between different libraries and API's. And how much of this you need to be aware of when making API's that might get used by developers with use cases you hadn't considered.
---
Will Rosecrans
Will Rosecrans is an independent software developer. He has recent experience working at global scale on the EdgeCast/Edgio content delivery network which handles billions of web requests and video streams. He worked on build and CI systems for C++ server software and his work led to a patent on rapid configuration propagation. He previously worked in the visual effects industry, working on pipeline technology at Moving Picture Company, and was responsible for the general care and feeding of pixels and servers for advertising, TV and film projects. His academic background in film production includes the UCLA film production Summer Institute.
---
Videos Filmed & Edited by Bash Films: www.BashFilms.com
RUclips Channel Managed by Digital Medium Ltd events.digital...
#cppcon #programming #gpu
This talk feels generally more relaxed than an average cppcon talk for some reason which i like a lot
Great talk. Hopefully it won't go unnoticed by the standard committee.
I can tell this is a good talk because I am not a programmer or an engineer and I got hooked 🙂
Great talk. Having worked with image processing in C++ I can feel that man's pain.
As many as 5-10% of people have some form of color blindness, but even when we talk about those differences we easily forget to label the colors on the slides! Would be interesting for presentation software to offer hints when it notices two WCAG-indistinguishable colors next to each other
C++23 has mdspan which should be useful to reference an image. In addition to mdspan, I would like to see something like a compact matrix class. The hardest part is that you need to multiply width and height to get the storage size, which may overflow if you're not careful. It would be useful if the library could handle that and throw an exception if I get an overflow.
Correct me if I'm wrong, but I don't think mdspan has tiled layout features? The options are just striding and col/row major.
mdspan... ah yeah, that crap that i always forget the name of :P
std::mdspan currently supports strided memory accesses, but not non-coherent memory layouts, so it's not super useful for images that separate color channels into different regions of memory.
@9:29, to be more specific a Question Mark block is made up of 4 images (sprites). Every tile is 4 sprites. Big Mario is 8 sprites.
This is a beautiful talk.
This was a great talk.
Amazing talk!
Amazing talk! Thank you very much
YUV is accounting for sensitivity in your eyes. You said it earlier, green in a bright room sure looks different than it does in a dark room. It makes less sense to pretend 255 has some sort of unit attached to it and the green recipe calls for 255 units of green. Instead, YUV starts from black, and white and calls Y the brightness of the picture, and any of the actual color is like a weighted formula with u and v as coefficients because the sort if the color was neon green, then that implies it can't be that dark. Black, and white is only taking the brightness into account. I think we tend to assume that you get BW by setting u, v to 0 but that's just a color tv that wouldn't work well. Making them related has to be based on something which is our sensitivity to color. By dragging around the Y value, they killed two birds which is why I'm guessing I remember sometimes a color channel would be broken but it still displayed BW fine.
As for translating between image sizes, just have a function that takes a normalised float/integer as it's value, it finds the 1 or more pixels of the source image and combines them to create the colour that should be in that spot to then pass back, then can just divide the target size by the source size to get the normalised interval to multiply the target xy pixel by to pass to said function
16bit floats - many/most compilers already do support that cause the hardware used does:
16bit floats are used in ARMs and in x86_64 for the past 10 years or so (that is even outside of its use in SSE/AVX extensions).
Yes but only when you do total cryptic instrinsic operations.
Great domain knowledge background video. Images are so much more than 2D arrays of data. Going beyond that is probably way too complex and non-standard to belong in the stdlib. Anything that could be standardized probably is strictly some form of matrix type and being able to define unusual pixel data types that aren't byte aligned. None of this, to me, qualifies as an Image - just data arrays.
Nice meet, interesting conference
What i can say about all this, is that the source of all problems comes from technology manufacturers.
Maybe instead of trying to find a way to wrap all this shenanigans, it is imperative to sit-down with those who have a foot in deciding how things should be represented and standardize it. Or at least adopt a way of stable API, or maybe a plug-ins platform, the way Qt implements its cross playform promises.
It is definitely a huge task to add this as a library to C++.
DirectX has different way to present a buffer, Open Gl also, and last Vulkan, vut they all represent a buffer!!!
I just want reflection-at-compiletime, so that I can tweak my shaders automatically. Also, allow for features that make it easy to represent SOA as AOS so that both readability and performance are brought to the table :)
Please :)
On the Amiga, nobody would even want to iterate through the screen image as the presenter suggests. The whole point of this is to have separate images for programs to work on, independent of one another, and only being composited into a single image at display time. Iterating through the combined image makes no sense.
If one wants to discuss pixels on the Amiga, I'd think the way the bits of a pixel are stored is more of an issue: the Amiga stores each bit of an image in its own bitplane, so a six-bit image (the maximum on older Amigas) would be spread over six bitplanes that can be located anywhere in memory. Good luck getting a pointer to that...
I thought time and geodesy was complex because of historical conventions and other exceptions, but the color is not bad either...
Very little c++ particulars, in other words a great talk. Engaging presenter.
4:05 ... an Image is defined by an array of unit8_t rather than just and array of the class Pixel? WHY?
Don't forget hyperspectral images!
3:13 I thought this was an accidental meme. Nope, USCPSC is actually posting memes (pretty good actually).
36:34, is it really that complicated? I think of the RGB we store as just the values between the maximum possible brightness of that colour vs the minimum possible brightness, translating that just requires identifying the min/max brightness of the target, dividing it by the RGB min/max by to get the intervals and then multiply that interval by the stored RGB to get the closest match the target supports, here's a more math based example:
Target RGB min/max = 0,50:0,50:0,100
Source RGB min/max = 0,100:0,100:0,100
Source RGB values = 33:75:50
We'll start with blue since that's a simple 1 to 1 translation:
100 / 100 = 1, 1 * 50 = 50
Now we'll move onto Green as that's also relatively intuitive:
50 / 100 = 0.5, 0.5 * 75 = 37.5, we round up from 37.5 to 38
Now we do red:
50 / 100 = 0.5, 0.5 * 33 = 16.5, we round up from 16.5 to 17
The target expects 17:38:50 to re-produce the closet equivalent to the original source values
**Edit:** btw with this method you want to avoid floats as they provide less accuracy than simply treating the top bit of a long or long long integer as the min/max of the brightness of said colour and treating every bit before it as the floating value part during translation to the target colour space, the nice side effect of this is that it's also much faster than using an actual float even if it's the same operation of math
Unfortunately, you have several misconceptions:
1. Floating point processing has been _faster_then integer processing for a decade+
2. 8-bit/channel is _not_ sufficient. 10-bit/channel or even 12-bit/channel is sometimes needed to _preserve_ precision & quality.
3. C++’s lack of native float8 and float16 support forces everyone to reimplement this functionality.
4. You are ignoring gamut.
5. Not everyone works in YUV.
Only at 52 min, you explained why all that rambling mattered to the audience. Maybe the talk could be restructured, so you keep going back to the problem: "what should a `std::image` cover, what use cases should be excluded?"
Maybe not everything belongs to the standard library. Considering that we got a "mediocre at best" hash table added into the standard library, should I trust the committee and the library authors to create a useful `std::image`? Do we really want another `std::async` situation? What do existing C++ projects do to deal with the "lack of a `std::image`"? Yes, there are infinite combinations of details, and we can't even attempt to try to solve half of them. Without a scope, a `std::image` (or `std::image_view`) would probably be a bad solution for every single use case.
If multidimensional arrays/vectors get added into the standard, the need for a `std::image` fades away quite quickly.
I agree that "what should a `std::image` cover" is a good topic for a talk. But I definitely didn't want to make it seem like I was trying to make a formal proposal. Images are a useful concept for C++ developers who are using existing libraries, working on new libraries, or thinking about standardization. I sort of shrug when I talk about what should be in the standard because I'm not at all convinced that my own opinions are correct so I don't think I should be at all prescriptive.
As for multidimensional arrays negating the need for a std::image, I disagree somewhat. std::string is a useful vocabulary type despite the fact that we could have gotten away with vector as the de facto string type. Libraries would use multidimensional vectors in slightly different ways as long as there is any possibility of using them in slightly different ways, and then you are back to writing a bunch of wrapper adapter interface glue between every library.
Can you please tell me a bit more about the std::async situation, what's the matter with std::async?
Indeed. There is no shortage of decent image libraries and frameworks and if one of those doesn't work for some reason, it's not exactly complicated to roll your own container for pixel data. And for any serious purpose involving graphics (think of games and whatnot), the majority of business logic happens not in C++ but in hardware, shaders and various low-level system dependant APIs and rolling a direct support for that in C++ would be like adding in Sound Blaster support.
"If multidimensional arrays/vectors get added into the standard"
They kinda exist in C++23 - multidimensional array operator and mdspan (gosh how bad the naming of C++ has become).
If someone shows up to a talk and is continually asking themselves "why does this mattter to me?" without the will or ability to generate some hypotheticals and connections in their mind on their own, that's not your problem. The focus should be on demanding people be better, not a cynical and strictly utilitarian pandering to the lowest common denominator. For example you, taking everything he said and condensing it to "rambling". That's a you problem buddy, don't try to put it on someone else.
Rgba should be standard
Edit: ok thank you for including alpha lol
C++ standard library shouldnt be offering what libraries like Vulkan already do better, C++ should just offer objects that would be used by graphics libraries API.
Vulkan actually would be the best thing to use (or start with) if you really wanted graphics to be integrated into the C++ standard. Even though Vulkan is not a graphics API but rather a GPU API.
? No, not Vulkan - that is still mostly centred around old c-style APIs. It is horrible.
watched up to 20:32
is it me or does he sound like "Gilfoyle" from Silicon Valley.