Digital Images - Computerphile
HTML-код
- Опубликовано: 11 сен 2024
- How are images represented in a computer? Image analyst & Research Fellow Mike Pound gives us a snapshot. (First in a series on computer vision)
Digital Image Capture: COMING SOON
Universe of Triangles: • A Universe of Triangle...
Math & Movies - Numberphile: • Math and Movies (Anima...
Real Life Holodeck: • Real Life Holodeck wit...
/ computerphile
/ computer_phile
This video was filmed and edited by Sean Riley.
Computer Science at the University of Nottingham: bit.ly/nottscom...
Computerphile is a sister project to Brady Haran's Numberphile. More at www.bradyharan.com
I just want to say this guy explains everything extreamly well and clear, really good!
I like to watch all your videos. One suggestion; if the show is a conversation between the camera man and the host. Please give the cameraman his own microphone. Then all of us can understand what the cameraman is saying. OK. For me all I can do is guess.
If it's not already planned, you should do a video on the jpeg compression. I learned that in design school and I think it would interest a lot of computerphile/numberphile viewers.
Yes please! Compression of images is confusing. Quantisation is already confusing. Are there other methods involved as well?
I like that guy. He manages to keep my attention but also go beyond the most basic explanation
4:34, someone in the background breaking a plate? :P
Sorry Linus, you've been out done...
It’s not even close
Ah, the pseudo-2D-array index formula. I figured out how to generalise it to higher dimensions when I was at uni.
For 3D it's: i = width*height*z + width*y + x
For 4D it's: i = width*height*depth*w + width*height*z + width*y + x
In general, each coordinate is multiplied by the product of all the lower-dimensional sizes and the results are added. The only time the highest-dimensional size gets used is in your boundary check code to make sure you're not asking for a location outside the array.
Pseudo-n-dimensional arrays are a handy way of circumventing programming language limitations on passing arrays to functions. When faced with a language requiring fixed sizes for all dimensions after the first (I recall C being such a case), you can instead pass a linear array (technically a pointer to one) and a few size integers and use some simple maths in the function body to work with arbitrary sizes of multi-dimensional arrays.
The birth of a legend
Can you please make a video or a series of videos about file formats, more about how files are stored, and how the bytes of each file are laid out in memory? Kind of like this lecture, but also for different file formats, such as images, videos, audios, pdfs, zip, tar, etc. Some possible topics to discuss would be different file headers and metadata of each file format, and the lay out of data in the actual file. That would be very interesting and informative. What makes a file a file?
Could you please get a second microphone? I cannot hear the questions.
or maybe a subtitle
3:03 The internet ruined my mind
What would you want padding for? Not criticising or anything, just curious.
The brighter than full scale colors are probably used when editing these videos in so called "video levels" range which has a small headroom to contain overbright values, and impossible colors like 'saturated white', where details can be recovered from with contrast/curves if needed.
Turns out it's not that complicated.
So how do you access the red color exactly?! Or blue?
Guys, it's pronounced "gif"
;)
pretty sure it's "jyfe" ...like knife but with a j
lol
Brice Shatzer So like jive?
No it's pronounced 'gif'
For a long time, I used to say "Jif is peanut butter, after all, it's not jraphic interchange format." Then I heard the inventor of the format said it's "jif" so I had to amend my thinking. And I was disappointed with that guy.
rchandraonline If i remember correctly it was not the creator, it was ONE contributor involved with the development of the format. Everyone else called it gif. With an actual G sound. Or else, it would be spelled jif!
There's an error in the very last bit. In order for it to be correct you needed to start with 0 and not with 1. So the correct formula for starting with 1 would be (y-1)*stride+(x-1) +1
Technically, alpha is alpha :) It has no intrinsic meaning, but can be (and usually is) used for blending, so people assume it's for transparency. You often get an alpha channel whether you want it or not for optimal memory alignment.
It could have been explained better, as some non-tech people will be confused with all this bits and channels and tech people already know how basic color information is stored.
***** Yes they would
*****
True,you can't,but when you have someone come here and ask ''What's a digital image?'' you have a problem. Everything can be explained to everyone,the question is how much in depth,and how far off the topic at hand one would need to dive in order to make sense to someone else even of basic digital image storing techniques to someone who has no technical knowledge of it at all.
That's why schools have grades,learning curves and increased level of different source information as you move up the curve.
You won't see a first grader walk into 7th grade and expect the teacher to include the information of past 6 grades just to enlighten this clueless child.
And you shouldn't expect this video to explain anything more than the subject it's devoted to.
I would love to see a follow up of this video, maybe explain a bit more about the most popular formats
These are raster images there are also vector images witch work pretty much how you would construct a shape on a graph in maths. Though I'm not sure on the details of how these are stored on a computer. Vector images are very useful for something you might want to have resized often as they don't lose quality as you make them bigger.
how about Vector images? How do they work?
In a very simplified way, they store the instructions on how to create the image. Rather than saying 'place red pixels at points(.....)', a vector image roughly says: 'there's a red line between here and there'. Therefore they can be easily scaled up/down without loss in quality as they are based on relative position of elements rather than information on individual pixels.
They store vectors instead of pixels, which describes how to draw the image.
Vector images describe coordinates to draw polygons. For example a triangle, so it describes three 2D-coordinates where the corners are, and the color of the triangle.
Lots and lots of those polygons drawn over eachother will create a picture. And these picture can be scaled up in size and re-calculated, so they won't become "jaggy" when you make them larger.
This is really great for some types of images, specifically artificially generated images with few colors, but is pretty pointless for photographs, for example.
Think of it kind of like "clipart", right?
interestingly, at least with .svg files, they're actually all XML, so you can open them with a text editor! i actually made one this way, it was easier for me then trying to learn the controls of Inkscape or Adobe Illustrator.
mind_combatant You can make simple shapes that way, but for more complex scenery you will have to use Inkscape or similar programs.
would love a video about different image compressions, and what their basic ideas are, also how some audio codecs work in principles would be cool
i believe this is the first video where Mike appears?
Why is it that the X value is on the Vertical axis and the Y value is on the Horizontal axis, when in normal graphing it is the opposite?
Well the idea was that the x represented the value of the length of the line next to it but I'd agree that it was potentially confusing! Apologies!
Sean Riley Oh, I see, I understand now. I misunderstood the Stride as the size of the 4 bytes for each pixel. The stride, being the width of the image, times Y, the row you want, plus X gives you that pixel, for when an image in represented in a single line. Okay, thanks for the reply.
There can be as many reasons as there are for "normal" graphing to be opposite.
Why is some images formats BGR vs RGB and BGRA vs ARGB vs ABGR vs ARGB...? It's all in how people "think" about the problem. There's an endless sea of inconsistencies because people developed new technology all over the world and didn't have the means to communicate a "best practices" with each other.
Cadde Endianness of various CPUs can be blamed for a lot of that.
***** And people made those CPUs. It's not the molecules that make up the structure of the CPUs fault that they are arranged that way.
Why index an image as a 1d array? Why not a 2d matrix where perhaps the x, y point in the matrix corresponds to the cartesian location of the pixel in the image? Or perhaps keep the location as a 1d array, and use the second to index the channels? Is this not as efficient?
This video is quite applicable to what I'm learning now, I found it very interesting! Thank you!
Sean, if you are going to do a bit of talking in the video, you should also wear a mic so we can hear you properly
Did something break at 4:35?
I need subtitles..
4:34 what happened?!
So, he mentioned the special rules for greyscale images, how instead of having each pixel tell all three values it only tells one in grey.
Well, couldn't this logic work the same for monochrome images? Couldn't you give each pixel "grey" values, and then specify the universal hue in the header?
Nooooooooo!!!! SO close to understanding what stride is. If I have a 12-bit (i.e. 16 bpp (bits per pixel)) monochrome 1280 x 800 image sensor and need to convert that to an 8-bit monochrome image, what is the stride? Is it simply the width? Width * bpp (i.e 20,480)? Width * Bytes per pixel (i.e. 2,560)? Or something else?
I can't get my output images to look right. They are indeed 8-bit monochrome, but there are artifacts. Upon close inspection, it seems there are 1 pixel wide vertical bands of 0-value in my image, and indeed when inspecting the buffer in memory whilst debugging, every other byte is 0x00. It seems the LSBs are getting truncated by the imager or something, but still ending up in memory. How do I address this? This actually seems like it may not be a stride issue. I'm working with a C#.NET app.
In no point of the video have you even mentioned the word bitmap.
3:50 . "Even brighter red" is very useful for HDR rendering. It is actually very common.
but it doesn't work in a way that normal human will assume when you say "even brighter red", which is why he said it's not possible. because it really isn't, the whole HDR thing is a trick to make it seem that it is possible.
MidnightSt also it doesn't make any sense in terms of monitors. nomal monitors are 8 bit monitors, so at the end of the day your 12 bit HDR photo is only an 8 bit image.
Yakhashe which is what i said, only in different words. ;)
but yes, your version is a better explanation :)
MidnightSt You are implying that "normal humans" are uneducated?
Is this a professor elaborating his field of expertise, or a sixth-grader student answering his first computer science quiz? Giving incomplete or incorrect information on a subject is more harmful than giving no information at all. :D
Yakhashe Incorrect. First of all, there is no such thing as a normal monitor.
I think that you are looking for a term "True color", as a most common color depth in today's consumer devices; Those are 24-bit per pixel.
In your defense, you might have thought 8 bits per channel; but that's not a regular expression. (Pun intended ;) )
this made me think of the pictures going around a couple of years ago that had malicious code written into them, that executed when whindows loaded a preview of them.. would be interesting to have some kind of exxplanation on how that worked. was it metadata or what?
If every pixel is encoded in 3 channels (r/g/b), black would be every value of these channels 0 and white would be every channel on their max value...
but what would be grey or "darker red" / "red mixed with white"?
If I experiment in a Image editing software I usually could measure the r/g/b values of a pixel but also the "brightness" where even if red is on its max. value (like 255) if I set it's brightness to 0 it still would be black.
I had difficulty hearing the questions. Perhaps this could be improved in future videos. Also, the speaker didn't explain some terms, like alpha channel. Maybe they were explained on a previous video that I missed.
and when you "liquify" a picture in photoshop whats going on behind the scenes?
How old is Mike?
May I suggest a video on how different file types record/interpret/compress/whatever the image?
Am I missing something or formula should be p=(stride-1)*y+x
p = y * width + x
x = p mod width
y = p \ width
C++ example (with no error checking/prevention).
struct RGBA8 // an 8bit per channel pixel with alpha channel.
{
unsigned char r, g, b, a;
};
struct Image // an image of RGBA8 pixels
{
int width, height; // "stride" and number of rows.
RGBA8 *pixels; // all the pixels in the image as one long "list" ("array").
RGBA8 getPixel(int x, int y) // returns the RGBA8 at location (x, y) in the image.
{
return pixels[y*width+x];
}
};
what if we took bits from a file, like a music file, then we create a picture with it? how it would look like?
How about *.gif* animated format?
It is not Brady's voice at the background, or is it?
6:30 look at is hand. that is a smooth as fuck cut.
GIMP is an awesome for photo editing. This video was really interesting thanks for making it. :-)
Could you please do the explanation for audio? From low quality mp3 to master file, how different can it be?
Absolutely amazing!
Will there be anything about bump/normal mapping?
If I converted RGB image file to header files in c language and then can I do some operations on it?
If yes than how?
That largely depends on how exactly have you converted it into .h file. You could, say, convert it into plain text and declare an array of pixels like
const char *pixels[] = {"100","72","230","91",...};
You can directly store it in .h file as an array of hex values, like
unsigned char pixels[] = {r,g,b,r,g,b,...};
Or you can just have a .h file full of those numbers/chars with some delimiter, in wich case it's no longer a true .h file, but rather more or less a raw data stored
Please do "discrete cosine transform" of the jpeg images. So so good
Update the information about digital image capture video..
7:42 That's a neat trick.
Huh, just noticed that the new auto-play pauses when you're down in the comments.
I would have liked seeing an explanation of that padding-thing.
Sir could you please tell me how the images stores in memory actually in deep and how digital data of array stores in memory ?.
Please Sir...
8 bit color would be allocated as R-G-B 2-2-3 (or similar, whichever you'd like to have the extra bit). Same would be for 16-bit depth graphics, with 5-5-6 or similar, or 4-4-4 plus alpha. The guy seems to be describing 24-bit color depth.
Interesting stuff... I'm not too sure on that formula, though (and I have done per-pixel image processing stuff before). To me, it should be something like _(width * stride * y) + (x * stride)_. Unless I've misunderstood what "stride" is referring to?
Stride refers to the width here. Since it is based on indices the 'stride' as stated in your comment will always be 1.
Oh, okay. Thanks.
Can anyone tell me what accent this guy has? I noticed that his th's and his r's are labiodental.
These videos are great, thanks, but imo you'd gain a lot by giving the guy asking the questions a microphone as well. Can't really hear him so lose some of the explanation.
i miss the old vga standard 256 color palette standard. A color picture with 1 byte per pixel.
You can still use it. Both png and gif support it as do most image editors.
To me you'll never beat the original bitmap format. It even had compression, not that anyone used it....
But why did it store the pixel data upside down?
JuddMan03
That my friend is a mystery only the ancients know.
Actually it can be right way up if you give the bitmap a negative height but most programs don't know this.
It originates from Presentation Manager in OS/2; it was run by mathematicians and you may notice on x\y graphs 0 is at the bottom and bigger numbers go up
ight. Thus bitmaps were originally to be displayed on a screen treated the opposite to how we do ours. The file was still read from top to bottom.
Very cool.
Sean, Please mic yourself. Your comments are to quiet compared to the person being interviewed.
I could keep watching these videos, yet it won't make me a better photographer.
I still don't understand how pixels are represented by binary code and electrons and then stored.. like where? How? My phone has 100s of pics and videos of moments in time that have now forever past. How tf do they just stay there for me to look at whenever I want. Same with this comment. It's now going to exist even when I die (unless deleted)
After learning that you can make an image grayscale if you average r,g,b i wrote a small python script to test it:
from PIL import Image
im = Image.open("your_image")
x,y, = im.size
img = im.load()
for x1 in range(0,x):
for y1 in range(0,y):
r,b,g = img[x1,y1]
avrg = (r+b+g)//3
img[x1,y1]= (avrg,avrg,avrg)
im.save("grayscale.jpg","JPEG")
JPEG is a lossy format, your pixels written are not actually averages at all.
Cadde
That's kinda moot anyway, since JPEG, like most raster images formats, doesn't actually save individual pixels.
also assumes that r, b and g are weighted equally when it comes to our eyes, they are not, we are a bit more sensitive to g, so your average which may be fine for your sample program is actually not that accurate... en.wikipedia.org/wiki/Relative_luminance
i actually stumbled upon this error when trying to figure out whether to add white eyes on top of a arbitrarily colored sprite or black eyes, i figured i could just get the grayscale and if it falls below .5 use white, if it falls above use black, however when the color was mostly green this failed miserably as green was just too bright, even when it shouldn't have...
vlademir1 How is it moot, are you saying the saving of the data and restoring of data into the image is lossless? Because that was my point.
I am constantly amazed how many apparently intelligent people cannot pronounce "th". (And why do university educated people commence an idea or sentence with the linking word "so" - this chap is not the only computerphile presenter who does this)
Submission fro future episodes: Zero knowledge proof
how about, how something is pronounced depends on the people pronouncing it. If a majority prefers a certain way that becomes the norm. If there is no clear norm. Both are right.
SubTitles please ...
Greet to see a lefty geek
Couldn't you just do: PR2G45B675, PR3G276B34
P equals pixel
R equals red and the number the red value
G equals green value
b equals blue value
And when you get to the end of the first row of pixels just put an E at the end for end of line. You don't even need the comma between each pixel code either.
The aim is to represent each pixel number with the least amount of characters as possible.
Or you could use an x and y location code to tell you which color it is based off a color wheel that it would reference when opening the file.
For example
PX125Y23, PX46Y234
That's two pixels and the exact color the program will understand the xy code to draw up the image, it will reference a color wheel or square with millions of combinations of x and y to choose the color from like GPS coordinates it will tell the program where to find the color of that pixel off a color wheel or square.
PR2G45B675 is 9 bytes of text which is 72.bits. no better than 32. also a color wheel that includes shades? they are larger than you might think.
but yes, you could do those things and create a working image format with them.
How many characters does it take to make a pixel, the lowest amount.
I had a square color xy color chart that my painting program used, you move the mouse around the square and it let's you pick a color, it also showed an xy number too.
You could simply input the xy number for each pixel and the image opening software would know what color to pick for each pixel.
In the video they made it seem like there was way too much unnesary info and text in the image file to describe each pixel, you could bring it down to six characters, 567275.
Maybe a bit more for extra shades depth but that should give you a lot.
This was supposed to be a basic, conceptual introduction to how some types of images are represented. You're not the first person to think about how images can be stored more efficiently.
GAME OVER "WE HAVE EVERY PIXEL COVERED"
but they're not actually storing it as text. a byte is one character that can represent from 0-255. imagine your x/y lookup except that instead of two or three digits you could use just one symbol for x and one for y, so long as you don't need to go above 255 in either direction. then imagine it's a cube, and that's essentially RGB. 3 'characters' per pixel
holy shit GREAT channel ! SUBBBBBBBBBBBBBBBBBBBBBBB£D
Anyone after multi dimensional data
You need another microphone.
Creepy face reflection in the window.
"gives us a snapshot" badump tchhhss
"JIFF"!!!!
fist.
8 minutes of waffling
First! Sorry I had to :(
FAIL!
nope
You're not sorry. If you were you would have also said something worth saying. What comment would you have made if you weren't first?
Rabid Rabbit Rabbi I probably wouldn't have said anything. I don't comment much.
I can understand my grandparent mispronouncing gif but an "Image analyst" ? Really ??