The History of the Camera

When we explain how a camera works, we usually put it like this: a lens gathers light, that light hits a light-sensitive surface, and it leaves a mark where it touches. One press records once, and that recording becomes a single photograph. For a long time, that explanation was true.

Today, things are a little different. The method widely used in smartphone cameras captures a rapid burst of multiple shots the moment you press the shutter. Then, it aligns and blends them together, outputting the result as a single photo. A research dataset released in 2016, in which each capture consisted of 2 to 10 raw frames, is a clear trace of this shift. Pressing the shutter once and getting a single photo are no longer a one-to-one match.

So far, this might sound like a modern story. But the real twist in the narrative starts much earlier. Color digital photography has relied on computation right from the beginning.

In the most common color imaging method, light-sensitive photosites are laid out in a grid, with a single color filter sitting over each one. A single photosite measures only one of the three colors. Where do the other two colors come from? Since they were never measured, they are calculated and filled in from neighboring pixels. To display a photo on a screen, every pixel needs all three color values, yet only one of them was ever directly measured.

We need to be careful about one thing here: not all sensors look like this. Some split beams of light across multiple sensors, while others measure all three colors at a single spot. So to be precise, we have to say: this is how the most common method works. This qualification will come up again later.

So a "composite photograph" is not a recent invention. At the same time, this setup has not been around forever either. A camera was originally a machine that recorded light once. How that machine turned into one that blends multiple glimpses into a single frame is what this story is about.

Let's start from the beginning. Though even deciding where to begin is no simple task.

If you credit the start of photography to just one person, someone gets erased. Once you define what kind of "first" you mean, the story splits into three.

First, there is the oldest surviving camera photograph. It was made by Nicéphore Niépce, and its holding institution, the Harry Ransom Center, dates it to 1827. Widely cited accounts place it between 1826 and 1827. Here is how it was made: he dissolved light-sensitive bitumen in lavender oil, coated a thin layer onto a pewter plate, and set the plate by a window inside a dark box. The exposure lasted several days. That is an exposure time where you don't even need a shutter.

Second, there is the first widely sold process. On August 19, 1839, the daguerreotype was unveiled at a meeting of the French Academy of Sciences in Paris. It produced a positive image directly onto a silver-plated copper plate without a negative. In other words, because that single plate was the final product, copying it meant you had to photograph the plate all over again. Even so, the process spread rapidly: by 1850, New York City alone had more than seventy photo studios.

Third, there is the method of shooting once to make multiple prints. The protagonist here is William Henry Fox Talbot. The concept was to capture a reversed image of lights and darks on light-sensitive paper, then press it against another sheet to produce multiple positive prints. Its holding institution records that the oldest surviving paper negative was made by him at Lacock Abbey in August 1835. The method of developing and bringing out a latent, invisible image formed by a short exposure was patented in 1841.

Line up these three pillars side by side, and you will notice something fascinating. The oldest photograph, the first widely sold process, and the ancestor of modern photography all belong to different people. And of the three, the one that led to the way we take pictures today was the third—the underdog in sheer numbers at the time. A daguerreotype was one shot for a single image, while a paper negative was one shot for many.

Black-and-white photography only needs to measure one thing: brightness. Color is different. You have to capture at least three different values separately. Yet, there is only one light-sensitive surface. This fundamental mismatch is the starting point for every method ever used to create color in photography.

In 1861, photographer Thomas Sutton, working with James Clerk Maxwell, demonstrated one solution. Setting up a tartan ribbon, he took three separate shots: once through a red filter placed before the lens, once through a green one, and once through blue. Then, they printed those three images onto glass plates and projected them onto a screen using three projectors fitted with the exact same filters, overlapping the beams. The single color image existed only in that overlap. What remained on the light-sensitive plates were still just three black-and-white images.

This method came with obvious drawbacks. The subject couldn't move while the three shots were being taken, and every time you wanted to view the picture, you had to align three projectors. Naturally, the next answer focused on condensing those three steps into one.

That solution was the Autochrome plate, which entered commercial production in 1907. The way it was made was ingenious. Grains of potato starch were sifted to select only the tiniest particles, then separated into three batches and dyed red, green, and violet. Blended together and spread densely across a glass plate, they numbered about 4 million per square inch, with the microscopic gaps between grains filled with charcoal powder. Over this mosaic, a light-sensitive emulsion was coated that responded to light regardless of its color.

What happens when light strikes this plate? Passing through an individual grain, only that grain's color filters through, and the emulsion underneath simply records how much light got through in black and white. Each point records only one of the three colors. The color only reappears when you look at it. When you shine light back through the developed plate, it passes through those same grains once more, and the scattered dots of color blend together in your eye.

Keep this principle in mind: lay color filters densely over a light-sensitive surface, record only brightness underneath without sensing color directly, and recombine the colors later. This very mechanism, refined on film, would later migrate onto silicon.

Now let's move on to silicon. On July 20, 1976, a patent filed by Bryce Bayer under Eastman Kodak was granted. The title was "Color Imaging Array," and it had been filed in March of the previous year.

The structure described in the patent is as follows: Elements broadly sensitive to light are laid out in a grid across a planar surface, and a color filter mosaic is placed on top. The filters are overlaid in a one-to-one match with the elements, and the three types of patterns do not overlap. To quote the original wording, it is a "nonoverlapping sampling array." Consequently, each single element can measure only one of the three colors.

The arrangement was not chosen at random, either. Green—the color whose brightness details the human eye sees best—occupies every other spot, spaced evenly whether you look horizontally or vertically. Red and blue fill in the remaining spaces. In another embodiment described in the patent, blue accounts for only 1/8 of the total elements, and the reason is noted right alongside: the human eye is least able to discern fine detail in blue. What humans are bad at seeing was built directly into the component's blueprint.

What about the other two colors, then? This is the critical part: that answer is not in this patent. What the patent covers stops at the circuitry that separates and outputs the signals by color. How to fill in the missing values is found in subsequent patents filed by the same company.

A patent granted on November 27, 1979, is one of them. The inventors listed are Peter Dillon and Bryce Bayer—meaning the creator of the earlier patent was involved here as well. The very first sentence of the abstract states that "interpolation is performed along scanning lines to establish intermediate values between signal updates for the individual primary colors." In other words, values for unmeasured positions are calculated and filled in from neighboring samples.

Another Kodak patent granted in 1987 summarized the situation of that era in its background section: "values of luminance and chrominance are provided for each sample location, usually by linearly interpolating between samples." It then called out the problem: that naive interpolation creates false color fringes that never existed in areas with fine patterns. It also means that how colors appear in a photo depends not on the capture, but on the performance of the computation.

Still, we shouldn't take this a step too far. Not every sensor works this way. The background section of the Bayer patent itself notes that video camera manufacturers at the time chose to split the light beam and use multiple scanners. There is also a design that takes advantage of how longer wavelengths penetrate deeper into silicon, allowing three vertically stacked layers to measure all three primary colors at the very same location. That was Richard Merrill's patent, granted in 1999. There are even monochrome-only sensors that measure no color at all. So the accurate statement is this: In the most widely used method, each pixel measures only one color.

Let's put aside the inner workings for a moment and talk about cost. The entire second half of this story comes from this very point.

Take the Kodak camera released in 1888. It cost $25 and came preloaded with film for 100 photos. Once you finished shooting, you didn't just remove the film—you sent the entire camera to the factory in Rochester. For $10, they would develop the photos, print them, load fresh film, and send it back. The advertising slogan was: "You press the button, we do the rest."

Look at the numbers again, and the design becomes clear. The $25 camera price was paid just once. But the $10 was a cost you paid every time you finished a roll. It was a structure where taking one more shot cost money.

Cost was not just about money, either. Time was also a cost. The earliest daguerreotypes required an exposure of 3 to 15 minutes. The Library of Congress notes that this made them practically unusable for portraits. A person had to sit completely still for that entire time, and even the slightest movement blurred that part of the image. It was an era when the blur caused by movement during an exposure—motion blur—was not an artistic choice, but simply a failure. Although, as light-sensitive processes and lenses improved, exposures soon dropped below 1 minute.

In a world where every shot costs money, human behavior is shaped by that cost. You decide what to shoot before you press the button. A roll of film has a fixed number of frames, and you shoot while counting down how many remain. Taking the same scene ten times and picking the best one meant paying ten times the price.

What finally eliminated this cost? Swapping film for electronic parts wasn't enough on its own. As we will see in the next chapter, even after the shift to electronics, for quite a while, one press still yielded just one shot.

In April 1970, Willard Boyle and George Smith of Bell Telephone Laboratories published a short report in an academic journal. The manuscript had been received on January 29 of that year. Their concept was to create potential wells on a semiconductor surface to hold electrical charges, and then sequentially shift the lowest points of those wells to push and transport the charges across the surface.

What is interesting is the list of uses mentioned in this report: shift registers, imaging devices, displays, and logic operations. Imaging was listed as just one of those four. While this device would later replace the photosensitive surface in photography, in the beginning, it was closer to a general-purpose tool for moving electrical charges.

A few years later, the device found its way into a camera. In 1974, Steven Sasson, a young electrical engineer at Eastman Kodak, was exploring what could be done with Fairchild Semiconductor's Model 201 charge-coupled device. His supervisor suggested using the 100×100 pixel device to digitize an image. The following year, a working prototype came to life. It weighed 3.6 kg, and recording a single photo onto audio cassette tape took 23 seconds.

A patent issued in 1978 detailed its architecture, with Gareth Lloyd and Steven Sasson listed as the inventors. The basic framework was that the charge-coupled device formed a charge pattern corresponding to an image during exposure, a high-speed converter turned it into digital words to be held temporarily in memory, and then it was read out at a speed suitable for recording onto audio-grade magnetic tape.

Yet this machine was still one shot at a time. In fact, it was at a disadvantage compared to film, since recording a single frame took 23 seconds. At the time, Sasson himself calculated when its resolution might rival film. He estimated it would be between 1990 and 1995, and the company shelved the project. The calculation was that they could not commit production lines today to something more than a decade away.

A long time passed after that. Eastman Kodak filed documents with securities regulators seeking Chapter 11 protection under the U.S. Bankruptcy Code on January 19, 2012. In our story, that chapter ends right here. What this series follows is not the fate of a corporation, but what the machine actually does. And the moment when the machine's role truly changed has not arrived yet.

Let's examine the most persistent misconception in the world of cameras: the idea that more pixels always mean a better photo. Why this doesn't hold true can be clearly explained through the underlying mechanics.

First, let's start with what pixel count actually determines. The finest pattern a sensor can resolve is governed by the distance between pixel centers. That limit is half the reciprocal of this spacing; any pattern finer than that cannot be reproduced faithfully and shows up as distorted artifacts. Increasing the number of pixels raises this limit. All pixel count determines is how finely you sample the scene.

Yet something is lost in return. If you keep the sensor area fixed and increase the pixel count, each individual pixel must shrink. A paper published in 2000 by researchers at Stanford notes that dynamic range and the signal-to-noise ratio scale roughly in proportion to the square root of pixel size. This happens because light arrives as discrete packets. Even under steady brightness, the number of arriving photons fluctuates slightly each time, and the less light a pixel collects, the larger those fluctuations become relative to the signal. When pixels shrink, noise increases in dark areas, and the range for capturing both highlights and shadows in a single shot narrows.

There is also a hard ceiling. The same paper notes that shrinking pixels can improve a system's reproduction capability, but adds an important caveat: "as long as resolution is not limited by the imaging optics." The reproduction capability of the entire system is effectively the product of factors like the lens's contribution and pixel geometry. Details that the lens has already blurred cannot be recovered no matter how finely you divide the pixels. It is just like multiplication: if one term is close to zero, it doesn't matter how large you make the others.

Because these two effects pull in opposite directions, an optimal pixel size emerges for any given manufacturing process, lens, and intended use. Under the conditions of a 0.35µm process and a 30% fill factor, that paper calculated this value to be about 6.5µm. Because this figure depends on specific assumptions, it should not be cited out of context. Yet the very fact that "an optimal value exists" completely dismantles the belief that "more is always better."

Pixel count, then, is not a report card for image quality; it is simply one axis. Pixel count determines how finely the scene is sampled, pixel size determines how clean that sample is, and the lens sets the ceiling on how much information can be delivered in the first place. All three are doing entirely different jobs.

Now, two separate threads meet. One is the architecture we saw in Chapter 3: the idea that colors are measured separately and then combined. The other is the cost we saw in Chapter 5: the cost attached to taking one more shot.

In digital, that cost practically disappeared. With no film and no chemical developer, taking one more shot incurs no extra cost. Buying a camera and setting up storage costs money upfront once, but after that, taking a few shots or many makes little difference. When only fixed costs remain and the per-shot cost virtually vanishes, behavior changes. We move from deciding before pressing the button to pressing first and choosing later.

This opens up a brand-new option. If the cost of taking another shot is nearly 0 anyway, why not use multiple shots for a single photograph?

In 1997, Paul Debevec and Jitendra Malik published a method along these lines. You take multiple shots of the same scene at different exposures, trace backward how the camera converts light into numerical values to recover a radiance map, and then merge them into a single image. Capturing bright and dark areas together shifted from a matter of shooting to a matter of merging. This is where the term HDR, found in your phone's settings today, comes from.

Mobile photography research published in 2016 brought this right inside actual devices. When you press the shutter, it takes multiple shots in rapid succession, aligns and merges them to reduce noise, and expands dynamic range. In the dataset released alongside the paper, each set contained between 2 and 10 raw photographs.

And in 2019, that same lineage of research took another step forward. The paper's abstract begins like this: "We replace the traditional demosaicing used in single-frame and burst photography pipelines with a multi-frame super-resolution algorithm." The method is strangely elegant. When you hold a camera by hand, it usually trembles slightly, but instead of treating that tremor as noise to eliminate, they use it as raw material. Thanks to that handshake, multiple frames are captured offset by less than a single pixel, and when aligned and combined, the red, green, and blue values at each pixel location are filled with actually measured values.

Think back to Chapter 4. The 1976 array allowed only one color to be measured per pixel, guessing the rest from its neighbors. The 2019 method, instead of guessing, brings in values actually measured in other frames. It is a different answer to the same problem, but this latter answer is possible only because the cost of taking one more shot is virtually 0. On a mobile phone, this approach processed a single 12 million-pixel frame in 100 milliseconds, and was also used for zoom features. Between optical zoom that changes the focal length with lenses and digital zoom that crops and enlarges the image, a new path opened up: recovering detail by stacking multiple frames.

The final scene of this story centers on numbers you often see in the news. But here, you must never mix these two numbers into a single sentence.

The first number is shipment volume. An industry association called the Camera & Imaging Products Association (CIPA) tallies shipments collected from its participating member companies. In these statistics, shipments of digital still cameras peaked in 2010 at 121,463,000 units. Built-in lens cameras accounted for 108,576,000 units, and interchangeable-lens cameras accounted for 12,887,000 units. And according to a press release issued by this association in February 2026, shipments for the full year of 2025 stood at 9,438,876 units. Interchangeable-lens cameras made up 7,001,965 units, while built-in lens cameras made up 2,436,911 units.

What these statistics count are dedicated devices. The cameras attached to mobile phones are not included here. So what this number tells us isn't that "cameras have disappeared," but that "cameras bought separately aren't selling."

The second number is the volume of photos. On May 28, 2025, Google marked the tenth anniversary of its photo service by announcing that its monthly active users had surpassed 1.5 billion and more than 9 trillion photos and videos were stored there. This is a figure from just one company, for a single service. It is not a count of every photo taken across the world, and there are surely far more photos that never make it there. Even so, it has reached this scale.

When placing these two numbers side by side, you must always accompany them with this context: the former is the shipment volume of dedicated devices gathered by an industry association from its participating companies, while the latter is the count of photos and videos a single company stated are stored on its service. The entities doing the counting, the objects being counted, and the units of measurement are all different, so you must not subtract or divide them. Yet their trajectories are unmistakably pointing in opposite directions.

Only one object bridges the gap between these two numbers: the camera moved inside the phone. Released by Sharp in 2000, the J-SH04 was equipped with an 110,000-pixel image sensor, allowing users to send captured photos directly via email. The company describes this as the industry's first mobile phone with a built-in camera. Because this is the company's own claim, one must be cautious about recording it as a "world first," but the direction was already set from that point on. The camera shifted from an object bought separately to a component that comes attached to what you carry in your pocket.

And inside that component, what happened in Chapter 8 takes place: pressing the shutter once captures multiple frames and combines them into a single photograph.

There is no need to try to guess the future. Instead, based on what this story has shown us, let's just lay out where you can look to be the first to notice when the board begins to shift.

First is the benchmark for the numbers. The numbers that appear in camera articles are usually of two kinds: how many units were shipped, and how many photos have accumulated. The two differ in who is counting and what is being counted, and they move in opposite directions. A ranking article that leaves out which organization counted what and when is, on that count alone, about half wrong. On top of that, these statistics are updated every year. If you open the same URL the following year, the values have already changed.

Second is the word "original." Take raw files, which are said to preserve an untouched state. In a raw file coming from a sensor that uses a color filter array, there is only a single color value at each pixel site. That is what a 2019 paper meant when it described generating complete RGB photos directly from color filter array raw photos. While raw is an untouched state, it is not what human eyes actually see. The colors you see are filled in through computation afterward, and displaying them on screen or saving them usually adds another round of lossy compression.

Third is the ceiling of the megapixel race. The pixel count listed on a product sheet merely tells you the density of the sampling. What you should watch for is whether the size of each pixel and the amount of information delivered by the lens are listed alongside it. An announcement where only one of these three increases is simply an announcement that decided not to talk about the other two.

Fourth is the debate raging right now. With images not taken by humans sitting side by side with photographs, discussions are springing up everywhere asking "what counts as a record?" This chapter does not offer an answer to that question. It does, however, shift the starting line of the inquiry just a bit. If you ask when computation stepped in, the answer is not recently, but all the way back when we first began generating color. So what we need to watch is not "was computation involved?" but what kind of computation went in, how far it went, and where it is recorded.

Lastly, just one more thing. The side that was pushed aside in this story had its own calculations at the time. The person who built the filmless camera figured it would take more than ten years for its resolution to rival film, and the project was shelved based on that judgment. Saying that judgment was wrong is easiest now, once we already know the outcome. We, too, do not really know which value will soon approach 0 today—even though in this story, that was usually where the tide turned.