How to get better "true" colour images on Mars

[Phobos]

There are other threads that have mentioned the colour photography issues, but whilse the details are available if you hunt, they do take a fair
I though I should provide a little insight into what is behind the aparent discolouration aspect of the images being returned from Mars by the spirit and opportunity rovers.

Ok, first a description of what the problem is. The general public has been thrilled with the apparent quality of the images returned from Mars, but there has been some puzzlement over the overall red hue in the publically released images.

The following images illustrate this problem;

The Blue Nasa logo as released in spirit images;


Sundial as in Nasa released spirit images;


Sundial as photographed on Earth;



Image seen at JPL press releease with blue sky;



How we see colour

The human eye contains cones that are sensitive to specific frequencies of light that correspond to Red, Green and Blue.


How PanCam sees colour

Pancam is equipped with a monochrome camera. In order to create colour images various filters are placed over the lens. After the images are combined a colour picture is created.


Comparison between the human eye and the pancam

The following table compares light frequency sensitivity of the human eye cones with the filters available to the PanCam on Spirit and Opportunity rovers;

Red: Human eye 570nm PanCam L2 (750nm), L4 (600nm)
Green: Human eye 535nm PanCam L5 (530nm)
Blue: Human eye 445nm PanCam L6 (480nm)

Nasa are producing colour images using pancam filters L2, L5 and L6 resulting in images that are converting blue objects into muddy red objects.

If we use L4, L5 and L6 filters we end up with images that are closer to what would be observed by the human eye if a human observer were located where the pancams are.

Fortunaely, whilst most colour images returned by the spirit rover are using filters colourset that includes the L2 (near IR) filter instead of L4 (Red), I have managed to find one image that uses the L4, L5 and L6 filters.

To my knowledge this is the only image sent back from spirit that truely represents what the human eye would see;


Compare this to how the object looks back on Earth;


As you can see, using this combination of filters produces an image that better represents what would be seen by a human observer on Mars. Taking sets that include the L2 filtered image may well be more scientifically usefull, but it is annoying that this is the only example of it's type that is available to the public that paid for this mission.

Phobos

[TrAI]

Yes, it is correct that most color press pictures are based on the L2 as the red channel, but the one you have found is not the only set that includes an L4 too. There are several under the raw images section on the rover homepage, but you will have to combine them yourself to form color pictures...

As for why the majority of the sets returned is L2 based, I don't know for sure, but I would guess near IR is more useful for scientific studies, and you get to use the R2 on the right Pan Cam to create stereo images(2 is the only filter that are the same on both cameras). Also you save valuable bandwidth on the downlink by not transferring the L4, I guess...

By the way, I seem to remember that the 570nm is yellow sensitive, not red, in fact the chemistry of the cones are not very sensitive to red at all. I seem to remember the peak of this detection is in the blue part of the spectrum, but since only about 2% of the cones have what you can call the "blue filter", the peek for the eye is more towards the green part(I think there were a little more than 30% cones for green and the rest yellow).. The red frequencies is something the brain must compute by comparing how much less the green cones are stimulated than the yellow(the sensitivity does overlap, so the green has some sensitivity even bellow the peek of the yellow..).. But I guess I got far away from the point of this thread...

Anyway, I don't think all blue gets such results, but dye is often very different under IR, and I think the calibration target uses a blue dye that is IR reflective... Whether the sky is discolored or not I guess depends on how much IR light you get from it... Also the uncalibrated images often look too much blue, like the ones in the public Maestro Spirit sets.. Then one have the atmospheric conditions on mars, as I understand it the sky is supposed to be bluish when there are little dust in the air, but I don't know how often that happen(Heh, If the sky here on earth so really is blue, why would it be different on Mars... ;-p)

The argument about the ones who paid getting their what they paid for is a little flawed if they use IR since it is better for science, as the whole point of such a mission is to get the most scientific useful data down, not to waste BW to make pretty pictures for the press, that coupled with that its probably just the best sets that is used as press images will limit the sets with L4s I guess...

Hmm, At least that is how I see it.. But I might be wrong in my guesses about the reasons for the use of the L2 instead of the L4...

[Phobos]

Yes, it is correct that most color press pictures are based on the L2 as the red channel, but the one you have found is not the only set that includes an L4 too. There are several under the raw images section on the rover homepage, but you will have to combine them yourself to form color pictures...

I have searched through all raw images from both Sprit and Opportunity, and this is the only colour set that combines L4(red) with L5(green) and L6(blue) filters. Moreover, those images that do use L4 are primarily images that show objects on the lander. In the few examples where L4 has been used on Mars based objects neither L5(green), nor L6(blue) are used. I would love to combine coloursets of the Marsian surface and sky that include L4, L5 and L6, but unfortunately so far none have been taken.

As for why the majority of the sets returned is L2 based, I don't know for sure, but I would guess near IR is more useful for scientific studies, and you get to use the R2 on the right Pan Cam to create stereo images(2 is the only filter that are the same on both cameras). Also you save valuable bandwidth on the downlink by not transferring the L4, I guess...

By not taking any L4,L5,L6 colourset images of the Marsian surface science is missing out on comparative data. Whilst there may be an argument for maintaining a majority of images with the near IR filter included, abstaining from capturing images that approximate what the human eye would seem highly questionable.

The stereo arguement falls apart if you simply select periodic images to be taken using L2, L5, L6 AND L4. The addition of the one extra filtered image in such a colourset only represents a 25% increase in bandwidth requirement for these selected images. In return not only are more visually representative results obtainable, but science is afforded the opertunity to conduct more comparative studies

By the way, I seem to remember that the 570nm is yellow sensitive, not red, in fact the chemistry of the cones are not very sensitive to red at all. I seem to remember the peak of this detection is in the blue part of the spectrum, but since only about 2% of the cones have what you can call the "blue filter", the peek for the eye is more towards the green part(I think there were a little more than 30% cones for green and the rest yellow).. The red frequencies is something the brain must compute by comparing how much less the green cones are stimulated than the yellow(the sensitivity does overlap, so the green has some sensitivity even bellow the peek of the yellow..).. But I guess I got far away from the point of this thread...

In reply to a letter enquiring about the issue of the image discolourations, Dr Jim Bell (associate professor leading the pancam payload element for the mission) said;

The answer is that the color chips on the sundial have different colors in the near-infrared range of Pancam filters. For example, the blue chip is dark near 600 nm, where humans see red light, but is especially bright at 750 nm, which is used as "red" for many Pancam images. So it appears pink in RGB composites. We chose the pigments for the chips on purpose this way, so they could provide different patterns of brightnesses regardless of which filters we used. The details of the colors of the pigments are published in a paper I wrote in the December issue of the Journal of Geophysical Research (Planets), in case you want more details...

from http://science.howstuffworks.com/eye3.htm

In the diagram above, the wavelengths of the three types of cones (red, green and blue) are shown. The peak absorbancy of blue-sensitive pigment is 445 nanometers, for green-sensitive pigment it is 535 nanometers, and for red-sensitive pigment it is 570 nanometers.

Combining both sets of details it should be clear that the L4 (at 600nm) is by far the better approximation of true colour than the L2, 750nm filter.

Anyway, I don't think all blue gets such results, but dye is often very different under IR, and I think the calibration target uses a blue dye that is IR reflective... Whether the sky is discolored or not I guess depends on how much IR light you get from it... Also the uncalibrated images often look too much blue, like the ones in the public Maestro Spirit sets.. Then one have the atmospheric conditions on mars, as I understand it the sky is supposed to be bluish when there are little dust in the air, but I don't know how often that happen(Heh, If the sky here on earth so really is blue, why would it be different on Mars... ;-p)

The example photographs I provided for both calibration targets (DVD and Sundial) were calibrated using L4, L5 and L6 filters. Moreover comparison between the DVD image produced using the L4,L5,L6 colourset and the Sundial using L2,L5,L6 makes it perfectly clear which of the two methods is better suited for approximating true colours.

As you point out there is some debate as to the true appearance of the Marsian sky. How things really look is one of the great mysteries that motivated people to send these rovers to Mars in the first instance. If the correct filters were used we would be a lot closer knowing the answer to that question. We may also use that information when evaluating other details obtained from the images.

Various factors will affect sky colour, but in comparing with Earth you should take account that the atmosphere is only 1% the density of Earths with very little cloud cover, and with a different mix of gasses.

The argument about the ones who paid getting what they paid for is a little flawed if they use IR since it is better for science, as the whole point of such a mission is to get the most scientific useful data down, not to waste BW to make pretty pictures for the press, that coupled with that its probably just the best sets that is used as press images will limit the sets with L4s I guess...

Hmm, At least that is how I see it.. But I might be wrong in my guesses about the reasons for the use of the L2 instead of the L4...

Without public support funds will not be made available for further missions - complete disregard for public opinion would be a foolhardy attitude to take. I have already shown that the amount of data required to satisfy the public thurst for a true approxiamation of the colours of Mars is fairly small (I suspect 10 percent of bandwidth or less would be sufficient).

As well as satisfying the desires of the public we should remember that science will be gaining new comparitive data. Moreover by better approximating the true visual appearance of the sky and surface science is more likely to inspire the next generation of scientists.

Finally, given that the images were released to the public as "approximate true color images", it would seem more honest to supply them with images based on the red, green, blue filter coloursets which we know to be a more accurate representation. The general public will assume that the images they receive are the best approximations that Nasa can provide given the limitations of the equipment. The fact is this is just not true - if the coloursets were L4,L5,L6 based they would be far better approximations. Failure to deliver approximations to the highest standard possible can be seen as at best negligence, and at worst dishonest by the general public.

Nasa needs to look after it's image. It is very unwise to disregard public desires in this way - the public did indeed pay for this mission, and whilst they will accept the necessity to ensure the scientific goals are met, they will also expect that some efforts are made to meet their own aspirations.

Phobos

[parejkoj]

I would hazzard a guess that even if they released L456 combined images, someone would complain that the filters do not approximate the human eye enough. Face it, taking a picture with a CCD/filters will NOT be the same as what a person sees (completely ignoring the fact what each person perceives is different). There are just too many variables, and that is NOT the goal of the mission. I hate it when people who should know better say this -- I've heard it about Hubble, Galileo, my own CCD pictures and comparing the view through a telescope with published CCD or plate images of the same object. From what you've posted here Phobos, you should know better. What they've released is closer than we've ever gotten before and I would say is a quite good representation of what your eye would see (excepting that these images are at undoubtedly at much higher contrast, since it is a CCD after all).

Looking at the Pancam technical briefing, the only filter shared between the two CCDs is the 750nm (20nm bandpass) filter. Stereo images take precedence right now, since they are trying to map out the surroundings to get distance and size measurements to see where to go next. And I am certain that there is no spare bandwidth right now to send anything more than they are. Bandwidth is the limiting factor right now, and it is a very hard limit, believe me. An extra 25% doesn't exist; heck, an extra 5% probably doesn't exist.

But I would bet that images in other filters will be released in the coming weeks when they have more bandwidth to play with, and the full mission gets underway. But the goal of the first few weeks of any mission is NOT public relations but rover safety and initial science assessment. That is exactly what they are doing. And I would not expect any differently.

[JonClarke]

I can't see what the issue of the "true" colour of Mars is so important. 2-5-6 and 4-5-6 combination produce a landscape indistinguishable to the human eye. Using L2 (near IR) rather than L4 (red) for the red channel increases clarity slightly, which is good.

The trouble is, most of those who get obsessed about the colour of Mars are skirting woo-woo territory. There is a spectrum here, but fairly soon most of those who complain about the true colour move on to saying NASA is hiding the true colour, and that if only the true colour were released we would see blue sky, green veegtation and lakes of open water.

Jon

[TrAI]

Hmmm... This at least seems to have an L4:

http://marsrovers.jpl.nasa.gov/galle...1P2262L4M1.JPG
http://marsrovers.jpl.nasa.gov/galle...1P2262L5M1.JPG
http://marsrovers.jpl.nasa.gov/galle...1P2262L6M1.JPG

And this is the L2
http://marsrovers.jpl.nasa.gov/galle...1P2261L2M1.JPG

As far as I can see the combination of these by using L4 and L2 are not to different, the L2 version has more cyan in it, but I dont know how to calibrate for the cameras sensitivity. The RGB channels of the resultant picture seems to have a similar intensity profile, probably the imaging system optimizes it(so that the picture fills the entire grayscale), so that it isn't too dark, but since this is done on a grayscale basis, its hard to accurately compensate...

Anyway, I don't think the point of the mission is to answer what color the ground and sky would be to someone standing there(color is subjective, the eyes would adapt to the light and so would see something different than what the camera sees), the filters are there to help the scientists do an initial identification of the geology of the site, before employing the MiniTES and spectrometers, and for this I think the IR filter might be better suited. And it is going to be mapping the site in 3D...

10% of bandwidth is quite a lot, I think the com.windows is quite full as it is.. And it seems if one were to take 1 extra picture, this would stand for 20% of the bandwidth used by the Pan Cam in a color set(L2,R2,L5,L6 is an often used combination, it seems)..

Now I don't know what the public thinks about this over in the USA, But here, people doesn't go demanding the money back be course the water in the travel catalogs looks bluer than real life, or the photos in the album isn't exactly the same as RL... People are used to pictures not being true color. Why one should think the images from the rovers are any different, I don't know... Its not like other pictures of the universe is done in true color. I might be wrong, but I think its how the light of each frequency is reflected that is important, and perhaps to compare pictures of two frequencies, and i would think mapping false color would be more suited for that than watching a true color image...

Anyway, color might be eye candy, but for a truly dramatic picture you can't beat grayscale... :P

[George]

Nasa needs to look after it's image. It is very unwise to disregard public desires in this way - the public did indeed pay for this mission, and whilst they will accept the necessity to ensure the scientific goals are met, they will also expect that some efforts are made to meet their own aspirations.

Phobos, I agree. The scientific benefits for true color imaging may be minimal, however, the public wants to “see” the real deal.

The contrast between public interest last Saturday night and this coming Saturday night should be noted. I love playing football, but images from another world are far more wonderful to me. You can bet, this Saturday, we better see a green football field and the right color jerseys or there will be a huge outcry.

I want to believe that there may be some way to generate the necessary support (funds) to stir the public into experiencing the magnitude of our landing on another world. How this can be accomplished without beer commercials, for instance, may be the bigger issue. Another avenue might be a “JPL, Jr.” imaging group of qualified volunteers that could do the job?

What was the line from “The Right Stuff” – “No bucks, no Buck Rogers”! The astronauts got their windows in the capsule. The scientists/engineers were burdened but for a good dramatic cause.

Is it fair to expect true color images this soon? Yep, even if I have to volunteer you. [Ya'll seem qualified]

[Phobos]

After a lot of hunting I think I have found one image that has been produced as a combination of L4, L5 and L6 filters.



Made based on perceptual data of the eye's sensitivity to light in the filter spectra. From images;

2P126991536EFF0205P2530L2M1
2P126991733EFF0205P2530L3M1

2P126991781EFF0205P2530L4M1
2P126991583EFF0205P2530L5M1
2P126991633EFF0205P2530L6M1


This would appear to be the closest image yet to what would be seen if you were stood on Mars looking at the surface.

As you can see this particular image is the marks left from the airbags on the soil after Spirit's landing.

Phobos

[majic]

For everyone still questioning color rendition of NASA-mars photos please read :

http://www.badastronomy.com/phpBB/vi...=199009#199009

[JonClarke]

Phobos

Where did you find that last image? It looks as if I would just reach out and touch it...

Jon

[Phobos]

Phobos

Where did you find that last image? It looks as if I would just reach out and touch it...

Jon

Yes, that is what I was meaning abou the impact about using the right filters.

The image came from here; http://www.lyle.org/mars/
Its a pretty interesting site with some neat automation going on. When the first RAW images arived from opportunity's landing I watched on Nasa TV. As the monochromes were being shared with the world I was already looking at the colour composites

Phobos

[rwald]

Hmm. I wonder what they did to counter for the color normalization that the rovers do automatically? These sure look better than what I got with plain-vanilla color mixing.