For those who have ever wondered how they decide what Hubble looks at - here's a report from this year's proposal review meeting, recently concluded at a scenic hotel next to Baltimore-Washington Airport. Looking back at my curriculum vitae, I find that this is my fifth time to be involved, going all the way back to the bad old days of aberrated images in 1991. I like the new images better.
Although this is a very personal judgment, I found this year's crop of proposals (more particularly, the set we looked at dealing with galaxies near and far) to be at least as exciting as the last couple of round I worked on. Although the capabilities of the observatory are somewhat circumscribed by the loss of STIS and the move to a longer-lasting pointing technique, Hubble can now reap the harvest of objects discovered by the Chandra and Spitzer observatories, the GALEX ultraviolet sky survey, and increasingly sophisticated winnowing of the results of the Sloan Digital Sky Survey. And, of course, we can now follow up things known only from previous round of observations using Hubble itself.
In the two months or so leading up to the review, the proposals were distributed to reviewers and the Institute prepared for the onslaught. This year, each of us got a CDROM instead of huge stacks of paper (although some parts of the review still seem to work better if there is paper in front of you). The reviewers are asked to remain anonymous until after the review results are announced. This prevents subtle forms of jury tampering such as gifting them with envelopes stuffed full of preprints of someone's latest unpublished results, or hearing subtle reminders of a colleague's brilliance over coffee at a meeting.
Avoiding conflicts of interest in proposal review is a Big Deal for several reasons. First, it's just basic honesty, and good science, to secure the most unbiased review possible. On top of that, there are government regulations with legal teeth that apply, since US-based investigators receive grant funding to cover expenses of analyzing the data (which may pay grad students, postdocs, summer faculty salary, meeting travel...). And since it is not unknown for disappointed proposers to complain to members of Congress that they have been unfairly treated, many problems can be avoided by worrying beforehand. The Hubble operation has been under a public microscope for a long time and, in addition to making reviewers with personal or institutional conflicts of interest recuse themselves, tries hard to minimize the number of times that this happens. Much of this review structure serves well enough that it has been taken over for Chandra, and in relevant part by other facilities as well.
Panel members, chairs, and additional at-large members of the final Telescope Allocation Committee (TAC) are selected. As they read proposals, there are some cases in which technical issues are called to the attention of instrument scientists. Can this observation really be done in 45 minutes for something so faint in that filter? Is it a problem to get a deep ACS galaxy image looking so near a bright star? How fast can the camera cycle between pictures at adjacent places in different filters? Panel members don't have the authority to decide whether something is feasible or not; this decision gets handed off to people who work with operating the instruments constantly.
The European Space Agency has been an important partner throughout the whole Hubble project (and JWST as well). Under agreement with NASA, ESA scientists are supposed to get at least 15% of the observing time on Hubble. The system tracks how many investigators on each proposal work at institutions in ESA member states; so far, getting the 15% has never been a problem, and has never required juggling the peer-review results. This adds a prominent European presence to the review as well; three of nine members on our panel flew the Atlantic for this.
The first stage of this review includes eleven panels - one for solar-system proposals, and five each for galactic and extragalactic topics. Topics with many proposals are twinned, with the topic split between two panels so that conflicts of interest (for which panelists have to leave the room, or for more indirect cases, refrain from voting) are minimized.
This year there were 733 proposals, requesting almost 15,000 orbits of telescope use (the "orbit" being the normal unit of telescope allocation - Hubble can look at most areas of the sky for about 50 minutes during each 94-minute orbit). That means that only 1 in 5 can be scheduled; we're guaranteed to go home disappointed that some interesting projects simply won't fit. These numbers, as daunting as they are to proposers, are down a bit from the peak values each time new instruments are installed in the telescope - in some years, there have been as many as 1298 proposals. The main reason the number was a bit low the last two years was the failure of the STIS electronics, which left Hubble without a dedicated spectrograph and all the scientific flexibility that went with it. Any spectral observations it does now have to use one of the cameras with a diffraction grating or thin prism inserted, smearing every object into a spectrum and letting them overlap and interfere where they may. The available instruments now are the venerable Wide Field Planetary Camera 2 (WFPC2), installed in the first refurbishment at the end of 1993, the Near Infrared Camera and Multiobject Spectrometer (NICMOS), and Advanced Camera for Surveys (ACS). The Fine-Guidance Sensors can also be used for science measurements, but our panels didn't see any such programs.
Each proposal is assigned a primary and a secondary reviewer, who are charged with presenting it to the rest of the panel (who should all have read it as well). They should summarize the science goals and specific approach, highlight special features or difficulties, and generally lead its discussion. The reviewers send in preliminary ratings about a week in advance - first this makes sure they actually read them ahead of time, and next it allows the discussion to be more focussed. After (as best I can tell) a certain amount of trepidation, the Institute found that the discussion could be made more productive by introducing a form of triage (something which has also been adopted by other facilities as well). The idea is that, when the oversubscription is this high, there is no point in spending a lot of time detailing the shortcomings of a proposal which ranks in the bottom 25% of the preliminary grades, and which no panel member wishes to bring up for discussion (we did see a few proposals pulled out of the initial triage pile for that reason). This made sense once several years' worth of data were in hand to show that proposals starting this low essentially never made the cut to be scheduled. Reviewers could easily spend time discussing the flaws of a poorly-rated proposal in great detail, and contributing nothing to the outcome of proposals actually scheduled.
Following discussion of the proposals, new numerical grades are worked out by averaging individual grades via secret ballot (the point of that being to avoid anyone being influenced by how they see someone else vote). Our panel reviewed about 70 proposals, which meant that each one brought up could be discussed for no more than 15 minutes (which is one more reason that it was important to read them ahead of time!). Very few proposals were not scientifically worth doing - there used to be a few clown proposals that had no reason to be submitted, showing basic misunderstanding of the instrument or a scientific point, but they have been worked out of the system. We usually have to make an effort to downgrade proposals for even fairly minor shortcomings, simply to be able to make some distinctions at all. We grouped proposals for similar science to be talked about in sequence, since there are often several proposals that are so similar that they should be compared in some detail.
The proposal reviews used to be at the Space Telescope Science Institute in Baltimore, bringing everything but operations to a screeching halt for a week. With broadband connections routinely supported, the review can now be more conveniently hosted at an airport hotel, with instrument scientists only a call or email away for details and documents (and we can also quickly look at archived ourselves if needed). There are downsides to everyone being connected; last time I was here, the committee chairwoman had to call down panelists for watching CNN on their laptops (because that was the first day of ground fighting in Iraq war). With everyone taking the hotel shuttle bus, we could play a round of "spot the astronomer" - my wife has always claimed that you could spot astronomers in an airport, even if they were well dressed. Worked pretty well this time, but then I already knew most of the folks I spotted.
[This is set up like a blog, but is showing up several weeks after the event for several reasons. First, I had to verify with the STScI folks that it was approrpiately purged of individually identifiable information. Then it took me a while to fill out some of the details, and finally I had some discussion with the BA on how to post this. Each day of deoiberations gets it own post.]