1.1.

Electron Multiplying Charge Coupled Device Readout Noise

EMCCDs work on the principal of impact ionization, whereas the signals in a given pixel (electrons) are transferred along a many-stage multiplication register; there is a small probability (typically of order 1%, $p=0.01$) that each photoelectron will generate an additional electron. These registers are many hundreds of elements long, and so a large mean multiplication (or gain) can be achieved, equal to ${(1+p)}^n$ where $n$ is the number of stages. Unfortunately, this multiplication process is stochastic, and for a given number of input photons in a given pixel, there is a wide range in the possible measured EMCCD output value⁵ in addition to photon shot noise which is always present. Typically, a gain of order 500 to 1000 is used for astronomical AO systems. After the signal has been multiplied in this way, it is then read out of the detector and digitized, introducing readout noise to the signal. This readout noise is dependent on readout speed and typically has a root mean square (RMS) of about 50 electrons for an EMCCD operated at high-frame rates. We ignore thermal noise, since EMCCDs typically operate at high frame rates and are usually cooled to temperatures of around 220 K in commercial camera models.

When modeling the impact of detector performance on instrument designs, the combination of these sources of uncertainty leads to increased complexity. Therefore, simplified models are often used (for example, in Ref. 6): typically, when modeling an EMCCD, the detector quantum efficiency (QE) is halved (i.e., the input flux is halved) as an approximation of the effect of the stochastic gain mechanism, and a readout of around 0.1 electrons is assumed (the true readout noise divided by the gain). Here, we investigate the effect of these assumptions.

1.2.

Scientific CMOS Readout Noise

An sCMOS detector is an active pixel sensor, with each pixel having its own individual readout, rather than a single, or small number of, readout ports as in the case of a CCD. Each sCMOS pixel will, therefore, have an associated readout noise level, which will differ from other readouts due to manufacturing imperfections, etc. Additionally, the readout noise introduced at each pixel will also vary with each frame readout, i.e., the readout noise of a given pixel has some RMS values, with all pixels forming an RMS readout noise probability distribution. Therefore, manufacturers of sCMOS cameras usually quote the median RMS readout of the device, which is at the level of 0.8 electrons for the best current cameras. As with an EMCCD, this level will be dependent on readout speed, which is generally not user-selectable for current commercial sCMOS cameras. Figure 1 shows the histogram of variation of an individual pixel RMS readout for a typical sCMOS device.⁷

Fig. 1

The probability distribution for root mean square (RMS) readout noise of individual scientific CMOS (sCMOS) detector pixels, scaled to the frequency of the modal average. Inset shows the distribution on a logarithmic scale.

Instrument modeling of sCMOS detectors has to date typically used a single RMS readout value for all pixels (see, for example, Ref. 8), and readout noise is often described using a single (unspecified) parameter, for example, Ref. 9. However, this can lead to an overestimation of instrument performance, since the occasional pixels with far greater readout noise are not modeled.

1.3.

Accurate Readout Noise Modeling for Shack–Hartmann Wavefront Sensors

This paper seeks to investigate the effect of accurate readout noise models on the performance of Shack–Hartmann WFSs commonly used for astronomical AO systems. In Sec. 2, we describe the models used, our performance verification, and the implemented tests. In Sec. 3, we discuss our findings and summarize the results. We conclude in Sec. 4.

2.1.

EMCCD Models

We introduce three models for EMCCD technology readout noise:

For the stochastic and probability distribution models, we use a mean gain of 500 (unless otherwise stated), with 520 multiplication stages, and, thus a probability of about 1.2% of a new electron being generated at each stage for each input electron. We also do not investigate other readout noises, which could be introduced at different detector readout speeds. To achieve the same performance at other readout noise levels, the EMCCD gain could be altered.

The probability distribution for EMCCD output is given by Eq. (2), taken from Ref. 5. Additionally, here we also introduce an approximation for this distribution at higher light levels (e.g., for greater than 50 input photoelectrons):

And our high light level approximation

2.2.

Scientific CMOS Models

We also introduce a number of models for sCMOS readout noise:

It should be noted that in the cases where the RMS readout noise follows the probability distribution [Eq. (3)], some pixels will have a much larger RMS readout noise than others and thus will have a negative effect on centroid estimates. We use a probability distribution that closely matches manufacturer data,⁷ given by

The random nature of pixel RMS readout noise obtained from this distribution means that some subapertures will behave very well (with low noise throughout), while others will contain one or more noisy pixels, particularly for larger subapertures. To get some feel for this effect, we randomly generate noise patterns for 10 different subapertures, which are then used throughout the simulations (and for interest are shown in Fig. 2 for the $16\times 16\text{pixel}$ case). Additionally, to get a better estimate for the “mean” performance of the sCMOS detector, we also include results where a new RMS readout noise pattern was obtained for every Monte-Carlo iteration (using the probability distribution, Fig. 1), the “sCMOS random” model. It is important to note that these readout noise patterns are not a static offset added to the image. Rather, they represent the RMS readout noise of the individual pixels; for each Monte-Carlo iteration, this RMS value is used to generate the particular number of noise electrons introduced, randomly distributed in a Gaussian distribution with a standard deviation equal to the RMS.

Fig. 2

The per-pixel RMS readout noise for the 10 sample noise patterns used here for $16\times 16\text{pixel}$ subapertures. Each value represents the RMS readout noise of that particular pixel, which is then randomly sampled from a Gaussian distribution for every Monte-Carlo iteration.

3.1.

EMCCD Gain

Throughout our modeling, we have used a mean EMCCD gain of 500, which is close to the value that we frequently use on-sky with CANARY. However, Fig. 5 also shows the slope estimation error at different levels of mean gain for different light levels. It is clear here that at the lowest light levels, performance predicted by the “EMCCD simple” model is worse than that of other models. We note that at 100 photons per subaperture (and at higher light levels), the “EMCCD simple” model is optimistic. We also note that the “EMCCD stochastic” and “EMCCD distribution” models give almost identical performances.

Fig. 5

A figure showing slope estimation error as a function of EMCCD mean gain, for different EMCCD readout models (given in the legend) at different light levels (given on the graph, in photons per subaperture), for an $8\times 8\text{pixel}$ subaperture, with a spot of diameter 4 pixels (first Airy minimum).

3.2.

Impact of Thresholding Schemes

Figure 6 shows the improvement in slope error brought about by thresholding of the EMCCD output for different signal levels, as a function of EMCCD gain, when compared with an unprocessed stochastic multiplication model. It can be seen that by using the thresholding scheme and applying the photometric correction, a reduction in slope estimation error is achievable, reducing the error by up to 5% under certain signal level conditions. We note that the photometric correction is necessary, as applying thresholding without this correction results in poorer performance. The reduction in slope error is at best about 5%, and at the lowest light levels, performance is worse, therefore, we recommend that further investigation is required for a given situation (subaperture size, spot size, etc.) before this strategy should be considered.

Fig. 6

The improvement in slope estimation accuracy resulting from application of a thresholding scheme as a function of EMCCD gain at different signal levels (as given within the plots), compared to a stochastic multiplication gain model. An improvement is signified by a value greater than unity (the dotted line at unity refers to performance without thresholding). Results with, and without, photometric correction are given, and the subaperture size is $8\times 8\text{pixels}$, for a spot of diameter 4 pixels (first Airy minimum).

3.3.

Scientific CMOS Model Implications

The parameter most commonly given for sCMOS readout noise by camera manufacturers is the median value, which is as low as 0.8 photoelectrons for second generation devices. The RMS readout noise is also sometimes given, with typical values around 1.1 photoelectrons. For instrument design studies, it can be tempting to use either of these values, or something in between, when modeling sCMOS detectors, for example, Ref. 8 use a value of 1 photoelectron as representative of sCMOS readout noise.

Here, we compare slope estimation accuracy using both the typical median and mean values, and also using models with interpixel variation in readout noise, following the distribution given in Fig. 1. This probability distribution gives a median readout noise of 0.8 photoelectrons and a mean of 1.08.

Figure 7 shows slope estimation accuracy comparing these different models when large subapertures ($16\times 16\text{pixels}$) are used. The EMCCD stochastic model performance is also shown for comparison. It is interesting to note that using the median and mean sCMOS models provides a significantly better performance than the EMCCD model. At first sight, if one of these simple sCMOS models is used during instrument development, then it will appear that sCMOS technology is more appropriate for Shack–Hartmann wavefront sensing than EMCCD technology. However, once the probability distribution for readout noise is taken into account, this is clearly no longer the case. As Fig. 7 shows, true sCMOS performance is significantly worse than the predicted performance using the simple models.

Fig. 7

A figure showing slope estimation error as a function of signal level for different detector readout models, for a subaperture with $16\times 16\text{pixels}$.

3.3.2.

Performance dependence on subaperture size

Figure 8 shows the slope estimation accuracy for different detector readout models on a $4\times 4\text{pixel}$ subaperture. For all but the lowest light levels, the “average” expected performance using the sCMOS detector (the “sCMOS random” model) is better than that of the EMCCD. It is interesting to note that some subapertures are “lucky,” with a performance at the level of that predicted by simple sCMOS models (i.e., constant readout noise equal to median or mean). This is because, with far fewer pixels, there is a higher probability that all pixels within a subaperture can avoid the tail of the probability distribution. Of the 10 subaperture readout noise patterns used, the maximum RMS readout noise varied between 0.94 (for the best subaperture) and 7 electrons (for the worst). The mean RMS values ranged from 0.72 to 1.3 electrons.

Fig. 8

A figure showing slope estimation error as a function of signal level for different detector readout models, for a subaperture with $4\times 4\text{pixels}$.

Figure 9 shows slope estimation accuracy as a function of subaperture size. It can be seen that the EMCCD performance is better than the sCMOS performance for subaperture sizes equal to and greater than $6\times 6\text{pixels}$. For comparison, the simple sCMOS model results are also provided and show that performance will be greatly overestimated if these models are used.

Fig. 9

A figure showing slope estimation error as a function of subaperture size for different detector readout models as given in the legend. The subaperture size refers to the linear dimension, i.e., the square root of the total number of pixels within a subaperture.

Therefore, we recommend that proper models of sCMOS readout noise should always be used when modeling instrument performance.

3.3.3.

Performance dependence on spot size

Figure 10 shows slope estimation accuracy as a function of Shack–Hartmann spot size. For smaller subapertures, the sCMOS performance is better than the EMCCD performance. However, for larger subapertures, the EMCCD performance is generally better, particularly as spot size increases (with the available flux being spread over more pixels). We note that the performance of the sCMOS technology predicted using the full noise distribution model is always significantly worse than performance predicted using a simple (constant RMS readout noise) model for larger subapertures.

Fig. 10

A figure showing slope estimation error as a function of spot diameter (Airy ring first minimum diameter) for different detector readout models.

3.3.4.

Simple model for scientific CMOS readout noise

We have established that using the mean or median sCMOS RMS readout noise when estimating instrumental performance is optimistic. Unfortunately, using a full probability distribution will introduce additional complexity to instrumental modelling and increase the parameter space that requires exploration, in part due to the need to randomly sample different parts of the probability distribution (to sample different areas of a detector) to obtain an average expected performance. Therefore, if a single-parameter model sCMOS readout noise can be obtained, this will greatly simplify instrumental modeling.

Figure 11 compares the slope estimation error ($R$) for different readout noise models. These include the 10 “sCMOS fixed” models identified earlier (e.g., Fig. 2), the “sCMOS random” model, and also models with a range of constant RMS readout noise values. By comparing the “sCMOS random” model with the closest constant RMS model for a given signal level, we can get a feel for the “effective” readout noise of the detector for that particular case.

Fig. 11

A figure showing slope estimation error as a function of signal level for different sCMOS readout noise models, for a $4\times 4$ (a) and $16\times 16$ (b) pixel subaperture. The dashed lines show the slope error predicted using a constant RMS readout noise, the value of which is given adjacent to the lines. The solid line shows the average expected slope area from a sCMOS detector (the “sCMOS random” model). The dotted lines represent the “sCMOS fixed” model and are shown to give an idea of spread in performance.

To make sense of this information and to provide a useful reference for future instrument modelling, Fig. 12 shows the single-value RMS readout noise that will provide the same performance as predicted by the “sCMOS random” model, for different subaperture and spot sizes. To use this figure when modeling a specific AO instrument, the known subaperture and spot size can be used to read off an effective RMS Gaussian readout noise for an sCMOS detector on the figure, i.e., a single readout value for the detector. This effective RMS Gaussian readout noise can then be used to predict AO system performance, giving a similar result as that expected if the full randomly sampled RMS readout noise probability distribution had been used, but with reduced complexity.

Fig. 12

A figure showing the effective Gaussian sCMOS RMS readout noise that predicts the same slope estimation performance as that predicted by random sampling of the RMS readout noise probability distribution. Values are given as a function of subaperture size and spot diameter (Airy ring first minimum diameter). Six different signal levels are shown, with the number of photons per subaperture given inset in the figures.

It is important to note that different sCMOS detector generations and chip sizes will have a different RMS readout noise probability distribution. We, therefore, recommend that an equivalent to Fig. 12 should be generated for the specific detector family under consideration in an instrument design. Using this information will allow a more accurate prediction of instrumental performance to be made.

3.3.5.

Elongated spots for laser guide stars

So far, we have only considered Shack–Hartmann point spread functions (PSFs) with circular symmetry. However, it is also important to consider the case when extended LGS sources are used, producing elongated PSFs. Figure 13 shows slope estimation error as a function of elongation for a $16\times 16\text{pixel}$ subaperture. A two-dimensional Gaussian model has been used for the LGS spot PSF, with the Gaussian standard deviation in one dimension investigated. We note that using a standard deviation of unity gives a spot of size broadly equivalent to an Airy disk with the diameter of the first minimum being six pixels. We apply the different readout noise models to these elongated spots, as described previously.

Fig. 13

A figure showing slope estimation error as a function of LGS elongation, with the relevant elongated spot PSFs shown inset ($16\times 16\text{pixel}$ subapertures). The different detector readout noise models are given in the legend. Ten different fixed-pattern sCMOS RMS readout noise patterns are shown undifferentiated, to give an idea of the spread in performance due to the random nature of the readout noise probability distribution.

It can be seen that EMCCD technology provides the lowest error. All three EMCCD readout noise models predict very similar performances, thus only the “EMCCD stochastic” model is shown for clarity. It is interesting to note that as the spot becomes more elongated, the sCMOS performance predicted using the “sCMOS mean” model becomes closer to that predicted by the “sCMOS random” model, thus suggesting that a simple model for sCMOS readout noise is applicable for elongated LGS spots. We note that in a real AO system, the degree of LGS elongation will depend on subaperture position within the telescope pupil, and some subapertures will remain almost unelongated. In this case, the simple sCMOS readout noise model is optimistic, and so we recommend that a full sCMOS readout noise model based on the RMS readout noise probability distribution should be used whenever readout noise is a key instrument design consideration.

3.3.6.

Considerations of quantum efficiency

We have so far ignored detector QE and assumed identical QE for all detector models (though we halve the effective QE for the simple EMCCD model). The QE of EMCCD devices can reach 95% (e.g., the Andor iXon3), while for sCMOS detectors, it is closer to 70% (e.g., the Andor Zyla 4.2). Figure 14 shows the slope estimation error once the QE is taken into account and can be compared directly with Fig. 8 (which assumes identical QEs). It can be seen here that EMCCD performance is now at least as good as that predicted by the “sCMOS random” model, i.e., in practice, an EMCCD detector is likely to perform as well as a sCMOS detector for $4\times 4\text{pixel}$ subapertures (and, as we have seen previously, better for larger subapertures).

Fig. 14

A figure showing slope estimation error as a function of incident signal level for a $4\times 4\text{pixel}$ subaperture. A 70% quantum efficiency (QE) is assumed for sCMOS models, and 95% QE for EMCCD models.

3.4.

Astrometric Accuracy

We have investigated the effect of detector readout noise models on image centroiding accuracy. In addition to importance for AO systems, accurate position determination is critical for astrometric techniques.

We have shown that there are only small differences in estimation accuracy between the commonly used simple EMCCD model and a full stochastic gain mechanism model. However, for some astrometric observations, this difference may be critical, therefore, we recommend that the full stochastic gain mechanism (or the EMCCD output probability distribution model, which is almost identical) should be used, until it can be demonstrated that the simple model is sufficient for each particular instrument study. We note here that the stochastic model is computationally more expensive than other models.

When using sCMOS technology for astrometric applications, greater care is required. We have shown that a simple model of sCMOS readout noise based on a single RMS value for all pixels (whether the median or mean) is optimistic. Therefore, a model for sCMOS readout noise that uses the per-pixel probability distribution for RMS noise is essential. Further model improvements can be made if the precise RMS readout noise pattern for a physical detector under consideration can be used (i.e., once the detector has been acquired), though we do not consider this further here.