Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The first ISG alpha version was 1.7 . Described here is version 1.23 24 (You can query the version using the method isg.version() .

...

Help on function generateImagingScript in module almaqa2isg:


generateImagingScript(vis='', draft_threshold='', reqchanwidth=1, makecubes=True, docontsub=None, chanwidthtol=0.05, restfreqs={}, additionalim

g=['CALIBRATE_POLARIZATION', 'OBSERVE_CHECK_SOURCE'], spwmap=[], perchanweightdensity=False) The ALMA QA2 Imaging Script Generator

vis - the MS(s) to image, can be a list.
If wildcards are used, a list is created automatically.

draft_threshold - the cleaning threshold to use as a initial value in the cleaning (as a string e.g. "5 mJy")

reqchanwidth - the requested channel width (for cubes) as a string (e.g. "200MHz", "20km/s")
or as an integer which indicates units of or as an integer which indicates units of original channels; original channels;
if == None, the draft_threshold is used for the agg. bw. default: 1 makecubes
default: 1

makecubes - Boolean to indicate whether cubes should be made (set to False for continuum only)
default: True

docontsub - Boolean to indicate whether continuum cubessubtraction should be made done
(setonly torelevant False for continuum only) default: True docontsub - Boolean to indicate whether continuum subtraction should be done (only relevant if makecubes==True) default: True for standard polarisation, False for full polarisation data sets chanwidthtol - the tolerance in units of channel widths to apply when deciding whether the grids of corresponding SPWs from two different MSs are aligned default: 0.05 (i.e. 5%) restfreqs - the restfrequencies to be used for the restfreq parameter in the cube cleaning in the format of a dictionary, e.g. {25: '230GHz', 27: '231GHz', ... } with one entry for each science SPW (optional). default: {} (empty dictionary; the restfreqs, if needed, will be extracted from the SOURCE table or, if not available there, the central freqs will be used) additionalimg - optional list of additional intents to be imaged. For each field found with the given intent (give name omitting the "#ON_SOURCE"), code for making an aggregate bandwidth image is going to be added. Possible values: 'CALIBRATE_PHASE', 'CALIBRATE_BANDPASS', 'CALIBRATE_FLUX', 'OBSERVE_CHECK_SOURCE', 'CALIBRATE_POLARIZATION' default: ['CALIBRATE_POLARIZATION', 'OBSERVE_CHECK_SOURCE'] (image check source(s) and/or polcal) example: ['CALIBRATE_BANDPASS', 'CALIBRATE_PHASE'] spwmap - SPW IDs to be used in the final image names. If a non-empty list is specified, it must contain at least as many elements as there are science SPWs in the input MS. The given SPW IDs are then used to replace in the image names the ones used in the MS, e.g. if there are science spws [0,1,2,3] in the MS, then setting spwmap=[19,21,23,25] will result in ID 19 being used instead of 0 in the image names, 21 instead of 1 etc. default: [] (empty list) - use the SPW IDs as they are in the MS. perchanweightdensity - the setting of the tclean parameter perchanweightdensity default: False Example: import almaqa2isg as isg isg.generateImagingScript('uid*.ms.split.cal', draft_threshold='0.1mJy', reqchanwidth='20km/s', spwmap=[17,19,21,23]) Discussion: You can run the Imaging Script Generator (ISG) like so: import almaqa2isg as isg cd <the directory where the MS(s) are which you want to image, e.g. "calibrated"> isg.generateImagingScript(vis='uid___A00*.ms.split.cal', draft_threshold='0.1mJy', reqchanwidth='20km/s') The vis parameter can be a single string or a list. If it is a list, each element must be the name of an existing MS. If it is a single string, it can either be a single name of an existing MS or an expression using wildcards "*" or "?" to specify a group of MSs. This will internally be converted to a list of MSs. If makecubes is true (default), code for cleaning cubes for each science SPW is created. If docontsub is true, code for continuum subtraction (separate for each science field) is created. This parameter is only active is makecubes is true. draft_threshold specifies the threshold which is entered into the cleaning commands. It is related to the required rms from the MOUS proposal. It is used as the cleaning threshold in cube cleaning if makecubes is true (then the threshold for the agg. bw. image is left for editing by the analyst). If makecubes==False, then draft_threshold is used as the cleaning threshold for the agg. bw. image. reqchanwidth is the bandwidth on which draft_threshold is defined. It is used as the channel width for the cubes if makecubes is true. Like for the calibration script generator, the ISG is not meant to produce a script which one can run blindly. We are not trying to reinvent the imaging pipeline. So, if different channel widths are required for different SPWs, the analyst needs to edit the channel width (and the corresponding thresholds) for some of the cubes. A general solution to permit different thresholds and widths for different SPWs was deemed too complex for the moment. NOTE that if there is more than one input MS and docontsub is True, then a concat step will be inserted such that the uvcontsub commands can operate on the concatenated MS. If the SPWs are not yet reindexed to have IDs starting at 0, uvcontsub will do this reindexing. The ISG will take this into account and use reindexed SPW IDs when making the cubes but leave the old SPW ids in the image names (this is what archive needs). Furthermore, if constinuum subtraction is requested, the ISG will check if the corresponding SPWs of the MSs are shifted w.r.t. each other and if so, find the largest common grid for each SPW and generate mstransform code to transform all science SPWs of all the MSs into these grids before concatenation. At the end of the isg run, you obtain a file "scriptForImaging.py". On the terminal (in CASA) you will also get messages which explain what you need to do to complete this script before you can run it. These messages look, e.g., like this: Script generation completed. Please find scriptForImaging.py in the current directory. NOTE that you still need to edit the script: - You need to edit the array "therestfreqs" to set the rest frequencies for the cubes of each SPW. - You need to adjust the threshold for the aggregate bandwidth/continuum image(s). - You need to edit the fitspw parameter in each uvcontsub command. Iterative use for continuum identification ------------------------------------------ If cubes are to be created and a decision needs to be taken about whether contsub is necessary and how it should be done, one generally can go two routes: a) use plotms to generate amp vs. channel plots to identify lines and line-free regions b) create image cubes for each spw once to see if they contain detectable spectral lines and identify continuum channels. When going route (b), it is recommended that the user runs the ISG twice, first with docontsub=False and then with docontsub=True: With the first version, one can create some shallowly cleaned cubes to check for lines and then move the first script version and the created images to a different directory and start again with docontsub=True. In order to make sure that a new run of the ISG does not destroy your previous version of the script, the ISG prevents you from overwriting a pre-existing scriptForImaging.py SPW Numbering ------------- If you have used split or mstransform *with* reindexing to split out certain SPWs to obtain the input MS(s) to the ISG, i.e. if your input MS(s) do *not* use the same SPW numbering as the original ASDM, you need to use the parameter "spwmap" to get the archive-compliant numbering in the image names. Script generation completed. Please find scriptForImaging.py in the current directory. NOTE that you still need to edit the script: - You need to edit the array "therestfreqs" to set the rest frequencies for the cubes of each SPW. - You need to adjust the threshold for the aggregate bandwidth/continuum image(s). - You need to edit the fitspw parameter in each uvcontsub command. Iterative use for continuum identification ------------------------------------------ If cubes are to be created and a decision needs to be taken about whether contsub is necessary and how it should be done, one generally can go two routes: a) use plotms to generate amp vs. channel plots to identify lines and line-free regions b) create image cubes for each spw once to see if they contain detectable spectral lines and identify continuum channels. When going route (b), it is recommended that the user runs the ISG twice, first with docontsub=False and then with docontsub=True: With the first version, one can create some shallowly cleaned cubes to check for lines and then move the first script version and the created images to a different directory and start again with docontsub=True. In order to make sure that a new run of the ISG does not destroy your previous version of the script, the ISG prevents you from overwriting a pre-existing scriptForImaging.py SPW Numbering ------------- If you have used split or mstransform *with* reindexing to split out certain SPWs to obtain the input MS(s) to the ISG, i.e. if your input MS(s) do *not* use the same SPW numbering as the original ASDM, you need to use the parameter "spwmap" to get the archive-compliant numbering in the image names. The spwmap parameter should contain an ordered list of the *original* SPW IDs of *all* the science SPWs in the input MS(s). The input MSs, if there is more than one, need to all use the same numbering. A typical example is [19,21,23,25]. You need to look at the original raw MS after importasdm to obtain this information. Full Polarization support ------------------------- The ISG will detect whether an input MS contains full polarisation data. In this case it will use stokes='IQUV' and dconvolver='clarkstokes'. Furthermore, if the intent CALIBRATE_POLARISATION is present, it will by default make the appropriate calibrator image and add operations to calculate polcal parameters. tclean parameter "perchanweightdensity" ----------------------------- The tclean parameter "perchanweightdensity" was introduced in CASA 5.6 and is True by default. However, in ALMA QA2 data analysis it is always set to False in order to achieve the same image parameters as expected by scheduling. In the ISG, the default is False but you can override this by setting the ISG parameter "perchanweightdensity" (same name as in tclean) to True. If you run the ISG with CASA versions earlier than 5.6, the parameter is ignored and not inserted into the tclean commands in scriptForImaging.py. if makecubes==True)
default: True for standard polarisation, False for full polarisation data sets

chanwidthtol - the tolerance in units of channel widths to apply when deciding whether the
grids of corresponding SPWs from two different MSs are aligned
default: 0.05 (i.e. 5%)

restfreqs - the restfrequencies to be used for the restfreq parameter in the cube cleaning
in the format of a dictionary, e.g. {25: '230GHz', 27: '231GHz', ... } with
one entry for each science SPW (optional).
default: {} (empty dictionary; the restfreqs, if needed, will be extracted from
the SOURCE table or, if not available there, the central freqs will be used)

additionalimg - optional list of additional intents to be imaged.
For each field found with the given intent (give name omitting the "#ON_SOURCE"),
code for making an aggregate bandwidth image is going to be added.
Possible values: 'CALIBRATE_PHASE', 'CALIBRATE_BANDPASS', 'CALIBRATE_FLUX',
'OBSERVE_CHECK_SOURCE', 'CALIBRATE_POLARIZATION'
default: ['CALIBRATE_POLARIZATION', 'OBSERVE_CHECK_SOURCE'] (image check source(s) and/or polcal)
example: ['CALIBRATE_BANDPASS', 'CALIBRATE_PHASE']

spwmap - SPW IDs to be used in the final image names.
If a non-empty list is specified, it must contain at least as many
elements as there are science SPWs in the input MS. The given SPW IDs are then
used to replace in the image names the ones used in the MS,
e.g. if there are science spws [0,1,2,3] in the MS,
then setting spwmap=[19,21,23,25] will result in ID 19
being used instead of 0 in the image names, 21 instead of 1 etc.
default: [] (empty list) - use the SPW IDs as they are in the MS.

perchanweightdensity - the setting of the tclean parameter perchanweightdensity
for "briggs" weighting. Starting with CASA 6.2, "briggs" weighting will
only be used for mfs imaging. For cubes, the weighting option "briggsbwtaper"
will be used and perchanweightdensity will be forced to True.
default: False

Example:
import almaqa2isg as isg
isg.generateImagingScript('uid*.ms.split.cal', draft_threshold='0.1mJy', reqchanwidth='20km/s', spwmap=[17,19,21,23])

Discussion:
You can run the Imaging Script Generator (ISG) like so:

import almaqa2isg as isg
cd <the directory where the MS(s) are which you want to image, e.g. "calibrated">
isg.generateImagingScript(vis='uid___A00*.ms.split.cal', draft_threshold='0.1mJy', reqchanwidth='20km/s')

The vis parameter can be a single string or a list. If it is a list, each element must be the name of an
existing MS. If it is a single string, it can either be a single name of an existing MS or an expression
using wildcards "*" or "?" to specify a group of MSs. This will internally be converted to a list of MSs.

If makecubes is true (default), code for cleaning cubes for each science SPW is created.

If docontsub is true, code for continuum subtraction (separate for each science field) is created.
This parameter is only active is makecubes is true.

draft_threshold specifies the threshold which is entered into the cleaning commands. It is related to
the required rms from the MOUS proposal. It is used as the cleaning threshold in cube
cleaning if makecubes is true (then the threshold for the agg. bw. image is left for editing by the analyst).
If makecubes==False, then draft_threshold is used as the cleaning threshold for the agg. bw. image.

reqchanwidth is the bandwidth on which draft_threshold is defined. It is used as the channel width for the cubes if
makecubes is true.

Like for the calibration script generator, the ISG is not meant to produce a script which one can run
blindly. We are not trying to reinvent the imaging pipeline. So, if different channel widths are required
for different SPWs, the analyst needs to edit the channel width (and the corresponding thresholds) for some
of the cubes. A general solution to permit different thresholds and widths for different SPWs was deemed too
complex for the moment.

NOTE that if there is more than one input MS and docontsub is True, then a concat step will be inserted such
that the uvcontsub commands can operate on the concatenated MS. If the SPWs are not yet reindexed to have
IDs starting at 0, uvcontsub will do this reindexing. The ISG will take this into account and use reindexed
SPW IDs when making the cubes but leave the old SPW ids in the image names (this is what archive needs).

Furthermore, if continuum subtraction is requested, the ISG will check if the corresponding SPWs of the MSs
are shifted w.r.t. each other and if so, find the largest common grid for each SPW and generate mstransform code
to transform all science SPWs of all the MSs into these grids before concatenation.

At the end of the isg run, you obtain a file "scriptForImaging.py". On the terminal (in CASA) you will also
get messages which explain what you need to do to complete this script before you can run it. These messages
look, e.g., like this:

Script generation completed. Please find scriptForImaging.py in the current directory.

NOTE that you still need to edit the script:
- You need to edit the array "therestfreqs" to set the rest frequencies for the cubes of each SPW.
- You need to adjust the threshold for the aggregate bandwidth/continuum image(s).
- You need to edit the fitspw parameter in each uvcontsub command.

Iterative use for continuum identification
------------------------------------------

If cubes are to be created and a decision needs to be taken about whether contsub is necessary and how it
should be done, one generally can go two routes:

a) use plotms to generate amp vs. channel plots to identify lines and line-free regions

b) create image cubes for each spw once to see if they contain detectable spectral lines and identify
continuum channels.

When going route (b), it is recommended that the user runs the ISG twice, first with docontsub=False and then
with docontsub=True: With the first version, one can create some shallowly cleaned cubes to check for lines
and then move the first script version and the created images to a different directory and start again with
docontsub=True.

In order to make sure that a new run of the ISG does not destroy your previous version of the script, the
ISG prevents you from overwriting a pre-existing scriptForImaging.py

SPW Numbering
-------------

If you have used split or mstransform *with* reindexing to split out certain SPWs to obtain the input MS(s) to the ISG,
i.e. if your input MS(s) do *not* use the same SPW numbering as the original ASDM, you need to use the parameter
"spwmap" to get the archive-compliant numbering in the image names.
The spwmap parameter should contain an ordered list of the *original* SPW IDs of *all* the science SPWs
in the input MS(s). The input MSs, if there is more than one, need to all use the same numbering.
A typical example is [19,21,23,25]. You need to look at the original raw MS after importasdm to obtain this information.

Full Polarization support
-------------------------

The ISG will detect whether an input MS contains full polarisation data. In this case it will use stokes='IQUV' and
dconvolver='clarkstokes'. Furthermore, if the intent CALIBRATE_POLARISATION is present, it will by default make
the appropriate calibrator image and add operations to calculate polcal parameters.



Feedback, bug reports, feature requests

...