The ALMA QA2 Calibration Script Generator almaqa2csg.py (CSG) for speeding up and standardising the generation of scripts used for manual calibration of ALMA data. On this page you find the documentation of the public version of it (which may be a version or two behind the latest version in use at the ALMA observatory).
The CSG has a number of optional input parameters. However, the only necessary input parameter is the name of the ASDM (raw ALMA data set) to be calibrated. Based on the properties of the dataset, almaqa2csg creates a "scriptForCalibration.py" tailored to the needs of the particular case. The result is a nearly complete script draft which may work as is but should be inspected by the analyst to make sure all steps are correct. So, the simplest possible call to the CSG looks like this
import almaqa2csg as csg
csg.generateReducScript('uid___A002_X7c0875_Xd65')
This will import the given ASDM into an MS (named uid___A002_X7c0875_Xd65.ms in this case) and produce a calibration script draft file named "uid___A002_X7c0875_Xd65_scriptForCalibration.py" for it.
NOTE: The CSG expects the ASDM to be named by only the Execution Block (EB) UID, e.g., uid___A002_X7c0875_Xd65 as above. Any extensions like ".asdm.sdm" need to be removed. Soft links are accepted.
The calibration scripts produced by the CSG use the now common "stepping mechanism" (which was added by D. Petry in 2012):
the idea is that the user of the script controls the execution of the different procedural steps of the calibration without having to edit the script,
thus avoiding the risk of introducing bugs in the script. The mechanism works as follows:
In the script header, the user can inspect a Python dictionary which contains the different steps of the script.
Example:
step_title = {0: 'Import of the ASDM',
1: 'Fix of SYSCAL table times',
2: 'listobs',
3: 'A priori flagging',
4: 'Generation and time averaging of the WVR cal table',
5: 'Generation of the Tsys cal table',
6: 'Generation of the antenna position cal table',
7: 'Application of the WVR, Tsys and antpos cal tables',
8: 'Split out science SPWs and time average',
9: 'Listobs, and save original flags',
10: 'Initial flagging',
11: 'Putting a model for the flux calibrator(s)',
12: 'Save flags before bandpass cal',
13: 'Bandpass calibration',
14: 'Save flags before gain cal',
15: 'Gain calibration',
16: 'Save flags before applycal',
17: 'Application of the bandpass and gain cal tables',
18: 'Split out corrected column',
19: 'Save flags after applycal'}For a correct calibration, all these steps have to be executed in the given order.
But the user can execute one or a few steps at a time in order to inspect intermediate results or rerun certain steps after modification.
At the CASA prompt, the user sets the variable "mysteps" (note the "s" at the end!), e.g.
mysteps = [2,3,4,5,6]
Then the calibration script is run using
execfile('uid___A002_X7c0875_Xd65_scriptForCalibration.py')
and this will execute only the steps 2, 3, 4, 5, and 6 of the script and then stop. The user can then look at the various plots produced by the script or use the task plotms to inspect calibration tables.
Later, the user can either make modifications to the script and repeat steps or just continue setting, e.g.
mysteps = [4,5,6]
and calling execfile again.
NOTE: the order in which you enumerate the steps in the mysteps variable does not matter. The steps will always be executed in the order given in the script.
If you don't set mysteps, running the script will execute all steps from 0 to the last.
The CSG is somewhat computing intensive. The execution time depends on the size of the input dataset and can take several 10 minutes.
Pay close attention to error messages on the terminal.
The CSG internally uses the analysisUtils and needs CASA 5.1.1 or later. CASA 6 is recommended.
Access
The CSG is contained in a single Python module almaqa2csg which is part of the analysis_scripts.tar package which also contains the "analysisUtils" which you will also need since the CSG uses a few of their functions.
See the Zenodo analysisUtils page and the Analysis Utilities CASA Guide for details on how to install the package.
Usage
With your sys.path set up to import the analysisUtils, type
import almaqa2csg as csg
help csg.generateReducScript
to obtain the following help:
Help on function generateReducScript in module almaqa2csg:
generateReducScript(msNames='', step='calib', corrAntPos=True, timeBinForFinalData=0.0, refant='', bpassCalId='', chanWid=1, angScale=0, run=False, lowSNR=False, projectCode='', schedblockName='', schedblockUid='', queue='',
state='', upToTimeForState=2, useLocalAlmaHelper=False, tsysChanTol=1, sdQSOflux=1, runPhaseClosure=False, skipSyscalChecks=False, lazy=False, lbc=False, phaseDiff=False, remcloud=False, bdfflags=True, phaseDiffPerSpwSetup=
False, tsysPerField=False, splitMyScienceSpw=True, bpassCalTableName='', reindexMyScienceSpw=False, useCalibratorService=False)
The ALMA QA2 calibration script generator
msNames: a string or a list of strings of UIDs (either ASDM or MS) to process
NOTE: rigorous regression testing is presently only done on single UIDs, not lists
default=''
step: calib, fluxcal, wvr, calsurvey, SDeff, SDcalibLine, SDcalibCont, SDscience, SDampcal
default='calib'
corrAntPos: if True, then run correctMyAntennaPositions
default=True
timeBinForFinalData: a value in seconds (string, int, or float), passed to split
default=0.
refant: the reference antenna to use (instead of automatic selection), must be a string
default='', i.e. determine automatically
bpassCalId: use the specified source for bandpass (rather than determine from the intents)
default='', i.e. determine from the intents
chanWid: integer, used by runCleanOnSource and searchForLines
default=1
angScale: deprecated
run: deprecated
lowSNR: Boolean passed to doBandpassCalibration to use whole spw for pre-bandpass phase-up
default=False
projectCode, schedblockName, queue, state, upToTimeForState: deprecated
useLocalAlmaHelper: if True, run tsysspwmap inside generator, rather than in the resulting script
default=False
tsysChanTol: integer argument passed to tsysspwmap
default=1
sdQSOflux: flux density to use for quasar in single dish case (step='SDeff')
default=1
runPhaseClosure: deprecated
skipSyscalChecks: if True, then don't check for negative Tsys problems
default=False
lazy: value of the 'lazy' parameter in importasdm. If True, reference the ASDM instead
of copying the visibilities into the DATA column of the MS. Saves disk space.
default=False
lbc: if True, invoke long-baseline campaign usage in correctMyAntennaPositions, i.e.
search='both_latest' and maxSearchDays=30;
secondly, in bandpass calibration, use solint='inf,8MHz' instead of 'inf,20ch'
default=False
remcloud: runs the recipe remove_cloud prior to running wvrgcal
default=False
bdfflags: passed to importasdm to invoke the application of BDF flags
default=True
bpassCalTableName: to use instead of default name
default='', i.e. use the bp table created for bpassCalId with the standard naming
phaseDiff: force the combination of spw. If it is not enabled, and a DGC scan is detected, then it will
automatically enable it.
default=False
phaseDiffPerSpwSetup: controls the iteration loop for the calculation of gaincal table with spw combined.
If False, the iteration will be done over the fields; if True, the iteration will be done over
the groups of spw from the same setup.
default=False
tsysPerField: passed to the perField parameter of tsysspwmap
default=False
splitMyScienceSpw: In the final split-out, only include the SPWs corresponding to intent OBSERVE_TARGET
and BANDPASS.
default=True
reindexMyScienceSpw: perform reindexing in the split out after the apriori calibration.
default=False
useCalibratorService: if True, then, in the call to aU.getALMAFluxForMS in the setjy step, use
aU.calibratorService(), otherwise use aU.getALMAFlux()
default=False
Feedback, bug reports, feature requests
Feedback, bug reports, feature requests should be made by email to dpetry@eso.org .