101. LSSTCam visits database

101. LSSTCam visits database¶

For the Rubin Science Platform at data.lsst.cloud.
Container Size: Large
LSST Science Pipelines version: v29.2.0
Last verified to run: 2026-06-09
Repository: github.com/lsst/tutorial-notebooks
DOI: 10.11578/rubin/dc.20250909.20

Learning objective: How to query and retrieve data from the temporary LSSTCam visits database file.

LSST data products: The file prelsst_20260607_visits.db has been made temporarily available until it is superseded by upcoming data releases.

Packages: sqlite3, rubin_sim

Credit: Developed by the Rubin Community Science team together with the Rubin Survey Scheduling team. Please consider acknowledging them if this notebook is used for the preparation of journal articles, software releases, or other notebooks.

Get Support: Everyone is encouraged to ask questions or raise issues in the Support Category of the Rubin Community Forum. Rubin staff will respond to all questions posted there.

1. Introduction¶

This tutorial demonstrates how to query and load data from a temporary SQL-formatted table of LSSTCam visits that is available in the Rubin Science Platform. An early version of this file, for visits up to the end of September 2025, was first made available on the Science Validation survey summary webpage . This is a temporary, static database file with non-standard schema and formatting. It only includes LSSTCam visit metadata from April 2025 to June 2026. For more recent visits and a forecast of the Rubin scheduler, see the tutorial notebook for the Rubin Schedule Viewer. The future Rubin data releases will have similar information in their Visit and CcdVisit tables.

Science Validation surveys.

Science images with LSSTCam began on 04 April 2025, at first acquiring small field survey visits in sequences of $\sim$10 visits per filter with small dithers. These small field survey visits included the images which contributed toward Rubin First Look. The Science Validation (SV) survey began on 20 June 2025, acquiring visits in a manner consistent with the planned operations survey for the LSST, but within a limited area. Review the Science Validation survey summary webpage for details on the strategy. The contiguous part of the SV area follows the ecliptic plane from dense regions of the Galactic Bulge through low-dust regions within the planned LSST Wide Fast Deep (WFD). Four of the planned LSST Deep Drilling Fields (DDFs) were included in the SV survey, and a secondary area within the low-dust WFD was included to provide targets when the primary or DDF fields were not available. The scientifically validated subset of these images obtained prior to Jan 7 2026 will be released as Data Preview 2 (DP2).

Pre-LSST, AOS commissioning, and engineering visits.

Throughout the first half of 2026, pre-LSST visits were obtained in the DDFs where templates exist, for the purpose of alert production, and for engineering and Active Optics System (AOS) commissioning with the goal of reaching stable image quality metrics which meet the conditions for starting the LSST. Many of the latter visits have been tagged as suitable for science (pending processing and science validation).

Caveats.

• Not all of these visits lead to scientifically validated data products. Some will end up excluded from the DP2 and the Prompt Products datasets. Although an initial cut of bad visits have been made on the inputs to the database, users should expect that additional cuts post-processing.
• Image quality (IQ) is variable. The database file excludes bad visits, but includes visits with a wide range of data quality due to both cloud extinction and/or delivered IQ or engineering issues. Keep in mind that while these visits were obtained, the AOS was being commissioned.
• Measured IQ values may change. Some columns contain NaNs, where the summit quicklook processing did not provide a useful value. Many of these problems will be resolved with later processing. Users should anticipate that some measured IQ values will change.

1.1. Import packages¶

Import sqlite3 to read the SQL-formatted database file, and import the maf module from the rubin_sim package to use the (Metric Analysis Framework) functions. Import the skyproj package (skyproj.readthedocs.io) for plotting all-sky projection plots, the healpy package (healpy.readthedocs.io) for dealing with HEALPix. Also import standard python science packages and the lsst.utils.plotting package.

In [1]:

import sqlite3
from rubin_sim import maf

import skyproj
import healpy as hp

import os
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import astropy.units as u
from astropy.coordinates import SkyCoord
from tabulate import tabulate
from astropy.time import Time
from lsst.utils.plotting import (get_multiband_plot_colors,
                                 get_multiband_plot_linestyles)

1.2. Define parameters¶

Define the path and name of the database file.

In [2]:

db_filename = '/rubin/cst_repos/tutorial-notebooks-data/data/prelsst_20260607_visits.db'

Define the default path for output files. Use the shared scratch directory, /deleted-sundays/, and create a subdirectory with your username if one does not already exist.

In [3]:

output_path = os.getenv('SCRATCH_DIR')
if os.path.exists(output_path):
    print('Already exists: ', output_path)
else:
    os.system('mkdir ' + output_path)
    print('Created: ', output_path)

Already exists:  /deleted-sundays/melissagraham

Define the LSST filter names and the colors and linestyles to represent the filters.

In [4]:

filter_names = ['u', 'g', 'r', 'i', 'z', 'y']
filter_colors = get_multiband_plot_colors()
filter_linestyles = get_multiband_plot_linestyles()
filter_colors_list = [filter_colors['u'], filter_colors['g'],
                      filter_colors['r'], filter_colors['i'],
                      filter_colors['z'], filter_colors['y']]

2. Explore the database¶

The information contained in the database is an aggregation of entries in the "Consolidate Database" (ConsDB), including per-visit summary values from summit quicklook processing (the ConsDB is not yet released to users). The database generally follows the current LSST scheduler output schema, but additional columns were added in post-processing. Rubin data releases have similar information in their Visit and CcdVisit tables.

As this is an aggregate file, descriptions for its columns can be found among those at:

• ConsDB Schema
• Scheduler Output Schema

2.1. Key columns¶

The database contains many columns, but these are the key columns used in this tutorial.

• observation_reason : The source of the visit in the Feature Based Scheduler (FBS).
• target_name : The name of the sky region for the visit.
• exp_midpt_mjd : The midpoint time of the exposure at the fiducial center of the focal plane (in TAI).
• fieldRA : The boresight Right Ascension for the visit (degrees).
• fieldDec : The boresight Declination for the visit (degrees).
• band : The LSST filter used for the visit, one of $ugrizy$.
• airmass : The airmass of the visit ($1/\cos(\Theta_z)$, where $\Theta_z$ is the zenith angle).
• seeingFwhmEff : The full-width half-max of the point spread function (PSF; arcseconds).
• fiveSigmaDepth : The magnitude of a five-sigma point source detection in the visit (magnitudes).

2.2. Connect with sqlite3¶

Connect to the database file using sqlite3.

In [5]:

db_conn = sqlite3.connect(db_filename)
cursor = db_conn.cursor()

Print the names of all tables in the database. There is only one, the observations table.

In [6]:

cursor.execute("SELECT name FROM sqlite_master WHERE type='table';")
tables = cursor.fetchall()
table_names = [table[0] for table in tables]
print("Tables:", table_names)
del tables, table_names

Tables: ['observations']

Option to print the schema (column names and types) for the observations table. It is a long list, and not printed by default.

In [7]:

# table_name = 'observations'
# query = f"""SELECT sql FROM sqlite_master WHERE type='table'
#             AND name='{table_name}';"""
# cursor.execute(query)
# create_table_sql = cursor.fetchone()[0]
# print(f"Schema for {table_name}:\n{create_table_sql}")
# del query, table_name, create_table_sql

Example query.

As an example, create a query to return columns fieldRA, fieldDec, and band from the observations table for all $r$-band visits obtained at an airmass less than 1.5 and with an observation_reason of "field_survey_science". The column observation_reason is discussed in Section 3.1.

In [8]:

query = """SELECT fieldRA, fieldDec, band FROM observations
           WHERE observation_reason = 'field_survey_science'
           AND band='r' AND airmass < 1.5; """
cursor.execute(query)
results = cursor.fetchall()
print("Number of rows returned: ", len(results))

Number of rows returned:  1463

Option to display the query results.

In [9]:

# results

Clean up, and close the connection with the database.

In [10]:

del query, results
cursor.close()
del cursor

2.3. Read as a pandas dataframe¶

The file is small enough to be loaded in its entirety as a pandas dataframe.

Read the SQL-formatted table as a pandas dataframe, df, and print the number of rows.

In [11]:

df = pd.read_sql_query("SELECT * FROM observations", db_conn)
print(len(df))

81454

Option to display the table (it will automatically truncate).

In [12]:

# df

Option to print the column names.

In [13]:

# list(df.columns)

3. Observation reason and target¶

There are a few rows of the database that indicate the motivation behind each observation (each visit). All rows of the visits database have img_type = science, but as explained in Section 1 this does not necessarily mean they will pass science validation and be released. The column science_program is an internal designation, BLOCK-XXX, and probably not useful for most users.

Every visit has both an observation_reason and a target_name, and these are useful to explore.

3.1. Observation reason¶

The observation_reason is the motivation as to why the visit was obtained. In other words, it indicates the scheduler mode or the survey.

Option to print all unique values of the observation_reason column and the number of visits for each.

Warning: The feature-based scheduler (FBS) and Rubin operations were in the early stages when these visits were obtained, and the values of the observation_reason column exhibit diversity; explanations for every single value are not provided at this time.

In [14]:

# values, counts = np.unique(df['observation_reason'], return_counts=True)
# for value, count in zip(values, counts):
#     print('%25s %5i' % (value, count))
# del values, counts

Print the number of visits done for small field surveys, deep drilling fields, target of opportunity, AOS commissioning, and alert production.

In [15]:

Nsfs = len(df.query("observation_reason == 'field_survey_science'"))
Nddf = len(df.query("observation_reason.str.contains('ddf')"))
Ntoo = len(df.query("observation_reason.str.contains('too')"))
Naos = len(df.query("observation_reason.str.contains('aos')"))
Nalrt = len(df.query("observation_reason.str.contains('alert')"))
print("Number of visits done for")
print("small field surveys:    ", Nsfs)
print("deep drilling fields:   ", Nddf)
print("target of opportuntity: ", Ntoo)
print("AOS commissioning:      ", Naos)
print("alert production:       ", Nalrt)
del Nsfs, Nddf, Ntoo, Naos, Nalrt

Number of visits done for
small field surveys:     6995
deep drilling fields:    2486
target of opportuntity:  1003
AOS commissioning:       10262
alert production:        2719

3.2. Target names¶

The target_name is either a region of the LSST WFD (e.g., bulge, low-dust) or a proper name (for DDFs, or the commissioning small field survey fields). Since regions can overlap, or a single visit can overlap the boundaries of multiple regions, the target_name is often a comma-separated list. For example, most DDFs are also in the low-dust extragalactic regions of the LSST WFD.

Small field surveys.

Get the unique values of target_name for each of the small field survey areas, and print the number of visits done for each.

In [16]:

df_sfs = df.query("observation_reason == 'field_survey_science'")
values, counts = np.unique(df_sfs['target_name'], return_counts=True)
for value, count in zip(values, counts):
    print('%20s %5i' % (value, count))
del df_sfs, values, counts

          Abell_2764     7
              COSMOS   659
         DESI_SV3_R1     4
            ELAIS_S1   166
                 M49  1105
        New_Horizons   359
               Prawn   621
     Rubin_SV_212_-7   496
    Rubin_SV_216_-17   121
    Rubin_SV_225_-40  2346
    Rubin_SV_280_-48   148
    Rubin_SV_300_-41    38
    Rubin_SV_320_-15   257
       Trifid-Lagoon   668

Wide-fast-deep regions.

Option to print the target_name for all visits that were not done with an observation_reason related to the small field surveys, the deep drilling fields, or targets of opportunity - the result will be inclusive of all WFD sky regions.

In [17]:

# query = "(observation_reason != 'field_survey_science') & "
# query += "(observation_reason.str.contains('ddf') == 0) & "
# query += "(observation_reason.str.contains('too') == 0) "
# df_wfd = df.query(query)

# print("All target_name values, omitting non-WFD observation reasons.")
# values, counts = np.unique(df_wfd['target_name'], return_counts=True)
# for value, count in zip(values, counts):
#     print('%30s %5i' % (value, count))
# print(" ")
# print("Unique target_name values for WFD observation reasons.")
# temp = []
# for value in values:
#     for name in str(value).split(','):
#         temp.append(name.strip())
# targets = np.unique(temp)
# print(targets)

# del query, values, counts, temp, targets, df_wfd

The WFD sky regions covered during commissioning are:

• LMC_SMC - Large and Small Magellanic Clouds
• bulgy - Galactic bulge region
• dusty_plane - dusty regions of the Galactic plane
• euclid_overlap - overlap with Euclid space telescope field overlap
• field_m49 and virgo - Virgo cluster
• lowdust - low dust sky region
• nes - North Ecliptic Spur (NES)
• scp - south celestial pole

Learn more about all of the planned LSST WFD regions on the LSST Baseline Strategy webpage.

Notice: These WFD sky regions are not mutually exclusive; the Euclid fields are within the low-dust WFD region, for example, just like the DDFs are.

List the number of visits that overlap with each WFD sky region. As sky regions are not mutually exclusive, the sum is greater than the total number of visits.

In [18]:

WFD_targets = ['LMC_SMC', 'bulgy', 'dusty_plane', 'euclid_overlap',
               'field_m49', 'lowdust', 'nes', 'scp', 'virgo']
for target in WFD_targets:
    print('%15s %5i' % (target, len(df.query("target_name.str.contains(@target)"))))
del WFD_targets

        LMC_SMC   736
          bulgy  8649
    dusty_plane  9582
 euclid_overlap   203
      field_m49   648
        lowdust 46164
            nes  1608
            scp  2038
          virgo   115

Deep Drilling Fields.

The LSST will include five Deep Drilling Fields, four of which (all except COSMOS) were observed as part of the LSST Science Validation survey. Note the EDFS (Euclid Deep Field South) field is actually two side-by-side fields.

Table 1: DDF locations, from the DDF webpage.

Option to print the target_name for all visits that were done with an observation_reason related to the deep drilling fields.

In [19]:

# df_ddf = df.query("observation_reason.str.contains('ddf')")

# print("All target_name values with DDF observation reasons.")
# values, counts = np.unique(df_ddf['target_name'], return_counts=True)
# for value, count in zip(values, counts):
#     print('%30s %5i' % (value, count))
# print(" ")
# print("Unique target_name values for DDF observation reasons.")
# temp = []
# for value in values:
#     for name in str(value).split(','):
#         temp.append(name.strip())
# targets = np.unique(temp)
# print(targets)

# del values, counts, temp, targets, df_ddf

The naming convention for the DDFs has changed since the start of LSSTCam observations.

• DDF ECDFS --> ddf_ecdfs
• DDF EDFS_a --> ddf_edfs_a
• DDF EDFS_b --> ddf_edfs_b
• DDF ELAISS1 --> ddf_elaiss1
• DDF XMM_LSS --> ddf_xmm_lss

Standardize the target_name strings for DDF-related visits.

In [20]:

df['target_name'] = df['target_name'].str.replace('DDF ECDFS', 'ddf_ecdfs', regex=False)
df['target_name'] = df['target_name'].str.replace('DDF EDFS_a', 'ddf_edfs_a', regex=False)
df['target_name'] = df['target_name'].str.replace('DDF EDFS_b', 'ddf_edfs_b', regex=False)
df['target_name'] = df['target_name'].str.replace('DDF ELAISS1', 'ddf_elaiss1', regex=False)
df['target_name'] = df['target_name'].str.replace('DDF XMM_LSS', 'ddf_xmm_lss', regex=False)

List the number of visits that overlap with each DDF sky region.

In [21]:

DDF_targets = ['ddf_cosmos', 'ddf_ecdfs', 'ddf_edfs_a', 'ddf_edfs_b', 'ddf_elaiss1', 'ddf_xmm_lss']
for target in DDF_targets:
    print('%15s %5i' % (target, len(df.query("target_name.str.contains(@target)"))))
del DDF_targets

     ddf_cosmos  1633
      ddf_ecdfs   490
     ddf_edfs_a   729
     ddf_edfs_b   717
    ddf_elaiss1   850
    ddf_xmm_lss   214

4. Visit metadata¶

Examples of how to visualize and calculate statistics for a few key columns of visit metadata.

4.1. Plot histograms¶

Create a histogram of the Modified Julian Dates of all visits, stacked by filter.

In [22]:

t = Time("2025-04-17T12:00:00", scale='tai')
mjd_to_jd = t.mjd - t.jd
df.loc[:, 'jd'] = np.floor(df.observationStartMJD - mjd_to_jd)
df.loc[:, 'jd'] = df.jd.astype(int)
jds = np.arange(df.jd.min(), df.jd.max()+1, 7)
jdsbins = np.arange(df.jd.min(), df.jd.max()+7, 7)
days = [t.split('T')[0] for t in Time(jds, format='jd', scale='tai').isot]
bar_bottom = np.zeros(len(jds))

plt.figure(figsize=(8, 4))
for b in 'ugrizy':
    heights, _ = np.histogram(df.query("band == @b ").jd, bins=jdsbins)
    plt.bar(jds, heights, bottom=bar_bottom, width=4, color=filter_colors[b], alpha=0.8, label=b)
    bar_bottom += heights
plt.legend(loc='upper left', ncol=6)
_ = plt.xticks(jds[::7], labels=days[::7], rotation=90)
plt.grid(alpha=0.2)
plt.ylabel("Number of visits", fontsize='large')
plt.ylim([0, 4500])
plt.title("LSSTCam Science Visits in 7-day bins")
plt.show()

Figure 1: The number of visits over time, stacked by filter, in 7-day bins.

Create a histogram of the airmass of each visit, by filter.

In [23]:

fig = plt.figure(figsize=(8, 4))
for f, filt in enumerate(filter_names):
    plt.hist(df.query("band == @filt")['airmass'],
             bins=40, histtype='step', log=True,
             ls=filter_linestyles[filt],
             color=filter_colors_list[f], label=filt)
plt.legend(loc='best')
plt.xlabel("Airmass")
plt.ylabel("Number of visits")
plt.show()

Figure 2: The number of visits in bins of airmass, by filter.

Create a histogram of the measured seeing of each visit, by filter.

In [24]:

fig = plt.figure(figsize=(8, 4))
for f, filt in enumerate(filter_names):
    plt.hist(df.query("band == @filt")['seeingFwhmEff'],
             bins=40, histtype='step', log=True,
             ls=filter_linestyles[filt],
             color=filter_colors_list[f], label=filt)
plt.legend(loc='best')
plt.xlabel("Seeing (FWHM of the PSF; arcsec)")
plt.ylabel("Number of visits")
plt.show()

Figure 3: The number of visits in bins of seeing, by filter.

Create a histogram of the $5\sigma$ depth (for point sources) of each visit, by filter.

In [25]:

fig = plt.figure(figsize=(8, 4))
for f, filt in enumerate(filter_names):
    plt.hist(df.query("band == @filt")['fiveSigmaDepth'],
             bins=40, histtype='step', log=True,
             ls=filter_linestyles[filt],
             color=filter_colors_list[f], label=filt)
plt.legend(loc='best')
plt.xlabel("5-Sigma Depth (mag)")
plt.ylabel("Number of visits")
plt.show()

Figure 4: The number of visits in bins of 5$\sigma$ depth for point sources, by filter.

4.2. Summary statistics¶

Recreate parts of the tables on the Science Validation survey summary webpage. This section uses code from the LSSTCam summary notebook in the Sims SV Survey repo.

Small field survey areas.

Display a table of the number of visits by band for each of the small field survey areas with at least 50 visits total.

In [26]:

query = "observation_reason == 'field_survey_science'"
df_sfs = df.query(query).groupby(['target_name', 'band']).agg({'seq_num': 'count'})
df_sfs.rename({'seq_num': 'count'}, axis=1, inplace=True)
df_sfs = df_sfs.reset_index('band').pivot(columns=["band"]).droplevel(0, axis=1)
df_sfs = df_sfs[['u', 'g', 'r', 'i', 'z', 'y']]
df_sfs['all'] = df_sfs.sum(axis=1)
df_sfs = df_sfs.query("all > 50").sort_values('all')
table = tabulate(pd.DataFrame(df_sfs.round(0)), headers='keys')
table = table.replace('nan', '  0')
print('Number of visits, for small fields with at least 50 visits total.')
print(table)
del query, df_sfs, table

Number of visits, for small fields with at least 50 visits total.
target_name         u    g    r    i    z    y    all
----------------  ---  ---  ---  ---  ---  ---  -----
Rubin_SV_216_-17    0   28   30   63    0    0    121
Rubin_SV_280_-48   30   30   29   30   29    0    148
ELAIS_S1           11   30   30   30   35   30    166
Rubin_SV_320_-15    0   24   17   99   83   34    257
New_Horizons       36   48   70  108   74   23    359
Rubin_SV_212_-7     0  139  234  123    0    0    496
Prawn             196  160  142   93   30    0    621
COSMOS             96   82  166  139  110   66    659
Trifid-Lagoon     235  196  122  115    0    0    668
M49               234  280  378  213    0    0   1105
Rubin_SV_225_-40  343  559  480  470  330  164   2346

Display a table of the number of visits, median seeing (FWHM), mean airmass, and timespan (time between first and last visit) for each of the small field survey areas with at least 50 visits total.

In [27]:

query = "observation_reason == 'field_survey_science'"
df_sfs = df.query(query).groupby(['target_name']).agg({'seq_num': 'count',
                                                       'seeingFwhmEff': 'median',
                                                       'airmass': 'mean',
                                                       'exp_midpt_mjd': np.ptp})
df_sfs.rename({'seq_num': 'nvisits',
               'seeingFwhmEff': 'median fwhm (arcsec)',
               'airmass': 'mean airmass',
               'exp_midpt_mjd': 'timespan (days)'}, axis=1, inplace=True)
df_sfs = df_sfs.query("nvisits > 50").sort_values('nvisits')
df_sfs['timespan (days)'] = df_sfs['timespan (days)'].astype(int) + 1
df_sfs.round(2)
table = tabulate(pd.DataFrame(df_sfs.round(2)), headers='keys')
table = table.replace('nan', '  0')
print(table)
del query, df_sfs, table

target_name         nvisits    median fwhm (arcsec)    mean airmass    timespan (days)
----------------  ---------  ----------------------  --------------  -----------------
Rubin_SV_216_-17        121                    1.39            1.09                  2
Rubin_SV_280_-48        148                    1.71            1.62                  1
ELAIS_S1                166                    1.59            1.03                 17
Rubin_SV_320_-15        257                    1.33            1.08                  6
New_Horizons            359                    1.17            1.1                  62
Rubin_SV_212_-7         496                    1.33            1.26                  7
Prawn                   621                    1.55            1.2                  79
COSMOS                  659                    1.38            1.41                 15
Trifid-Lagoon           668                    1.28            1.15                 10
M49                    1105                    1.44            1.34                 10
Rubin_SV_225_-40       2346                    1.42            1.17                334

Deep drilling fields.

Make similar versions of the two tables above, but for the DDFs.

First, create a new column that contains only the DDF name, for visits of a DDF (and is a null string otherwise).

In [28]:

df['target_name_ddf'] = df['target_name']
df.loc[~df['target_name_ddf'].str.contains("ddf"), 'target_name_ddf'] = ''
df['target_name_ddf'] = df['target_name_ddf'].str.replace(', lowdust', '', regex=False)
df['target_name_ddf'] = df['target_name_ddf'].str.replace('lowdust, ', '', regex=False)

For visits of DDF with >50 visits, display tables similar to the above.

In [29]:

query = "target_name_ddf.str.contains('ddf')"
df_ddf = df.query(query).groupby(['target_name_ddf', 'band']).agg({'seq_num': 'count'})
df_ddf.rename({'seq_num': 'count'}, axis=1, inplace=True)
df_ddf = df_ddf.reset_index('band').pivot(columns=["band"]).droplevel(0, axis=1)
df_ddf = df_ddf[['u', 'g', 'r', 'i', 'z', 'y']]
df_ddf['all'] = df_ddf.sum(axis=1)
df_ddf = df_ddf.query("all > 50").sort_values('all')
table = tabulate(pd.DataFrame(df_ddf.round(0)), headers='keys')
table = table.replace('nan', '  0')
print(table)
del query, df_ddf, table

target_name_ddf      u    g    r    i    z    y    all
-----------------  ---  ---  ---  ---  ---  ---  -----
ddf_xmm_lss         71   21   18   54   26   24    214
ddf_ecdfs           47  101   85  140   84   33    490
ddf_edfs_b          21  153  151  215  156   20    716
ddf_edfs_a          31  151  159  218  152   17    728
ddf_elaiss1         82  165  136  229  154   74    840
ddf_cosmos          88  315  310  556  301   63   1633

In [30]:

query = "target_name_ddf.str.contains('ddf')"
df_ddf = df.query(query).groupby(['target_name_ddf']).agg({'seq_num': 'count',
                                                           'seeingFwhmEff': 'median',
                                                           'airmass': 'mean',
                                                           'exp_midpt_mjd': np.ptp})
df_ddf.rename({'seq_num': 'nvisits',
               'seeingFwhmEff': 'median fwhm (arcsec)',
               'airmass': 'mean airmass',
               'exp_midpt_mjd': 'timespan (days)'}, axis=1, inplace=True)
df_ddf = df_ddf.sort_values('nvisits')
df_ddf['timespan (days)'] = df_ddf['timespan (days)'].astype(int) + 1
df_ddf.round(2)
df_ddf = df_ddf.query("nvisits > 50").sort_values('nvisits')
table = tabulate(pd.DataFrame(df_ddf.round(2)), headers='keys')
table = table.replace('nan', '  0')
print(table)
del query, df_ddf, table

target_name_ddf      nvisits    median fwhm (arcsec)    mean airmass    timespan (days)
-----------------  ---------  ----------------------  --------------  -----------------
ddf_xmm_lss              214                    1.31            1.47                193
ddf_ecdfs                490                    1.39            1.48                240
ddf_edfs_b               716                    1.2             1.38                240
ddf_edfs_a               728                    1.21            1.41                240
ddf_elaiss1              840                    1.49            1.43                213
ddf_cosmos              1633                    1.11            1.31                174

5. MAF sky maps¶

The commissioning visits database file has been generated using the same format and schema as the Operations Simulations (opsim) databases, and so can be read by the MAF (Metric Analysis Frameworks) package from the rubin_sim package. This section follows the Jupyter Notebook tutorial for visualizing the survey footprint in the rubin_sim_notebook repository.

Set the path to the folder containing rubin_sim data in the RSP at data.lsst.cloud.

In [31]:

os.environ['RUBIN_SIM_DATA_DIR'] = '/rubin/rubin_sim_data'

Define the "opsim" filename and the "run name" -- in this case, the name of the commissioning visits database.

In [32]:

opsim_fname = db_filename
temp = db_filename.split('/')[-1]
run_name = temp.split('.')[0]
print(run_name)

prelsst_20260607_visits

Define the metric to be plotted, in this case, Nvisits: the number of visits. Define the size (nside) of the healpix to use for the map. Do not define any constraints, and include all visits.

In [33]:

metric = maf.metrics.CountMetric(col='observationStartMJD',
                                 metric_name='Nvisits')
nside = 64
slicer = maf.slicers.HealpixSlicer(nside=nside)
constraint = None

Healpix slicer using NSIDE=64, approximate resolution 54.967783 arcminutes

Bundle together the components of the metric.

In [34]:

bundle = maf.MetricBundle(metric, slicer, constraint, run_name=run_name)

Define the bundle group to plot.

In [35]:

group = maf.MetricBundleGroup({'nvisits': bundle},
                              opsim_fname, out_dir=output_path)

Calculate the metric. The following step will generate the files starting with prelsst_20260607 in the output_path defined in Section 1.2.

In [36]:

group.run_all()

Show the plots for this metric.

In [37]:

plot_dict = {'color_min': 10, 'color_max': 100,
             'x_min': -2, 'x_max': 100, 'bins': 50, 'extend': 'both'}
bundle.set_plot_dict(plot_dict)
_ = bundle.plot()

Figure 5: Top, the sky map of the number of visits obtained during commissioning. Bottom, the sky area binned by number of visits, showing that most of the commissioning area was shallow.

Read more about they sky coverage for the SV wide-area survey on the Science Validation survey summary webpage .

AOS testing visits.

In the top panel of Figure 5, stripes of visits with constant declination stand out in the sky distribution. The cause of this non-LSST-like survey pattern is in their observation reason: "fbs_driven_aos_stability_test". FBS stands for "feature-based scheduler" and AOS for "active optics system". These visits were obtained by the FBS for the purpose of testing the AOS, but are still anticipated to be, potentially, scientifically useful.

Visualize only the FBS AOS testing visits.

In [38]:

constraint = "observation_reason like \'%aos%\'"
bundle = maf.MetricBundle(metric, slicer, constraint, run_name=run_name)
group = maf.MetricBundleGroup({'nvisits': bundle},
                              opsim_fname, out_dir=output_path)
group.run_all()

In [39]:

ph = maf.PlotHandler(savefig=False, fig_format='png', thumbnail=False, dpi=270)
ph.set_metric_bundles([bundle])
ph.plot(plot_func=maf.plots.HealpixSkyMap(),
        plot_dicts={'color_min': 20, 'color_max': 100, 'figsize': (6, 4),
                    'labelsize': 'x-large', 'fontsize': 'x-large', 'extend': 'max',
                    'title': 'AOS Testing LSSTCam Visits'})
plt.show()

Figure 6: The FBS AOS testing visits.

DDF visits.

Create a sky map for only the DDFs.

In [40]:

constraint = "observation_reason like \'%ddf%\'"
bundle = maf.MetricBundle(metric, slicer, constraint, run_name=run_name)
group = maf.MetricBundleGroup({'nvisits': bundle},
                              opsim_fname, out_dir=output_path)
group.run_all()

In [41]:

ph = maf.PlotHandler(savefig=False, fig_format='png', thumbnail=False, dpi=270)
ph.set_metric_bundles([bundle])
ph.plot(plot_func=maf.plots.HealpixSkyMap(),
        plot_dicts={'color_min': 20, 'color_max': 100, 'figsize': (6, 4),
                    'labelsize': 'x-large', 'fontsize': 'x-large', 'extend': 'max',
                    'title': 'Deep Drilling Field LSSTCam Visits'})
plt.figtext(0.40, 0.62, 'XMM LSS', fontsize='large', fontweight='bold', color='white')
plt.figtext(0.36, 0.50, 'ECDFS', fontsize='large', fontweight='bold', color='white')
plt.figtext(0.36, 0.33, 'EDFS', fontsize='large', fontweight='bold', color='white')
plt.figtext(0.47, 0.43, 'ELAIS-S1', fontsize='large', fontweight='bold', color='white')
plt.figtext(0.05, 0.65, 'COSMOS', fontsize='large', fontweight='bold', color='white')
plt.show()

Figure 7: The locations of the five Deep Drilling Fields (DDFs), with their names labeled.

6. Might my target be covered?¶

Define two sets of coordinates for two hypothetical targets. The first has been covered by LSSTCam visits and the second has not.

In [42]:

target1_ra, target1_dec = 330.0, -10.0
target2_ra, target2_dec = 330.0, +10.0

To show the location of the two targets on the sky on top of a 2D density plot of the number of visits, instead of using the MAF metric to plot the sky map (as above), use the healpy and skyproj packages.

Use the healpy package to show that the area of one 19-sided HEALPix is similar to the LSSTCam FOV of 9.6 square degrees.

In [43]:

print('One 19-sided HEALPix is ', np.round(hp.nside2pixarea(19, degrees=True), 2),
      ' square degrees.')
print('It takes ', int(41253.0 / hp.nside2pixarea(19, degrees=True)),
      ' 19-sided HEALPix to cover the full sky.')

One 19-sided HEALPix is  9.52  square degrees.
It takes  4332  19-sided HEALPix to cover the full sky.

Create the 2D distribution of LSSTCam visits and mark the locations of the two targets.

In [44]:

fig, ax = plt.subplots(figsize=(8, 6))
sp = skyproj.McBrydeSkyproj(ax=ax)
vras = np.asarray(df['fieldRA'], dtype='float')
vdecs = np.asarray(df['fieldDec'], dtype='float')
sp.draw_hpxbin(vras, vdecs, nside=19, alpha=1, cmap='Greys', vmin=-10)
sp.ax.plot(target1_ra, target1_dec, 'o', ms=10, mec='darkorange', color='None',
           mew=2, label='target 1')
sp.ax.plot(target2_ra, target2_dec, 's', ms=10, mec='darkgreen', color='None',
           mew=2, label='target 2')
sp.ax.set_xlabel("Right Ascension", fontsize=14)
sp.ax.set_ylabel("Declination", fontsize=14)
plt.legend(bbox_to_anchor=(1.05, 1), loc='upper left')
plt.show()

Figure 8: A 2D histogram illustrating the distribution of visits on the sky (for all filters, combined), with the two hypothetical targets marked.

Calculate the 2D sky separations of all visits from the two targets, and the subset to only visits within a sky distance of 1.75 degrees, the approximate radius of the LSSTCam field of view. Print the number of visits that might overlap the two targets.

In [45]:

coords_target1 = SkyCoord(target1_ra, target1_dec, unit="deg")
coords_target2 = SkyCoord(target2_ra, target2_dec, unit="deg")
coords_df = SkyCoord(ra=df['fieldRA'].values * u.deg,
                     dec=df['fieldDec'].values * u.deg, frame='icrs')
df['sep_t1'] = coords_df.separation(coords_target1).degree
df['sep_t2'] = coords_df.separation(coords_target2).degree

df_target1 = df.query('sep_t1 < 1.75')
df_target2 = df.query('sep_t2 < 1.75')

print('Number of visits that potentially overlap')
print('target 1: ', len(df_target1))
print('target 2: ', len(df_target2))

Number of visits that potentially overlap
target 1:  72
target 2:  0

As expected, target 1 has many potentially overlapping visits within a radius of 1.75 degrees, and target 2 has none. Keep in mind that the LSSTCam FOV is not round and not exactly 1.75 degrees, and that there are gaps between the chips (detectors), so the number of visits that actually include target 1 might be different (but this estimate will be close).

6.1. DP1 fields¶

Visualize the overlap of the Data Preview 1 (DP1) fields, which were observed with the LSST Commissioning Camera (LSSTComCam), with these LSSTCam visits.

In [46]:

dp1_field_names = ['47 Tuc globular cluster',
                   'Low Ecliptic Latitude Field',
                   'Fornax Dwarf Spheroidal Galaxy',
                   'Extended Chandra Deep Field South',
                   'Euclid Deep Field South',
                   'Low Galactic Latitude Field',
                   'Seagull Nebula']
dp1_field_ras = [6.02, 37.86, 40.00, 53.13, 59.10, 95.00, 106.23]
dp1_field_decs = [-72.08, 6.98, -34.45, -28.10, -48.73, -25.00, -10.51]
dp1_field_symbols = ['o', 's', 'p', '*', '^', 'v', 'x']
dp1_field_symsizes = [6, 6, 6, 8, 6, 6, 8]
dp1_field_colors = ['red', 'lightseagreen', 'darkviolet',
                    'magenta', 'yellow', 'lime', 'darkorange']

In [47]:

fig, ax = plt.subplots(figsize=(8, 6))
sp = skyproj.McBrydeSkyproj(ax=ax)
vras = np.asarray(df['fieldRA'], dtype='float')
vdecs = np.asarray(df['fieldDec'], dtype='float')
sp.draw_hpxbin(vras, vdecs, nside=19, alpha=1, cmap='Greys', vmin=-10)
for i in range(len(dp1_field_names)):
    sp.ax.plot(dp1_field_ras[i], dp1_field_decs[i], dp1_field_symbols[i], ms=dp1_field_symsizes[i],
               color=dp1_field_colors[i], label=dp1_field_names[i])
sp.ax.set_xlabel("Right Ascension", fontsize=14)
sp.ax.set_ylabel("Declination", fontsize=14)
plt.legend(bbox_to_anchor=(1.05, 1), loc='upper left')
plt.show()

Figure 9: Same as Figure 8, but with the seven DP1 LSSTComCam fields marked.

7. Exercises for the learner¶

Of the columns in the commissioning visits database, only nine were mentioned as key columns in Section 2.1. This did not include the column cloud_extinction.

Review the description of the cloud_extinction on the Science Validation survey summary webpage :

The visit database "also includes an estimate of the mean cloud extinction in the images. These are estimates based on the measured zeropoints for the images, compared to the expected zeropoint for an image in that bandpass at that airmass. A potential issue here is that visits with very heavy cloud extinction (or other problem with the quicklook image processing occuring immediately after image acquisition) may not succeed in measuring a zeropoint for the image at all, and thus no estimate for the cloud extinction will be possible either."

As in Section 4.1, create a histogram of the cloud_extinction values. Use the function defined in Section 4.2 to calculate the mean cloud extinction in magnitudes for a subset of the visits.

In [ ]: