201.1. Alert packets#
201.1. Alert packets¶
For the Rubin Science Platform at data.lsst.cloud.
Data Release: Prompt Products
Container Size: Large
LSST Science Pipelines version: r29.2.0
Last verified to run: 2026-06-17
Repository: github.com/lsst/tutorial-notebooks
DOI: 10.11578/rubin/dc.20250909.20
Learning objective: Understand the contents of the alert packets (in Avro format).
LSST data products: Alert packets.
Packages: lsst.rsp.RSPClient, lsst.rsp.get_service_url, fastavro.
Credit: Originally developed by Rubin Data Management team members. Please consider acknowledging them if this notebook is used for the preparation of journal articles, software releases, or other notebooks.
Get Support: Everyone is encouraged to ask questions or raise issues in the Support Category of the Rubin Community Forum. Rubin staff will respond to all questions posted there.
1. Introduction¶
An alert packet is a file that contains measurements for sources detected in difference images. Alerts are streamed to brokers within a few minutes of the acquisition of every new image to enable real-time analysis and spectroscopic follow-up, and brokers are the recommended way to do time-domain science with alerts. Alerts are world public and have no proprietary period.
Alert packets are generated by the Alert Production pipeline, which runs Difference Image Analysis (DIA) for each new exposure, provided a template image exists for that sky location.
The template image is subtracted from the new processed visit image (the 'science' image) to produce a difference image.
Source detection is run on the difference image, and every source (positive or negative) detected with signal-to-noise ratio $\geq 5$ becomes a diaSource.
Every diaSource is associated with either a diaObject (static sky coordinates) or an ssObject (Solar System object).
One alert is created per diaSource.
What is included in an alert packet?
Alert packets include the following information, and much more. See the Prompt Products documentation for a more complete list.
- The triggering
diaSourcerecord, which includes photometric and astrometric measurements and metadata. - The associated
diaObjectormpc_orbitrecord, which includes derived properties for the object. - Previous
diaSourcerecords associated with thediaObjectfrom the last 12 months, maximum. - Image cutouts (stamps) from the science, template, and difference images, for the triggering
diaSource.
How do alerts relate to Rubin's other Prompt Products?
The data in the alert packets will also be stored in a queryable database, the Prompt Products Database (PPDB), which will update on a ~24-hour timescale and will be available via the Rubin Science Platform (RSP).
The contents of the PPDB are public but RSP access is not (unlike broker access).
The processed visit images and difference images become available via the RSP after an 80-hour embargo, and are subject to the two-year proprietary period. (See the Rubin Data Policy.)
Additional resources:
- Alert Production database schema.
- Prompt Products documentation.
- Learn more about the LSST alert brokers.
This tutorial focuses on the details of the packets' schema and data contents, and how packets for static-sky and moving objects differ.
Related tutorials: The 100-level tutorial on the alert retrieval service focuses on how the service works, and options for packet retrieval formats (e.g., FITS, JSON, schema-only, cutouts-only).
1.1. Import packages¶
Import the fastavro package for reading alerts from the alert retrieval service in the default Avro format.
Import the RSPClient and get_service_url in order to access the alert retrieval service.
Also import a range of other plotting and analyis packages.
import fastavro
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import io
from astropy.io import fits
from astropy.wcs import WCS
from astropy.coordinates import SkyCoord
from astropy.nddata import CCDData
from astropy.visualization import (MinMaxInterval, ZScaleInterval,
AsinhStretch, LinearStretch,
ImageNormalize)
from astropy.wcs.utils import skycoord_to_pixel
from lsst.rsp import RSPClient, get_service_url
from lsst.utils.plotting import (get_multiband_plot_colors,
get_multiband_plot_symbols)
1.2. Define parameters¶
Establish the connection to the alert retrieval service by getting the url for the service and instantiating the RSPClient.
url = get_service_url("alerts", "prompt")
client = RSPClient("")
Use the recommended colorblind-friendly palette to represent Rubin filters.
filter_colors = get_multiband_plot_colors()
filter_names = list(filter_colors.keys())
filter_symbols = get_multiband_plot_symbols()
2. Avro alert packets¶
The "alert ID" is the same as the diaSourceId of the triggering difference-image detection.
The alert brokers were used to browse LSST alerts and find example alerts for two different types of time-domain objects.
A static-sky source alert:
The detection of Type Ia supernova (SNIa) SN 2026ivl in an LSST image on MJD $= 61152.083609$.
snia_alert_id $= 170301167371288762$
A solar system source alert:
The detection of main-belt asteroid (MBA) 2015 BC557 / 713454 (designation K15Bt7C) in an LSST image on MJD $= 61098.241373$.
mba_alert_id $= 170063717265309728$
Establish the alert_id values to use for the SNIa and MBA.
snia_alert_id = "170301167371288762"
mba_alert_id = "170063717265309728"
2.1. Retrieve the packets¶
An Avro Object Container File (OCF) is the default response format from the alert retrieval service. Other formats (JSON, FITS) are possible, as demonstrated in the 100-level tutorial on the alert retrieval service.
Pass the url to the client along with the alert_id.
Use the await command to pause execution of the cell's code until a response is received, then show the response.
Retrieve the alert for the snia_alert_id.
snia_response = await client.get(url, params={"ID": snia_alert_id})
snia_response?
Type: Response String form: <Response [200 OK]> File: /opt/lsst/software/stack/conda/envs/lsst-scipipe-10.1.0-exact/lib/python3.12/site-packages/httpx/__init__.py Docstring: <no docstring>
If an incorrect alert ID is used the client.get call will still execute, but a "404 Not Found" error message will appear next to "String form:" in the above output and in the output of the following cell instead of "200 OK".
snia_response.raise_for_status()
<Response [200 OK]>
Also retrieve the alert for the mba_alert_id.
mba_response = await client.get(url, params={"ID": mba_alert_id})
mba_response.raise_for_status()
<Response [200 OK]>
Show that the Avro OCF container file for the SNIa is larger than that for the MBA. The bulk of the packet size is the cutout images, and different objects will have different packet sizes mainly due to having different detection histories.
print(f"SNIa Avro OCF size: {len(snia_response.content):,} bytes")
print(f"MBA Avro OCF size: {len(mba_response.content):,} bytes")
SNIa Avro OCF size: 177,066 bytes MBA Avro OCF size: 135,154 bytes
Read the Avro packet with fastavro.reader.
snia_avro = fastavro.reader(io.BytesIO(snia_response.content))
mba_avro = fastavro.reader(io.BytesIO(mba_response.content))
2.2. Packet schema¶
The fastavro.reader used above reads the embedded schema automatically.
Extract the schema, and show that the keys and fields are the same for each (but for triggering diaSources that are associated with a diaObject instead of a ssObject, the fields ssSource and mpc_orbits won't be populated).
snia_schema = snia_avro.writer_schema
print("SNIa packet")
print(f" keys: {snia_schema.keys()}")
print(f" fields: {[f['name'] for f in snia_schema['fields']]}")
print()
mba_schema = mba_avro.writer_schema
print("MBA packet")
print(f" keys: {mba_schema.keys()}")
print(f" fields: {[f['name'] for f in mba_schema['fields']]}")
SNIa packet keys: dict_keys(['type', 'name', 'fields']) fields: ['diaSourceId', 'observation_reason', 'target_name', 'diaSource', 'prvDiaSources', 'prvDiaForcedSources', 'diaObject', 'ssSource', 'mpc_orbits', 'cutoutDifference', 'cutoutScience', 'cutoutTemplate'] MBA packet keys: dict_keys(['type', 'name', 'fields']) fields: ['diaSourceId', 'observation_reason', 'target_name', 'diaSource', 'prvDiaSources', 'prvDiaForcedSources', 'diaObject', 'ssSource', 'mpc_orbits', 'cutoutDifference', 'cutoutScience', 'cutoutTemplate']
The Alert Production Database table schema provides a browsable interface to the alert schema.
Option to print the entire schema.
# snia_schema
# mba_schema
2.3. Packet records¶
Extract the packet record as the first (and only) row of a list generated from the Avro OCF.
snia_records = list(snia_avro)
snia_record = snia_records[0]
mba_records = list(mba_avro)
mba_record = mba_records[0]
Show that the record's keys are the same as the packet's fields (this is true also for mba_record).
snia_record.keys()
dict_keys(['diaSourceId', 'observation_reason', 'target_name', 'diaSource', 'prvDiaSources', 'prvDiaForcedSources', 'diaObject', 'ssSource', 'mpc_orbits', 'cutoutDifference', 'cutoutScience', 'cutoutTemplate'])
The first three are:
diaSourceId: ID of the triggeringdiaSource.observation_reason: Mode of the Rubin scheduler when the observation was obtained.target_name: Name of the field targeted by the observation, e.g., a deep drilling field (DDF), the low-dust wide-fast-deep region (lowdust).
Print the values of the first three keys in the record for each object.
print("SNIa record")
print(f" diaSourceId: {snia_record['diaSourceId']}")
print(f" observation_reason: {snia_record['observation_reason']}")
print(f" target_name: {snia_record['target_name']}")
print()
print("MBA record")
print(f" diaSourceId: {mba_record['diaSourceId']}")
print(f" observation_reason: {mba_record['observation_reason']}")
print(f" target_name: {mba_record['target_name']}")
SNIa record diaSourceId: 170301167371288762 observation_reason: ddf_cosmos target_name: lowdust, ddf_cosmos MBA record diaSourceId: 170063717265309728 observation_reason: alert_m49 target_name: field_m49, lowdust
2.4. Triggering diaSource¶
The diaSource field contains the full record of the triggering difference image detection.
Extract the diaSource for each of the objects.
snia_diaSource = snia_record['diaSource']
mba_diaSource = mba_record['diaSource']
Option to print the keys of diaSource, which are the same for all objects.
# snia_diaSource.keys()
Key columns include:
band: LSST filter of the observation, one of: $ugrizy$.midpointMjdTai: The MJD at the midpoint of the exposure.psfFlux,psfFluxErr: The PSF flux and its error, measured on the difference image, in nJy.scienceFlux,scienceFluxErr: The PSF flux and its error, measured on the science image, in nJy.
Warning: while the science flux is useful for variable stars, for supernovae it is typically contaminated by host galaxy light and is inappropriate for use in a lightcurve.
Print the values in the key columns.
print("SNIa diaSource")
print(f" diaSource['band']: {snia_diaSource['band']}")
print(f" diaSource['midpointMjdTai']: {snia_diaSource['midpointMjdTai']}")
print(f" diaSource['psfFlux']: {snia_diaSource['psfFlux']}")
print(f" diaSource['scienceFlux']: {snia_diaSource['scienceFlux']}")
print()
print("MBA diaSource")
print(f" diaSource['band']: {mba_diaSource['band']}")
print(f" diaSource['midpointMjdTai']: {mba_diaSource['midpointMjdTai']}")
print(f" diaSource['psfFlux']: {mba_diaSource['psfFlux']}")
print(f" diaSource['scienceFlux']: {mba_diaSource['scienceFlux']}")
SNIa diaSource diaSource['band']: r diaSource['midpointMjdTai']: 61152.0836086934 diaSource['psfFlux']: 90499.59375 diaSource['scienceFlux']: 101317.328125 MBA diaSource diaSource['band']: r diaSource['midpointMjdTai']: 61098.24137255241 diaSource['psfFlux']: 7202.36572265625 diaSource['scienceFlux']: 7502.54541015625
2.5. Associated object¶
The diaObject and mpc_orbits/ssSource fields contain information about the associated static-sky time-domain object or the moving object, respectively.
Demonstrate that the snia_record is associated with a static-sky time-domain object, and the mba_record with a moving object, as expected.
if (snia_record['diaObject'] is not None) & (snia_record['mpc_orbits'] is None):
print('The snia_record is associated with a static-sky object.')
if (mba_record['diaObject'] is None) & (mba_record['mpc_orbits'] is not None):
print('The mba_record is associated with a moving object.')
The snia_record is associated with a static-sky object. The mba_record is associated with a moving object.
2.5.1. DIA object (static-sky)¶
The diaObject contains aggregate information about the static-sky time-domain object, based on all the previous detections plus the triggering detection.
For the SNIa, extract the diaObject record.
snia_diaObject = snia_record['diaObject']
Option to print the keys of snia_diaObject.
# snia_diaObject.keys()
Key columns include:
diaObjectId: Unique ID of thediaObject.ra,dec: Sky coordinates for thediaObject, in degrees.nDiaSources: Number of detections, total (over all filters), including the triggering detection.
Lightcurve characterization parameters such as the minimum, maximum, mean, and standard deviation in the flux values, plus the number of measurements, for the PSF flux (detection in the difference image), PSF forced flux (forced on the difference image), and the science flux (PSF photometry on the science image), for each filter, are also available. Flux values are in nJy.
Print a few of the key column values.
columns = ["diaObjectId", "ra", "dec", "nDiaSources", "r_psfFluxMean"]
for col in columns:
print(f"{col: <22} {snia_diaObject[col]}")
diaObjectId 170235219334922248 ra 148.4742795954378 dec 1.718237259899008 nDiaSources 34 r_psfFluxMean 25149.900390625
2.5.2. MPC orbit (moving objects)¶
The mpc_orbits contains orbital elements and related information of known Solar System objects, from the Minor Planet Center (MPC) database.
For the MBA, extract the mpc_orbits record.
mba_mpcOrbit = mba_record['mpc_orbits']
Option to print the keys of mba_mpcOrbit.
# mba_mpcOrbit.keys()
Key columns include:
designation: The MPC designation (name) of the associated moving object.q: MPC pericenter distance in au.e: MPC eccentricity.i: MPC inclination in degrees.h: MPC H magnitude.
Print a few of the key column values.
columns = ["designation", "q", "e", "i", "h"]
for col in columns:
print(f"{col: <22} {mba_mpcOrbit[col]}")
designation 2015 BC557 q 2.12651755819726 e 0.090786802364178 i 6.9786091591327 h 18.462
2.5.3. SS source (moving objects)¶
The ssSource contains computed quantities at the time of the triggering alert.
For the MBA, extract the ssSource record.
mba_ssSource = mba_record['ssSource']
Option to print the keys of ssSource.
# mba_ssSource.keys()
Key columns include:
ssObjectid: Unique ID in thessObjecttable.phaseAngle: Phase angle between the Sun, object, and observer, in degrees.eclBeta,eclLambda: The ecliptic latitude and longitude, in degrees.elongation: Solar elongation, in degrees.ephVmag: Predicted magnitude in V band.
The heliocentric and topocentric positions and velocities (helio_ and topo_ x, y, z, vx, vy, vz) are also included, among other parameters.
Print a few of the key column values.
columns = ["ssObjectId", "phaseAngle", "eclBeta", "eclLambda", "elongation", "ephVmag"]
for col in columns:
print(f"{col: <12} {mba_ssSource[col]}")
ssObjectId 21164728071239491 phaseAngle 10.14724349975586 eclBeta 9.892962667410773 eclLambda 181.76689955148512 elongation 154.61837768554688 ephVmag 0.0
2.6. Lightcurves¶
If they are not None, the prvDiaSources and prvDiaForcedSources fields contain the history of past detections and forced PSF photometry measurements, respectively.
Currently, past detections and forced photometry is not available for moving objects, only for static-sky objects.
Show the number of previous sources and forced sources for the two objects.
print("SNIa record")
if snia_record['prvDiaSources'] is not None:
print(f" Number of previous detections: {len(snia_record['prvDiaSources'])}")
else:
print(" No previous detections")
if snia_record['prvDiaForcedSources'] is not None:
print(f" Number of forced photometry measurements: {len(snia_record['prvDiaForcedSources'])}")
else:
print(" No forced photometry measurements")
print()
print("MBA record")
if mba_record['prvDiaSources'] is not None:
print(f" Number of previous detections: {len(mba_record['prvDiaSources'])}")
else:
print(" No previous detections")
if mba_record['prvDiaForcedSources'] is not None:
print(f" Number of forced photometry measurements: {len(mba_record['prvDiaForcedSources'])}")
else:
print(" No forced photometry measurements")
SNIa record Number of previous detections: 33 Number of forced photometry measurements: 39 MBA record No previous detections No forced photometry measurements
To plot the lightcurve for a static-sky object, start by extracting the current and past photometry from the records.
Get the band, MJD, and PSF difference-image flux for the triggering detection.
alert_band = str(snia_diaSource['band'])
alert_mjd = float(snia_diaSource['midpointMjdTai'])
alert_flux = float(snia_diaSource['psfFlux'])
Create pandas dataframes of the previous detections.
df = pd.DataFrame(snia_record['diaSource'], index=[0])
df_prv = pd.DataFrame(snia_record['prvDiaSources'])
df_lc = pd.concat([df, df_prv], ignore_index=True)
del df, df_prv
Create pandas dataframes of the forced PSF photometry on the past images, which includes the image with the triggering detection.
df = pd.DataFrame(snia_record['diaSource'], index=[0])
df_prv = pd.DataFrame(snia_record['prvDiaForcedSources'])
df_flc = pd.concat([df, df_prv], ignore_index=True)
del df, df_prv
Plot separately the lightcurves using the detections (plus the triggering detection), and using the forced photometry.
The fluxes in the g, r, and i bands overlap, so impose a flux offset by filter when plotting.
offsets = [0, 20000, 40000, 60000, 0, 0]
fig, ax = plt.subplots(1, 2, figsize=(9, 3))
for f, filt in enumerate(filter_names):
if alert_band == filt:
ax[0].plot(alert_mjd-61000, alert_flux+offsets[f], filter_symbols[filt], ms=7,
mew=2, alpha=1, color=filter_colors[filt])
df_pick_band = df_lc.loc[df_lc['band'] == filt]
if len(df_pick_band) > 0:
ax[0].plot(df_pick_band['midpointMjdTai']-61000, df_pick_band['psfFlux']+offsets[f], filter_symbols[filt],
ms=5, mew=1, alpha=0.7, mec=filter_colors[filt], color='None')
df_pick_band = df_flc.loc[df_flc['band'] == filt]
if len(df_pick_band) > 0:
ax[1].plot(df_pick_band['midpointMjdTai']-61000, df_pick_band['psfFlux']+offsets[f], filter_symbols[filt],
ms=5, mew=1, alpha=0.7, mec=filter_colors[filt], color='None',
label=filt + " +" + str(offsets[f]))
ax[0].set_xlabel('MJD-61000')
ax[1].set_xlabel('MJD-61000')
ax[0].set_ylabel('difference flux [nJy]')
ax[1].set_ylabel('forced difference flux [nJy]')
ax[1].legend(bbox_to_anchor=(1.05, 1), handletextpad=0, loc='upper left')
plt.suptitle('The SNIa lightcurves, with flux offsets for clarity')
plt.tight_layout()
plt.show()
Figure 1: At left, the difference-image detection flux as a function of MJD (open symbols), plus the triggering alert detection (filled symbol). At right, the difference-image forced photometry fluxes as a function of MJD, which includes a forced measurement in the triggering observation. Fluxes have been offset in the g, r, and i filters for clarity, as indicated in the legend.
2.7. Image stamps (cutouts)¶
Small image stamps (cutout images) from the science, template, and difference image are embedded in every alert.
Use the fits package to convert the string-format image for the SNIa science stamp into a Header Data Unit List (HDUL), and print the names of each HDU.
sci_hdul = fits.HDUList.fromstring(snia_record['cutoutScience'])
for hdu in sci_hdul:
print(hdu.name)
PRIMARY UNCERT PSFIMAGE
Extract the primary HDU and display the header. Notice the pixel flux units (BUNIT) is in nJy, and that the WCS is included.
pri_sci_hdu = sci_hdul['PRIMARY']
pri_sci_hdu.header
SIMPLE = T / conforms to FITS standard BITPIX = -32 / array data type NAXIS = 2 / number of array dimensions NAXIS1 = 51 NAXIS2 = 51 EXTEND = T CUTMINX = 2874 CUTMINY = 2784 ROTPA = 17.0390078992157 / Pos angle in deg of focal plane +Y wrt North BUNIT = 'nJy ' WCSAXES = 2 / Number of coordinate axes CRPIX1 = 25.981179621249 / Pixel coordinate of reference point CRPIX2 = 26.339120759041 / Pixel coordinate of reference point PC1_1 = -5.3000516248565E-05 / Coordinate transformation matrix element PC1_2 = 1.6186434587101E-05 / Coordinate transformation matrix element PC2_1 = 1.6276853088302E-05 / Coordinate transformation matrix element PC2_2 = 5.3082632371196E-05 / Coordinate transformation matrix element CDELT1 = 1.0 / Coordinate increment at reference point CDELT2 = 1.0 / Coordinate increment at reference point CRVAL1 = 148.47427912183 / Coordinate value at reference point CRVAL2 = 1.7182357179183 / Coordinate value at reference point LATPOLE = 90.0 / [deg] Native latitude of celestial pole MJDREF = 0.0 / [d] MJD of fiducial time
Get the SNIa stamp's World Coordinate System (WCS). Add in the missing ctype that is needed in order to use the WCS to convert between sky and pixel coordinates (this missing piece of FITS header information will be fixed soon). Use the RA and Dec of the SNIa from its Transient Name Server page, and convert it to pixels.
wcs = WCS(pri_sci_hdu.header)
wcs.wcs.ctype = ["RA---TAN", "DEC--TAN"]
snia_ra = 148.474283
snia_dec = +1.718233
coord = SkyCoord(snia_ra, snia_dec, unit="deg")
pixels = skycoord_to_pixel(coord, wcs, origin=0)
Also define the pixels for a point 4" north of the SNIa.
temp_dec = snia_dec + 4.0/3600.0
temp_coord = SkyCoord(snia_ra, temp_dec, unit="deg")
temp_pixels = skycoord_to_pixel(temp_coord, wcs, origin=0)
Display the image stamps for the SNIa. Several options for the image scaling are provided as commented-out code lines.
stamps = [snia_record['cutoutScience'],
snia_record['cutoutTemplate'],
snia_record['cutoutDifference']]
stamp_names = ['Science', 'Template', 'Difference']
fig, ax = plt.subplots(1, 3, figsize=(9, 3))
for s, (stamp, name) in enumerate(zip(stamps, stamp_names)):
hdul = fits.HDUList.fromstring(stamp)
data = hdul[0].data
image = CCDData(data.astype("float32"), unit='nJy')
# norm = ImageNormalize(image, interval=ZScaleInterval(),
# stretch=AsinhStretch())
# norm = ImageNormalize(image, interval=ZScaleInterval(),
# stretch=LinearStretch())
norm = ImageNormalize(image, interval=MinMaxInterval(),
stretch=LinearStretch())
plt.sca(ax[s])
plt.imshow(image, origin='lower', norm=norm, cmap='gray')
plt.colorbar()
ax[s].set_title(name)
plt.plot(pixels[0], pixels[1], 'o', ms=20, mew=1, color='None', mec='yellow')
plt.plot([pixels[0], temp_pixels[0]], [pixels[1], temp_pixels[1]], color='cyan')
plt.text(temp_pixels[0], temp_pixels[1], 'N', color='cyan', fontsize=12)
ax[s].set_title(name)
plt.suptitle('Image stamps from the SNIa alert')
plt.tight_layout()
plt.show()
Figure 2: Image stamps from the SNIa alert packet. The literature coordinates of the SNIa are marked with a yellow circle, and the north direction indicated with a cyan line.
Display the image stamps for the MBA.
stamps = [mba_record['cutoutScience'],
mba_record['cutoutTemplate'],
mba_record['cutoutDifference']]
fig, ax = plt.subplots(1, 3, figsize=(9, 3))
for s, (stamp, name) in enumerate(zip(stamps, stamp_names)):
hdul = fits.HDUList.fromstring(stamp)
data = hdul[0].data
image = CCDData(data.astype("float32"), unit='nJy')
norm = ImageNormalize(image, interval=ZScaleInterval(),
stretch=LinearStretch())
plt.sca(ax[s])
plt.imshow(image, origin='lower', norm=norm, cmap='gray')
plt.colorbar()
ax[s].set_title(name)
plt.suptitle('Image stamps from the MBA alert')
plt.tight_layout()
plt.show()
Figure 3: Image stamps from the MBA alert packet.